278 lines
8.1 KiB
Markdown
278 lines
8.1 KiB
Markdown
The git-annex P2P protocol is a custom protocol that git-annex uses to
|
|
communicate between peers.
|
|
|
|
There's a common line-based serialization of the protocol, but other
|
|
serializations are also possible. The line-based serialization is spoken
|
|
by [[git-annex-shell]], and by git-annex over tor. There is also
|
|
a translation of this protocol to a [[HTTP_API|p2p_protocol_over_http]].
|
|
|
|
One peer is known as the client, and is the peer that initiates the
|
|
connection and sends commands. The other peer is known as the server, and
|
|
is the peer that the client connects to. It's possible for two connections
|
|
to be run at the same time between the same two peers, in different
|
|
directions.
|
|
|
|
## Errors
|
|
|
|
Either the client or the server may send an error message at any
|
|
time.
|
|
|
|
When the client sends an ERROR, the server will close the connection.
|
|
|
|
If the server sends an ERROR in response to the client's
|
|
request, the connection will remain open, and the client can make
|
|
another request.
|
|
|
|
ERROR this repository is read-only; write access denied
|
|
|
|
## Authentication
|
|
|
|
The protocol generally starts with authentication. However, if
|
|
authentication already occurs on another layer, as is the case with
|
|
git-annex-shell, authentication will be skipped.
|
|
|
|
The client starts by sending an authentication command to the server,
|
|
along with its UUID. The AuthToken is some arbitrary token that has been
|
|
agreed upon beforehand.
|
|
|
|
AUTH UUID AuthToken
|
|
|
|
The server responds with either its own UUID when authentication
|
|
is successful. Or, it can fail the authentication, and close the
|
|
connection.
|
|
|
|
AUTH-SUCCESS UUID
|
|
AUTH-FAILURE
|
|
|
|
Note that authentication does not guarantee that the client is talking to
|
|
who they expect to be talking to. This, and encryption of the connection,
|
|
are handled at a lower level.
|
|
|
|
## Protocol version
|
|
|
|
The default protocol version is 0. The client can choose to
|
|
negotiate a new version with the server. This must come after
|
|
any authentication.
|
|
|
|
The client sends the highest protocol version it supports:
|
|
|
|
VERSION 3
|
|
|
|
The server responds with the highest protocol version it supports
|
|
that is less than or equal to the version the client sent:
|
|
|
|
VERSION 1
|
|
|
|
Now both client and server should use version 1.
|
|
|
|
## Cluster cycle prevention
|
|
|
|
In protocol version 2 and above, immediately after VERSION, the
|
|
client can send an additional message that is used to
|
|
prevent cycles when accessing clusters.
|
|
|
|
BYPASS UUID1 UUID2 ...
|
|
|
|
The UUIDs are cluster gateways to avoid connecting to when
|
|
serving a cluster.
|
|
|
|
The server makes no response to this message.
|
|
|
|
## Binary data
|
|
|
|
The protocol allows raw binary data to be sent. This is done
|
|
using a DATA message. In the line-based serialization, this comes
|
|
on its own line, followed by a newline and the binary data.
|
|
The Len value tells how many bytes of data to read.
|
|
|
|
DATA 3
|
|
foo
|
|
|
|
Note that there is no newline after the binary data; the next protocol
|
|
message will come immediately after it.
|
|
|
|
If the sender finds itself unable to send as many bytes of data as it
|
|
promised (perhaps because a file got truncated while it was being sent),
|
|
its only option is to close the protocol connection.
|
|
|
|
And if the receiver finds itself unable to receive all the data for some
|
|
reason (eg, out of disk space), its only option is to close the protocol
|
|
connection.
|
|
|
|
## Checking if content is present
|
|
|
|
To check if a key is currently present on the server, the client sends:
|
|
|
|
CHECKPRESENT Key
|
|
|
|
The server responds with either SUCCESS or FAILURE.
|
|
|
|
## Locking content
|
|
|
|
To lock content on the server, preventing it from being removed,
|
|
the client sends:
|
|
|
|
LOCKCONTENT Key
|
|
|
|
The server responds with either SUCCESS or FAILURE.
|
|
The former indicates the content is locked.
|
|
|
|
After SUCCESS, the content will remain locked until the
|
|
client sends its next message, which must be:
|
|
|
|
UNLOCKCONTENT Key
|
|
|
|
The server makes no response to that.
|
|
|
|
If the connection is broken before the client sends UNLOCKCONTENT,
|
|
the content will remain locked for at least 10 minutes from when the server
|
|
sent SUCCESS.
|
|
|
|
## Removing content
|
|
|
|
To remove a key's content from the server, the client sends:
|
|
|
|
REMOVE Key
|
|
|
|
The server responds with either SUCCESS or FAILURE.
|
|
|
|
Note that if the content was not present, SUCCESS will be returned.
|
|
|
|
In protocol version 2 and above, the server can optionally reply with
|
|
SUCCESS-PLUS or FAILURE-PLUS. Each has a subsequent list of UUIDs of
|
|
repositories that the content was removed from.
|
|
|
|
## Removing content before a specified time
|
|
|
|
This is only available in protocol version 3 and above.
|
|
|
|
To remove a key's content from the server, but only before a specified time,
|
|
the client sends:
|
|
|
|
REMOVE-BEFORE Timestamp Key
|
|
|
|
The server responds to the message in the same way as to REMOVE.
|
|
|
|
If the server receives the message at a time after the specified timestamp,
|
|
the remove must fail.This is used to avoid removing content after a point
|
|
in time where it is no longer locked in other repostitories.
|
|
|
|
## Getting a timestamp
|
|
|
|
This is only available in protocol version 3 and above.
|
|
|
|
To get the current timestamp from the server, the client sends:
|
|
|
|
GETTIMESTAMP
|
|
|
|
The server responds with TIMESTAMP followed by its current time, as a
|
|
number of seconds. Note that this uses a monotonic clock.
|
|
|
|
## Storing content on the server
|
|
|
|
To store content on the server, the client sends:
|
|
|
|
PUT AssociatedFile Key
|
|
|
|
Here AssociatedFile may be the name of a file in the git
|
|
repository, for information purposes only. Or it can be the
|
|
empty string. It will always have unix directory separators.
|
|
|
|
(Note that in the line-based serialization. AssociatedFile may not contain any
|
|
spaces, since it's not the last token in the line. Use '%' to indicate
|
|
whitespace.)
|
|
|
|
The server may respond with ALREADY-HAVE if it already
|
|
had the content of that key.
|
|
|
|
In protocol version 2 and above, the server can optionally reply with
|
|
ALREADY-HAVE-PLUS. The subsequent list of UUIDs are additional
|
|
UUIDs where the content is stored, in addition to the UUID where
|
|
the client was going to send it.
|
|
|
|
Otherwise, it responds with:
|
|
|
|
PUT-FROM Offset
|
|
|
|
Offset is the number of bytes into the file that the server wants
|
|
the client to start. This allows resuming transfers.
|
|
|
|
The client then sends a DATA message with content of the file from
|
|
the offset to the end of file.
|
|
|
|
In protocol version 1 and above, after the data, the client sends an
|
|
additional message, to indicate if the content of the file has changed
|
|
while it was being sent.
|
|
|
|
INVALID
|
|
VALID
|
|
|
|
If the server successfully receives the data and stores the content,
|
|
it replies with SUCCESS. Otherwise, FAILURE.
|
|
|
|
In protocol version 2 and above, the server can optionally reply with
|
|
SUCCESS-PLUS and a list of UUIDs where the content was stored.
|
|
|
|
## Getting content from the server
|
|
|
|
To get content from the server, the client sends:
|
|
|
|
GET Offset AssociatedFile Key
|
|
|
|
The Offset is the number of bytes into the file that the client wants
|
|
the server to skip, which allows resuming transfers.
|
|
See description of AssociatedFile above.
|
|
|
|
The server then sends a DATA message with the content of the file
|
|
from the offset to end of file.
|
|
|
|
In protocol version 1 and above, after the data, the server sends an additional
|
|
message, to indicate if the content of the file has changed while it
|
|
was being sent.
|
|
|
|
INVALID
|
|
VALID
|
|
|
|
The client replies with SUCCESS or FAILURE.
|
|
|
|
Note that the client responding with SUCCESS does not indicate to the
|
|
server that it has stored the content. It may receive it and throw it away.
|
|
|
|
## Connection to services
|
|
|
|
This is used to connect to services like git-upload-pack and
|
|
git-receive-pack that speak their own protocol.
|
|
|
|
The client sends a message to request the connection.
|
|
Service is the name of the service, eg "git-upload-pack".
|
|
|
|
CONNECT Service
|
|
|
|
Both client and server may now exchange DATA messages in any order,
|
|
encapsulating the service's protocol.
|
|
|
|
When the service exits, the server indicates this by telling the client
|
|
its exit code.
|
|
|
|
CONNECTDONE ExitCode
|
|
|
|
After that, the server closes the connection.
|
|
|
|
## Change notification
|
|
|
|
The client can request to be notified when a ref in
|
|
the git repository on the server changes.
|
|
|
|
NOTIFYCHANGE
|
|
|
|
The server will block until at least
|
|
one of the refs changes, and send a list of changed
|
|
refs.
|
|
|
|
CHANGED ChangedRefs
|
|
|
|
For example:
|
|
|
|
CHANGED refs/heads/master refs/heads/git-annex
|
|
|
|
Some servers may not support this command.
|