From b7aa7445a5d97d6f43fbea26ae2ee1773744ebc9 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Wed, 7 Mar 2018 15:08:17 -0400 Subject: add formal description of the P2P protocol This commit was sponsored by Fernando Jimenez on Patreon. --- doc/design/p2p_protocol.mdwn | 168 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 168 insertions(+) create mode 100644 doc/design/p2p_protocol.mdwn diff --git a/doc/design/p2p_protocol.mdwn b/doc/design/p2p_protocol.mdwn new file mode 100644 index 000000000..23ac4052a --- /dev/null +++ b/doc/design/p2p_protocol.mdwn @@ -0,0 +1,168 @@ +The git-annex P2P protocol is a custom protocol that git-annex uses to +communicate between peers. + +There's a common line-based serialization of the protocol, but other +serializations are also possible. The line-based serialization is spoken +by [[git-annex-shell], and by [[git-annex-remotedaemon]] when serving tor. + +One peer is known as the client, and is the peer that initiates the +connection. The other peer is known as the server, and is the peer that the +client connects to. It's possible for two connections to be run at the same +time between the same two peers, in different directions. + +## Authentication + +The protocol genernally starts with authentication. However, if +authentication already occurs on another layer, as is the case with +git-annex-shell, authentication will be skipped. + +The client starts by sending an authentication to the server, +along with its UUID. The AuthToken is some arbitrary token that has been +agreed upon beforehand. + + AUTH UUID AuthToken + +The server responds with either its own UUID when authentication +is successful. Or, it can fail the authentication, and close the +connection. + + AUTH_SUCCESS UUID + AUTH_FAILURE + +## Errors + +Either the client or the server may send an error message at any +time. + +When the client sends an ERROR, the server will close the connection. + +If the server sends an ERROR in response to the client's +request, the connection will remain open, and the client can make +another request. + + ERROR this repository is read-only; write access denied + +## Binary data + +The protocol allows raw binary data to be sent. This is done +using a DATA message. In the line-based serialization, this comes +on its own line, followed by a newline and the binary data. +The Len value tells how many bytes of data to read. + + DATA 3 + foo + +Note that there is no newline after the binary data; the next protocol +message will come immediately after it. + +## Checking if content is present + +To check if a key is present on the server, the client sends: + + CHECKPRESENT Key + +The server responds with either SUCCESS or FAILURE. + +## Locking content + +To lock content on the server, preventing it from being removed, +the client sends: + + LOCKCONTENT Key + +The server responds with either SUCCESS or FAILURE. +The former indicates the content is locked. It will remain +locked until the connection is broken, or the client +sends: + + UNLOCKCONTENT Key + +The server makes no response to that. + +## Removing content + +To remove a key's content from the server, the client sends: + + REMOVE Key + +The server responds with either SUCCESS or FAILURE. + +## Storing content on the server + +To store content on the server, the client sends: + + PUT AssociatedFile Key + +Here AssociatedFile may be the name of a file in the git +repository, for information purposes only. Or it can be the +empty string. It will always have unix directory separators. + +(Note that in the line-based serialization. AssociatedFile may not contain any +spaces, since it's not the last token in the line. Use '%' to indicate +whitespace.) + +The server may respond with ALREADY-HAVE if it already +had the conent of that key. Otherwise, it responds with: + + PUT-FROM Offset + +Offset is the number of bytes into the file that the server wants +the client to start. This allows resuming transfers. + +The client then sends a DATA message with content of the file from +the offset to the end of file. + +If the server successfully receives the data and stores the content, +it replies with SUCCESS. Otherwise, FAILURE. + +## Getting content from the server + +To get content from the server, the client sends: + + GET Offset AssociatedFile Key + +The Offset is the number of bytes into the file that the client wants +the server to skip, which allows resuming transfers. +See description of AssociatedFile above. + +The server then sends a DATA message with the content of the file +from the offset to end of file. + +The client replies with SUCCESS or FAILURE. + +## Connection to services + +This is used to connect to services like git-upload-pack and +git-receive-pack that speak their own protocol. + +The client sends a message to request the connection. +Service is the name of the service, eg "git-upload-pack". + + CONNECT Service + +Both client and server may now exchange DATA messages in any order, +encapsulating the service's protocol. + +When the service exits, the server indicates this by telling the client +its exit code. + + CONNECTDONE ExitCode + +## Change notification + +The client can request to be notified when a ref in +the git repository on the server changes. + + NOTIFYCHANGE + +The server will block until at least +one of the refs changes, and send a list of changed +refs. + + CHANGED ChangedRefs + +For example: + + CHANGED refs/heads/master refs/heads/git-annex + +Some servers may not support this command. -- cgit v1.2.3