summaryrefslogtreecommitdiff
path: root/doc/design/git-remote-daemon.mdwn
diff options
context:
space:
mode:
authorGravatar Joey Hess <joey@kitenet.net>2014-04-06 19:06:03 -0400
committerGravatar Joey Hess <joey@kitenet.net>2014-04-06 19:10:23 -0400
commit8c4bfe2f2141bce84ea22120da445c148b6f1168 (patch)
tree817f5951cec02c1acec3f26c886beea79cf0957c /doc/design/git-remote-daemon.mdwn
parent1eb96cc31a0f0ec0339f6b28a362b057444069af (diff)
added git-annex remotedaemon
So far, handling connecting to git-annex-shell notifychanges, and pulling immediately when a change is pushed to a remote. A little bit buggy (crashes after the first pull), but it already works! This commit was sponsored by Mark Sheppard.
Diffstat (limited to 'doc/design/git-remote-daemon.mdwn')
-rw-r--r--doc/design/git-remote-daemon.mdwn112
1 files changed, 64 insertions, 48 deletions
diff --git a/doc/design/git-remote-daemon.mdwn b/doc/design/git-remote-daemon.mdwn
index 0658d4583..db56bd633 100644
--- a/doc/design/git-remote-daemon.mdwn
+++ b/doc/design/git-remote-daemon.mdwn
@@ -37,35 +37,24 @@
# design
-Let git-remote-daemon be the name. It runs in a repo and
-either:
+Let git-remote-daemon be the name. Or for git-annex,
+`git annex remotedaemon`.
-* forks to background and performs configured actions (ie, `git pull`)
-* with --foreground, communicates over stdio
- with its caller using a simple protocol (exiting when its caller closes its
- stdin handle so it will stop when the assistant stops).
+It runs in one of two ways:
-It is configured entirely by .git/config.
+1. Forked to background, using a named pipe for the control protocol.
+2. With --foreground, the control protocol goes over stdio.
-# encryption & authentication
-
-For simplicity, the network transports have to do their own end-to-end
-encryption. Encryption is not part of this design.
-
-(XMPP does not do end-to-end encryption, but might be supported
-transitionally.)
-
-Ditto for authentication that we're talking to who we indend to talk to.
-Any public key data etc used for authenticion is part of the remote's
-configuration (or hidden away in a secure chmodded file, if neccesary).
-This design does not concern itself with authenticating the remote node,
-it just takes the auth token and uses it.
+Either way, behavior is the same:
-For example, in telehash, each node has its own keypair, which is used
-or authentication and encryption, and is all that's needed to route
-messages to that node.
+* Get a list of remotes to act on by looking at .git/config
+* Automatically notices when a remote has changes to branches
+ matching remote.$name.fetch, and pulls them down to the appropriate
+ location.
+* When the control protocol informs it about a new ref that's available,
+ it offers the ref to any interested remotes.
-# stdio protocol
+# control protocol
This is an asynchronous protocol. Ie, either side can send any message
at any time, and the other side does not send a reply.
@@ -82,25 +71,21 @@ the webapp.
* `CONNECTED $remote`
- Send when a connection has been made with a remote.
+ Sent when a connection has been made with a remote.
* `DISCONNECTED $remote`
- Send when connection with a remote has been lost.
-
-* `CHANGED $remote $sha ...`
-
- This indicates that refs in the named git remote have changed,
- and indicates the new shas.
+ Sent when connection with a remote has been lost.
-* `STATUS $remote $string`
+* `SYNCING $remote`
- A user-visible status message about a named remote.
+ Indicates that a pull or a push with a remote is in progress.
+ Always followed by DONESYNCING.
-* `ERROR $remote $string`
+* `DONESYNCING $remote 1|0`
- A user-visible error about a named remote.
- (Can continue running past this point, for this or other remotes.)
+ Indicates that syncing with a remote is done, and either succeeded
+ (1) or failed (0).
## consumed messages
@@ -119,21 +104,40 @@ the webapp.
Affects all remotes.
-* `PUSH $remote`
+* `CHANGED ref ...`
- Requests that a git push be done with the remote over the network
- transport when next possible. May be repeated many times before the push
- finally happens.
+ Indicates that a ref is new or has changed. These can be offered to peers,
+ and peers that are interested in them can pull the content.
* `RELOAD`
Indicates that configs have changed. Daemon should reload .git/config
and/or restart.
-# send-pack and receive-pack
+* `STOP`
-Done as the assistant does with XMPP currently. Does not involve
-communication over the above stdio protocol.
+ Shut down git-remote-daemon
+
+ (When using stdio, it also should shutdown when it reaches EOF on
+ stdin.)
+
+# encryption & authentication
+
+For simplicity, the network transports have to do their own end-to-end
+encryption. Encryption is not part of this design.
+
+(XMPP does not do end-to-end encryption, but might be supported
+transitionally.)
+
+Ditto for authentication that we're talking to who we indend to talk to.
+Any public key data etc used for authenticion is part of the remote's
+configuration (or hidden away in a secure chmodded file, if neccesary).
+This design does not concern itself with authenticating the remote node,
+it just takes the auth token and uses it.
+
+For example, in telehash, each node has its own keypair, which is used
+or authentication and encryption, and is all that's needed to route
+messages to that node.
# network level protocol
@@ -143,17 +147,29 @@ This seems to need to be network-layer dependant. Telehash will need
one design, and git-annex-shell on a central ssh server has a very different
(and much simpler) design.
-## git-annex-shell
+## ssh
+
+`git-annex-shell notifychanges` is run, and speaks a simple protocol
+over stdio to inform when refs on the remote have changed.
+
+No pushing is done for CHANGED, since git handles ssh natively.
-Speak a subset of the stdio protocol between git-annex-shell and
-git-remote-daemon, over ssh.
+TODO:
-Only thing that seems to be needed is CHANGED, actually!
+* It already detects changes and pulls, but it then dies with a protocol
+ error.
+* Remote system might not be available. Find a smart way to detect it,
+ ideally w/o generating network traffic. One way might be to check
+ if the ssh connection caching control socket exists, for example.
+* Remote system might be available, and connection get lost. Should
+ reconnect, but needs to avoid bad behavior (ie, constant reconnect
+ attempts.)
+* Detect if old system had a too old git-annex-shell and avoid bad behavior
## telehash
TODO
-## XMPP
+## xmpp
Reuse [[assistant/xmpp]]