implemented git-annex-shell p2pstdio

Not yet used by git-annex, but this will allow faster transfers etc than
using individual ssh connections and rsync.

Not called git-annex-shell p2p, because git-annex p2p does something
else and I don't want two subcommands with the same name between the two
for sanity reasons.

This commit was sponsored by Øyvind Andersen Holm.

3 files changed, 21 insertions, 2 deletions

diff --git a/doc/design/p2p_protocol.mdwn b/doc/design/p2p_protocol.mdwn
index 23ac4052a..be3c5e6cb 100644
--- a/doc/design/p2p_protocol.mdwn
+++ b/doc/design/p2p_protocol.mdwn
@@ -3,7 +3,7 @@ communicate between peers.
 
 There's a common line-based serialization of the protocol, but other
 serializations are also possible. The line-based serialization is spoken
-by [[git-annex-shell], and by [[git-annex-remotedaemon]] when serving tor.
+by [[git-annex-shell], and by git-annex over tor.
 
 One peer is known as the client, and is the peer that initiates the
 connection. The other peer is known as the server, and is the peer that the
@@ -29,6 +29,10 @@ connection.
 	AUTH_SUCCESS UUID
 	AUTH_FAILURE
 
+Note that authentication does not guarantee that the client is talking to
+who they expect to be talking to. This, and encryption of the connection,
+are handled at a lower level.
+
 ## Errors
 
 Either the client or the server may send an error message at any
diff --git a/doc/git-annex-shell.mdwn b/doc/git-annex-shell.mdwn
index 167f54012..cf72e091b 100644
--- a/doc/git-annex-shell.mdwn
+++ b/doc/git-annex-shell.mdwn
@@ -90,6 +90,12 @@ first "/~/" or "/~user/" is expanded to the specified home directory.
 
   Sets up a repository as a gcrypt repository.
 
+* p2pstdio directory
+
+  This causes git-annex-shell to communicate using the git-annex p2p
+  protocol over stdio. When supported by git-annex-shell, this allows
+  multiple actions to be run over a single connection, improving speed.
+
 # OPTIONS
 
 Most options are the same as in git-annex. The ones specific
diff --git a/doc/todo/accellerate_ssh_remotes_with_git-annex-shell_mass_protocol.mdwn b/doc/todo/accellerate_ssh_remotes_with_git-annex-shell_mass_protocol.mdwn
index dd6be9a30..ff4b8c59d 100644
--- a/doc/todo/accellerate_ssh_remotes_with_git-annex-shell_mass_protocol.mdwn
+++ b/doc/todo/accellerate_ssh_remotes_with_git-annex-shell_mass_protocol.mdwn
@@ -23,7 +23,7 @@ letting git-annex-shell on the remote work that out.
 So, it seems better to not use sftp, and instead roll our own simple
 file transfer protocol.
 
-So, "git-annex-shell -c multi" would speak a protocol over stdin/stdout
+So, "git-annex-shell -c p2pstdio" would speak a protocol over stdin/stdout
 that essentially contains the commands inannex, lockcontent, dropkey,
 recvkey, and sendkey.
 
@@ -31,3 +31,12 @@ P2P.Protocol already contains such a similar protocol, used over tor.
 That protocol even supports resuming interrupted transfers.
 It has stuff including auth that this wouldn't need, but it would be
 good to unify with it as much as possible.
+
+----
+
+Implementation todos:
+
+* git-annex-shell p2pstdio currently always verifies content it receives.
+  git-annex-shell recvkey has a speed optimisation, when it's told the file
+  being sent is locked, it can avoid an expensive verification.
+* Maybe similar for transfers in the other direction?