diff options
| author |  Joey Hess <joeyh@joeyh.name> | 2018-03-08 17:04:52 -0400 |
|---|---|---|
| committer | Joey Hess <joeyh@joeyh.name> | 2018-03-08 17:04:52 -0400 |
| commit | 1a772b2fff0692e6f17de2c168c732cacb526173 (patch) | |
| tree | 3ce12867d7f05535e83c4aadad702adc1b78ac04 | |
| parent | fa3db5c4d3704da5b4a7ffeb0b637f8f5dfda616 (diff) | |
devblog
| -rw-r--r-- | doc/devblog/day_488__groundwork_for_using_p2pstdio.mdwn | 30 |
1 files changed, 30 insertions, 0 deletions
diff --git a/doc/devblog/day_488__groundwork_for_using_p2pstdio.mdwn b/doc/devblog/day_488__groundwork_for_using_p2pstdio.mdwn new file mode 100644 index 000000000..e09da769a --- /dev/null +++ b/doc/devblog/day_488__groundwork_for_using_p2pstdio.mdwn @@ -0,0 +1,30 @@ +Spent most of the day laying groundwork for using git-annex-shell p2pstdio. +Implemented pools of ssh connections to it, and added uuid verification. +Then generalized code from the p2p remote so it can be reused in the git +remote. The types got super hairy in there, but the code reuse level is +excellent. + +Finally it was time to convert the first ssh remote method +to use the P2P protocol. I chose key removal, since benchmarking it doesn't +involve the size of annexed objects. + +Here's the P2P protocol in action over ssh: + + [2018-03-08 17:02:47.688627136] chat: ssh ["localhost","-S",".git/annex/ssh/localhost","-o","ControlMaster=auto","-o","ControlPersist=yes","-T","git-annex-shell 'p2pstdio' '/~/tmp/bench/a' '--debug' 'da72c285-2615-4a67-828f-eaae4f42fc3d' --uuid db017fac-eb8f-42d9-9d09-2780b193cef1"] + [2018-03-08 17:02:47.901897195] P2P < AUTH-SUCCESS db017fac-eb8f-42d9-9d09-2780b193cef1 + [2018-03-08 17:02:47.902025504] P2P > REMOVE SHA256E-s4--97b912eb4a61df5f806ca6239dde3e1a4f51ad20aced1642cbb83dc510a5fa6b + [2018-03-08 17:02:47.910074003] P2P < SUCCESS + [2018-03-08 17:02:47.914181701] P2P > REMOVE SHA256E-s4--6af2f5b785a8930f0bd3edc833e18fa191167ab0535ef359b19a1982a6984e96 + [2018-03-08 17:02:47.918699806] P2P < SUCCESS + +For a benchmark, I set up a repository with 1000 annexed files, +and cloned it from localhost, then ran `git annex drop --from origin`. + +before: 41 seconds +after: 10 seconds + +400% speedup for dropping is pretty great.. And when there's more latency +than loopback has, the improvement should be more pronounced. +Will test it this evening over my satellite internet. :) + +Today's work was sponsored by Trenton Cronholm on [Patreon](https://patreon.com/joeyh/). |