diff options
author | 2012-09-16 17:54:12 -0400 | |
---|---|---|
committer | 2012-09-16 17:54:12 -0400 | |
commit | 601ee470af277ad3d0041b6ce425d808075dc997 (patch) | |
tree | d15de1ea91f0bb7f4ddaa5d09324ee47cb0caae7 /doc/design/assistant | |
parent | e61512f42da04f424b70a345d42976296a1a7360 (diff) |
sync: Pushes the git-annex branch to remote/synced/git-annex, rather than directly to remote/git-annex.
This fixes a problem I was seeing in the assistant where two remotes would
attempt to sync with one another at the same time, and both failed pushing
the diverged git-annex branch. Then when both tried to resolve the failed
push, they each modified their git-annex branch, which again each blocked
the other from pushing into it. The result was that the git-annex
branches were perpetually diverged (despite having the same content!) and
once the assistant fell into this trap, it couldn't get out and always
had to do the slow push/fail/pull/merge/push/fail cycle.
Diffstat (limited to 'doc/design/assistant')
-rw-r--r-- | doc/design/assistant/syncing.mdwn | 33 |
1 files changed, 31 insertions, 2 deletions
diff --git a/doc/design/assistant/syncing.mdwn b/doc/design/assistant/syncing.mdwn index 85667301d..f874b9932 100644 --- a/doc/design/assistant/syncing.mdwn +++ b/doc/design/assistant/syncing.mdwn @@ -9,8 +9,37 @@ all the other git clones, at both the git level and the key/value level. (in a fresh clone each time) several times in a row, but then stops happening, which has prevented me from debugging it. This could possibly have been caused by the bug fixed in 750c4ac6c282d14d19f79e0711f858367da145e4. -* The git repository syncing sometimes fails due to the remote having updated. - While syncing retries, this sometimes doesn't work. + +* The transfer code doesn't always manage to transfer file contents. + + Besides reconnection events, there are two places where transfers get queued: + + 1. When the committer commits a file, it queues uploads. + 2. When the watcher sees a broken symlink be created, it queues downloads. + + Consider a doubly-linked chain of three repositories, A B and C. + (C and A do not directly communicate.) + + * File is added to A. + * A uploads its content to B. + * At the same time, A git syncs to B. + * Once B gets the git sync, it git syncs to C. + * When C's watcher sees the file appear, it tries to download it. But if + B had not finished receiving the file from A, C doesn't know B has it, + and cannot download it from anywhere. + + Possible solution: After B receives content, it could queue uploads of it + to all remotes that it doesn't know have it yet, which would include C. + + In practice, this has the problem that when C receives the content, + it will queue uploads of it, which can send back to B (or to some other repo + that already has the content) and loop, until the git-annex branches catch + up and break the cycle. + + Possible solution: C could record a download intent. (Similar to a failed + download, but with an unknown source.) When C next receives a git-annex + branch push, it could try to requeue downloads that it has such intents + registered for. ## TODO |