diff options
Diffstat (limited to 'doc/design')
-rw-r--r-- | doc/design/assistant/syncing.mdwn | 33 |
1 files changed, 31 insertions, 2 deletions
diff --git a/doc/design/assistant/syncing.mdwn b/doc/design/assistant/syncing.mdwn index 85667301d..f874b9932 100644 --- a/doc/design/assistant/syncing.mdwn +++ b/doc/design/assistant/syncing.mdwn @@ -9,8 +9,37 @@ all the other git clones, at both the git level and the key/value level. (in a fresh clone each time) several times in a row, but then stops happening, which has prevented me from debugging it. This could possibly have been caused by the bug fixed in 750c4ac6c282d14d19f79e0711f858367da145e4. -* The git repository syncing sometimes fails due to the remote having updated. - While syncing retries, this sometimes doesn't work. + +* The transfer code doesn't always manage to transfer file contents. + + Besides reconnection events, there are two places where transfers get queued: + + 1. When the committer commits a file, it queues uploads. + 2. When the watcher sees a broken symlink be created, it queues downloads. + + Consider a doubly-linked chain of three repositories, A B and C. + (C and A do not directly communicate.) + + * File is added to A. + * A uploads its content to B. + * At the same time, A git syncs to B. + * Once B gets the git sync, it git syncs to C. + * When C's watcher sees the file appear, it tries to download it. But if + B had not finished receiving the file from A, C doesn't know B has it, + and cannot download it from anywhere. + + Possible solution: After B receives content, it could queue uploads of it + to all remotes that it doesn't know have it yet, which would include C. + + In practice, this has the problem that when C receives the content, + it will queue uploads of it, which can send back to B (or to some other repo + that already has the content) and loop, until the git-annex branches catch + up and break the cycle. + + Possible solution: C could record a download intent. (Similar to a failed + download, but with an unknown source.) When C next receives a git-annex + branch push, it could try to requeue downloads that it has such intents + registered for. ## TODO |