summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorGravatar Joey Hess <joey@kitenet.net>2012-09-16 17:54:12 -0400
committerGravatar Joey Hess <joey@kitenet.net>2012-09-16 17:54:12 -0400
commit601ee470af277ad3d0041b6ce425d808075dc997 (patch)
treed15de1ea91f0bb7f4ddaa5d09324ee47cb0caae7
parente61512f42da04f424b70a345d42976296a1a7360 (diff)
sync: Pushes the git-annex branch to remote/synced/git-annex, rather than directly to remote/git-annex.
This fixes a problem I was seeing in the assistant where two remotes would attempt to sync with one another at the same time, and both failed pushing the diverged git-annex branch. Then when both tried to resolve the failed push, they each modified their git-annex branch, which again each blocked the other from pushing into it. The result was that the git-annex branches were perpetually diverged (despite having the same content!) and once the assistant fell into this trap, it couldn't get out and always had to do the slow push/fail/pull/merge/push/fail cycle.
-rw-r--r--Command/Sync.hs12
-rw-r--r--debian/changelog2
-rw-r--r--doc/design/assistant/syncing.mdwn33
-rw-r--r--doc/todo/assistant_git_sync_laddering.mdwn4
4 files changed, 42 insertions, 9 deletions
diff --git a/Command/Sync.hs b/Command/Sync.hs
index 630ceb053..bbe6a98b3 100644
--- a/Command/Sync.hs
+++ b/Command/Sync.hs
@@ -154,15 +154,15 @@ pushRemote remote branch = go =<< needpush
pushBranch :: Remote -> Git.Ref -> Git.Repo -> IO Bool
pushBranch remote branch g =
Git.Command.runBool "push"
- [ Param (Remote.name remote)
- , Param (show Annex.Branch.name)
- , Param refspec
+ [ Param $ Remote.name remote
+ , Param $ refspec Annex.Branch.name
+ , Param $ refspec branch
] g
where
- refspec = concat
- [ show $ Git.Ref.base branch
+ refspec b = concat
+ [ show $ Git.Ref.base b
, ":"
- , show $ Git.Ref.base $ syncBranch branch
+ , show $ Git.Ref.base $ syncBranch b
]
mergeAnnex :: CommandStart
diff --git a/debian/changelog b/debian/changelog
index cc567e581..06e227472 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -22,6 +22,8 @@ git-annex (3.20120826) UNRELEASED; urgency=low
rather than moving to .git/annex/bad/
* Avoid crashing on encoding errors in filenames when writing transfer info
files and reading from checksum commands.
+ * sync: Pushes the git-annex branch to remote/synced/git-annex, rather
+ than directly to remote/git-annex.
-- Joey Hess <joeyh@debian.org> Mon, 27 Aug 2012 13:27:39 -0400
diff --git a/doc/design/assistant/syncing.mdwn b/doc/design/assistant/syncing.mdwn
index 85667301d..f874b9932 100644
--- a/doc/design/assistant/syncing.mdwn
+++ b/doc/design/assistant/syncing.mdwn
@@ -9,8 +9,37 @@ all the other git clones, at both the git level and the key/value level.
(in a fresh clone each time) several times in a row, but then stops happening,
which has prevented me from debugging it.
This could possibly have been caused by the bug fixed in 750c4ac6c282d14d19f79e0711f858367da145e4.
-* The git repository syncing sometimes fails due to the remote having updated.
- While syncing retries, this sometimes doesn't work.
+
+* The transfer code doesn't always manage to transfer file contents.
+
+ Besides reconnection events, there are two places where transfers get queued:
+
+ 1. When the committer commits a file, it queues uploads.
+ 2. When the watcher sees a broken symlink be created, it queues downloads.
+
+ Consider a doubly-linked chain of three repositories, A B and C.
+ (C and A do not directly communicate.)
+
+ * File is added to A.
+ * A uploads its content to B.
+ * At the same time, A git syncs to B.
+ * Once B gets the git sync, it git syncs to C.
+ * When C's watcher sees the file appear, it tries to download it. But if
+ B had not finished receiving the file from A, C doesn't know B has it,
+ and cannot download it from anywhere.
+
+ Possible solution: After B receives content, it could queue uploads of it
+ to all remotes that it doesn't know have it yet, which would include C.
+
+ In practice, this has the problem that when C receives the content,
+ it will queue uploads of it, which can send back to B (or to some other repo
+ that already has the content) and loop, until the git-annex branches catch
+ up and break the cycle.
+
+ Possible solution: C could record a download intent. (Similar to a failed
+ download, but with an unknown source.) When C next receives a git-annex
+ branch push, it could try to requeue downloads that it has such intents
+ registered for.
## TODO
diff --git a/doc/todo/assistant_git_sync_laddering.mdwn b/doc/todo/assistant_git_sync_laddering.mdwn
index f210d41f6..7dbc6480a 100644
--- a/doc/todo/assistant_git_sync_laddering.mdwn
+++ b/doc/todo/assistant_git_sync_laddering.mdwn
@@ -5,4 +5,6 @@ that then has to be auto-resolved.
This seems similar to the laddering problem described in this old bug:
[[bugs/making_annex-merge_try_a_fast-forward]]
---[[Joey]]
+--[[Joey]]
+
+Think I've fixed this. [[done]] --[[Joey]]