diff options
3 files changed, 51 insertions, 3 deletions
diff --git a/doc/bugs/More_sync__39__ing_weirdness_with_the_assistant_branch_on_OSX.mdwn b/doc/bugs/More_sync__39__ing_weirdness_with_the_assistant_branch_on_OSX.mdwn index 63431aae6..00f425307 100644 --- a/doc/bugs/More_sync__39__ing_weirdness_with_the_assistant_branch_on_OSX.mdwn +++ b/doc/bugs/More_sync__39__ing_weirdness_with_the_assistant_branch_on_OSX.mdwn @@ -10,3 +10,6 @@ hint: See the 'Note about fast-forwards' in 'git push --help' for details. (Recording state in git...) manually running a 'git annex sync' usually fixes it, I guess once the sync command runs periodically this problem will go away, is this even OSX specific? I don't quite get the behaviour that is described in [[design/assistant/blog/day_15__its_aliiive]]. + +> With my changes today, I've seen it successfully recover from this +> situation. [[done]] --[[Joey]] diff --git a/doc/design/assistant/blog/day_16__more_robust_syncing.mdwn b/doc/design/assistant/blog/day_16__more_robust_syncing.mdwn new file mode 100644 index 000000000..e9343f846 --- /dev/null +++ b/doc/design/assistant/blog/day_16__more_robust_syncing.mdwn @@ -0,0 +1,44 @@ +I released a version of git-annex over the weekend that includes the `git +annex watch` command. There's a minor issue installing it from cabal on +OSX, which I've fixed in my tree. Nice timing: At least the watch command +should be shipped in the next Debian release, which freezes at the end of +the month. + +Jimmy found out how kqueue [[blows +up|bugs/Issue_on_OSX_with_some_system_limits]] when there are too many +directories to keep all open. I'm not surprised this happens, but it's nice +to see exactly how. Odd that it happened to him at just 512 directories; +I'd have guessed more. I have plans to fork watcher programs that each +watch 512 directories (or whatever the ulimit is), to deal with this. What +a pitiful interface is kqueue.. I have not thought yet about how the watcher +programs would communicate back to the main program. + +---- + +Back on the assistant front, I've worked today on making git syncing more +robust. Now when a push fails, it tries a pull, and a merge, and repushes. +That ensures that the push is, almost always, a fast-forward. Unless +something else gets in a push first, anyway! + +If a push still fails, there's Yet Another Thread, added today, that will +wake up after 30 minutes and retry the push. It currently keeps retrying +every 30 minutes until the push finally gets though. This will deal, to +some degree, with those situations where a remote is only sometimes +available. + +I need to refine the code a bit, to avoid it keeping an ever-growing queue +of failed pushes, if a remote is just dead. And to clear old failed pushes +from the queue when a later push succeeds. + +I also need to write a git merge driver that handles conflicts in the tree. +If two conflicting versions of a file `foo` are saved, this would merge +them, renaming them to `foo.X` and `foo.Y`. Probably X and Y are the +git-annex keys for the content of the files; this way all clones will +resolve the conflict in a way that leads to the same tree. It's also +possible to get a conflict by one repo deleting a file, and another +modifying it. In this case, renaming the deleted file to `foo.Y` may +be the right approach, I am not sure. + +I glanced through some Haskell dbus bindings today. I belive there are dbus +events available to detect when drives are mounted, and on Linux this would +let git-annex notice and sync to usb drives, etc. diff --git a/doc/design/assistant/syncing.mdwn b/doc/design/assistant/syncing.mdwn index 8173457c5..3e90e6b10 100644 --- a/doc/design/assistant/syncing.mdwn +++ b/doc/design/assistant/syncing.mdwn @@ -13,8 +13,9 @@ all the other git clones, at both the git level and the key/value level. [The watching can be done with the existing inotify code! This avoids needing any special mechanism to notify a remote that it's been synced to.] **done** -1. Periodically retry pushes that failed. Also, detect if a push failed - due to not being up-to-date, pull, and repush. +1. Periodically retry pushes that failed. **done** (every half an hour) +1. Also, detect if a push failed due to not being up-to-date, pull, + and repush. **done** 2. Use a git merge driver that adds both conflicting files, so conflicts never break a sync. 3. Investigate the XMPP approach like dvcs-autosync does, or other ways of @@ -41,7 +42,7 @@ This probably will need lots of refinements to get working well. ## other considerations It would be nice if, when a USB drive is connected, -syncing starts automatically. +syncing starts automatically. Use dbus on Linux? This assumes the network is connected. It's often not, so the [[cloud]] needs to be used to bridge between LANs. |