diff options
| -rw-r--r-- | doc/design/assistant/blog/day_61__network_connection_detection.mdwn | 36 | ||||
| -rw-r--r-- | doc/design/assistant/syncing.mdwn | 17 |
2 files changed, 50 insertions, 3 deletions
diff --git a/doc/design/assistant/blog/day_61__network_connection_detection.mdwn b/doc/design/assistant/blog/day_61__network_connection_detection.mdwn new file mode 100644 index 000000000..8ab40f516 --- /dev/null +++ b/doc/design/assistant/blog/day_61__network_connection_detection.mdwn @@ -0,0 +1,36 @@ +Today, added a thread that deals with recovering when there's been a loss +of network connectivity. When the network's down, the normal immediate +syncing of changes of course doesn't work. So this thread detects when the +network comes back up, and does a pull+push to network remotes, and +triggers scanning for file content that needs to be transferred. + +I used dbus again, to detect events generated by both network-manager and +wicd when they've sucessfully brought an interface up. Or, if they're not +available, it polls every 30 minutes. + +When the network comes up, in addition to the git pull+push, it also +currently does a full scan of the repo to find files whose contents +need to be transferred to get fully back into sync. + +I think it'll be ok for some git pulls and pushes to happen when +moving to a new network, or resuming a laptop (or every 30 minutes when +resorting to polling). But the transfer scan is currently really too heavy +to be appropriate to do every time in those situations. I have an idea for +avoiding that scan when the remote's git-annex branch has not changed. But +I need to refine it, to handle cases like this: + +1. a new remote is added +2. file contents start being transferred to (or from it) +3. the network is taken down +4. all the queued transfers fail +5. the network comes back up +6. the transfer scan needs to know the remote was not all in sync + before #3, and so should do a full scan despite the git-annex branch + not having changed + +--- + +Doubled the ram in my netbook, which I use for all development. Yesod needs +rather a lot of ram to compile and link, and this should make me quite a +lot more productive. I was struggling with OOM killing bits of chromium +during my last week of development. diff --git a/doc/design/assistant/syncing.mdwn b/doc/design/assistant/syncing.mdwn index 3aeb76afc..898081574 100644 --- a/doc/design/assistant/syncing.mdwn +++ b/doc/design/assistant/syncing.mdwn @@ -3,9 +3,15 @@ all the other git clones, at both the git level and the key/value level. ## immediate action items -* At startup, and possibly periodically, or when the network connection - changes, or some heuristic suggests that a remote was disconnected from - us for a while, queue remotes for processing by the TransferScanner. +* Sync with all available remotes on startup. +* TransferScanner should avoid unnecessary scanning of remotes. + This is paricilarly important for scans queued by the NetWatcher, + which can be polling, or could be after a momentary blip in network + connectivity. The TransferScanner could check the remote's git-annex + branch; if it is not ahead of the local git-annex branch, then + there's nothing to transfer. **Except** if the tree was not already + up-to-date before the loss of connectivity. So doing this needs + tracking of when the tree is not yet fully up-to-date. * Ensure that when a remote receives content, and updates its location log, it syncs that update back out. Prerequisite for: * After git sync, identify new content that we don't have that is now available @@ -157,3 +163,8 @@ redone to check it. finishes. **done** * Test MountWatcher on KDE, and add whatever dbus events KDE emits when drives are mounted. **done** +* Possibly periodically, or when the network connection + changes, or some heuristic suggests that a remote was disconnected from + us for a while, queue remotes for processing by the TransferScanner. + **done**; both network-manager and wicd connection events are supported, + and it falls back to polling every 30 minutes when neither is available. |