blog for the day

2 files changed, 50 insertions, 3 deletions

diff --git a/doc/design/assistant/blog/day_61__network_connection_detection.mdwn b/doc/design/assistant/blog/day_61__network_connection_detection.mdwn
new file mode 100644
index 000000000..8ab40f516
--- /dev/null
+++ b/doc/design/assistant/blog/day_61__network_connection_detection.mdwn
@@ -0,0 +1,36 @@
+Today, added a thread that deals with recovering when there's been a loss
+of network connectivity. When the network's down, the normal immediate
+syncing of changes of course doesn't work. So this thread detects when the
+network comes back up, and does a pull+push to network remotes, and
+triggers scanning for file content that needs to be transferred.
+
+I used dbus again, to detect events generated by both network-manager and
+wicd when they've sucessfully brought an interface up. Or, if they're not
+available, it polls every 30 minutes.
+
+When the network comes up, in addition to the git pull+push, it also
+currently does a full scan of the repo to find files whose contents
+need to be transferred to get fully back into sync.
+
+I think it'll be ok for some git pulls and pushes to happen when
+moving to a new network, or resuming a laptop (or every 30 minutes when
+resorting to polling). But the transfer scan is currently really too heavy
+to be appropriate to do every time in those situations. I have an idea for
+avoiding that scan when the remote's git-annex branch has not changed. But
+I need to refine it, to handle cases like this:
+
+1. a new remote is added
+2. file contents start being transferred to (or from it)
+3. the network is taken down
+4. all the queued transfers fail
+5. the network comes back up
+6. the transfer scan needs to know the remote was not all in sync
+   before #3, and so should do a full scan despite the git-annex branch
+   not having changed
+
+---
+
+Doubled the ram in my netbook, which I use for all development. Yesod needs
+rather a lot of ram to compile and link, and this should make me quite a
+lot more productive. I was struggling with OOM killing bits of chromium
+during my last week of development.
diff --git a/doc/design/assistant/syncing.mdwn b/doc/design/assistant/syncing.mdwn
index 3aeb76afc..898081574 100644
--- a/doc/design/assistant/syncing.mdwn
+++ b/doc/design/assistant/syncing.mdwn
@@ -3,9 +3,15 @@ all the other git clones, at both the git level and the key/value level.
 
 ## immediate action items
 
-* At startup, and possibly periodically, or when the network connection
-  changes, or some heuristic suggests that a remote was disconnected from
-  us for a while, queue remotes for processing by the TransferScanner.
+* Sync with all available remotes on startup.
+* TransferScanner should avoid unnecessary scanning of remotes.
+  This is paricilarly important for scans queued by the NetWatcher,
+  which can be polling, or could be after a momentary blip in network
+  connectivity. The TransferScanner could check the remote's git-annex
+  branch; if it is not ahead of the local git-annex branch, then
+  there's nothing to transfer. **Except** if the tree was not already
+  up-to-date before the loss of connectivity. So doing this needs
+  tracking of when the tree is not yet fully up-to-date.
 * Ensure that when a remote receives content, and updates its location log,
   it syncs that update back out. Prerequisite for:
 * After git sync, identify new content that we don't have that is now available
@@ -157,3 +163,8 @@ redone to check it.
   finishes. **done**
 * Test MountWatcher on KDE, and add whatever dbus events KDE emits when
   drives are mounted. **done**
+* Possibly periodically, or when the network connection
+  changes, or some heuristic suggests that a remote was disconnected from
+  us for a while, queue remotes for processing by the TransferScanner. 
+  **done**; both network-manager and wicd connection events are supported,
+  and it falls back to polling every 30 minutes when neither is available.