update; split out hard todo

3 files changed, 42 insertions, 16 deletions

diff --git a/doc/design/assistant/inotify.mdwn b/doc/design/assistant/inotify.mdwn
index 7b600090a..fd81366d4 100644
--- a/doc/design/assistant/inotify.mdwn
+++ b/doc/design/assistant/inotify.mdwn
@@ -18,6 +18,17 @@ available!
   I may need to fork off multiple watcher processes to handle this.
   See [[bugs/Issue_on_OSX_with_some_system_limits]].
 
+## todo
+
+* Run niced and ioniced? Seems to make sense, this is a background job.
+* configurable option to only annex files meeting certian size or
+  filename criteria
+* option to check files not meeting annex criteria into git directly,
+  automatically
+* honor .gitignore, not adding files it excludes (difficult, probably
+  needs my own .gitignore parser to avoid excessive running of git commands
+  to check for ignored files)
+
 ## beyond Linux
 
 I'd also like to support OSX and if possible the BSDs.
@@ -65,17 +76,6 @@ I'd also like to support OSX and if possible the BSDs.
 
 * Windows has a Win32 ReadDirectoryChangesW, and perhaps other things.
 
-## todo
-
-- Run niced and ioniced? Seems to make sense, this is a background job.
-- configurable option to only annex files meeting certian size or
-  filename criteria
-- option to check files not meeting annex criteria into git directly,
-  automatically
-- honor .gitignore, not adding files it excludes (difficult, probably
-  needs my own .gitignore parser to avoid excessive running of git commands
-  to check for ignored files)
-
 ## the races
 
 Many races need to be dealt with by this code. Here are some of them.
diff --git a/doc/design/assistant/syncing.mdwn b/doc/design/assistant/syncing.mdwn
index 2871ec216..98749d244 100644
--- a/doc/design/assistant/syncing.mdwn
+++ b/doc/design/assistant/syncing.mdwn
@@ -10,7 +10,8 @@ all the other git clones, at both the git level and the key/value level.
   them. Also, enqueue Downloads for any files we're missing.
 * The TransferWatcher does not notice ongoing transfers, because inotify is
   waiting for the info file to be closed, but that never happens, it's left
-  open to keep it locked.
+  open to keep it locked. May need to separate the transfer info files
+  into an info file, and a lock file.
 
 ## longer-term TODO
 
@@ -32,10 +33,6 @@ all the other git clones, at both the git level and the key/value level.
   only uploading new files but not downloading, and only downloading
   files in some directories and not others. See for use cases:
   [[forum/Wishlist:_options_for_syncing_meta-data_and_data]]
-* Running external commands from one thread blocks all of them until
-  it completes. Try to switch to haskell's threaded runtime, which I 
-  think fixes this. Failing that, make sure all network accessing
-  commands are run by separate processes or something.
 
 ## data syncing
 
diff --git a/doc/todo/threaded_runtime.mdwn b/doc/todo/threaded_runtime.mdwn
new file mode 100644
index 000000000..095ffa435
--- /dev/null
+++ b/doc/todo/threaded_runtime.mdwn
@@ -0,0 +1,29 @@
+The [[design/assistant]] would be better if git-annex used ghc's threaded
+runtime (`ghc -threaded`).
+
+Currently, whenever the assistant code runs some external command, all
+threads are blocked waiting for it to finish.
+
+For transfers, the assistant works around this problem by forking separate
+upload processes, and not waiting on them until it sees an indication that
+they have finished the transfer. While this works, it's messy.. threaded
+would be better.
+
+When pulling, pushing, and merging, the assistant runs external git
+commands, and this does block all other threads. The threaded runtime would
+really help here.
+
+---
+
+Currently, git-annex seems unstable when built with the threaded runtime.
+The test suite tends to hang when testing add. `git-annex` occasionally
+hangs, apparently in a futex lock. This is not the assistant hanging, and
+git-annex does not otherwise use threads, so this is surprising. --[[Joey]]
+
+---
+
+It would be possible to not use the threaded runtime. Instead, we could
+have a child process pool, with associated continuations to run after a
+child process finishes. Then periodically do a nonblocking waitpid on each
+process in the pool in turn (waiting for any child could break anything not
+using the pool!). This is probably a last resort...