summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorGravatar Joey Hess <joey@kitenet.net>2013-12-17 14:07:37 -0400
committerGravatar Joey Hess <joey@kitenet.net>2013-12-17 14:07:37 -0400
commita0a0dc4735f971198e8ab88fb7ee80a9c7001259 (patch)
tree696f62485b44dbc50643500eb4c866ecf0478058
parent9f802f6665cec59b6d8c1dbdf1cbc22e4c01c9d6 (diff)
I think I've convinced myself that the assistant is safe on windows despite the lack of lsof
-rw-r--r--doc/design/assistant/inotify.mdwn45
-rw-r--r--doc/todo/windows_support.mdwn3
2 files changed, 45 insertions, 3 deletions
diff --git a/doc/design/assistant/inotify.mdwn b/doc/design/assistant/inotify.mdwn
index d1afe63c5..5e1b4496c 100644
--- a/doc/design/assistant/inotify.mdwn
+++ b/doc/design/assistant/inotify.mdwn
@@ -64,8 +64,53 @@ I'd also like to support OSX and if possible the BSDs.
* <http://pypi.python.org/pypi/MacFSEvents/0.2.8> (good example program)
* <https://github.com/gorakhargosh/watchdog/blob/master/src/watchdog/observers/fsevents.py> (good example program)
+ *hfsevents is now supported*
+
* Windows has a Win32 ReadDirectoryChangesW, and perhaps other things.
+ It was easy to get watching to work in windows. But there is no lsof,
+ to check if a file can safely be added. So, need to carefully consider
+ how to make adding a file safe in windows.
+
+ Without lsof, an InodeCache is generated in "lockdown" (which doesn't
+ do anything to prevent new writers), and is compared with the stat of the
+ file after it's ingested (and checksummed). This will detect many changes
+ to files, which change the size or mtime.
+
+ So, we have 2 cases to worry about.
+
+ 1. A process has the file open for write as it's added, does not change
+ it until the add is done.
+
+ As long as an event is generated once the file does get closed, this
+ is fine -- the modified version will be re-added. And such events are
+ indeed generated on windows.
+
+ 2. A process has the file open for write as it's added, and changes it
+ in some way that does not affect size or mtime.
+
+ If an event is generated when the file does get closed, this is the
+ same as a scenario where a process opens the file after it's added,
+ makes such a change, and closes it. In either case, a file closed
+ event is generated, and the Watcher will not detect any change
+ using the inode cache, so will not re-add the file.
+
+ So, this scenario is a potential problem, but it seems at least
+ unlikely that a program would modify a file without affecting its
+ mtime. Note that this same scenario can happen even with lsof, and
+ even on linux (although on linux the InodeCache includes an actual
+ inode, which might detect the change too).
+
+ Conclusion: It's probably ok to run without lsof on Windows.
+
+ Corrolary: lsof might not generally be needed in direct mode, on
+ systems that do generate file close events (but not when
+ eventsCoalesce).
+ The same arguments given above seem to apply to !Windows. Note that lsof
+ is needed in indirect mode, as discussed below.
+
+ **windows is now supported**
+
## the races
Many races need to be dealt with by this code. Here are some of them.
diff --git a/doc/todo/windows_support.mdwn b/doc/todo/windows_support.mdwn
index 0cfd919b2..fea8241cc 100644
--- a/doc/todo/windows_support.mdwn
+++ b/doc/todo/windows_support.mdwn
@@ -12,9 +12,6 @@ now! --[[Joey]]
* Ssh connection caching does not work on Windows, so `git annex get`
has to connect twice to the remote system over ssh per file, which
is much slower than on systems supporting connection caching.
-* `git annex watch` works, but does not detect when files are still open
- for write as they're being added (no lsof). So watch/webapp/assistant
- may be unsafe.
* `git annex assistant` has not been tested, is probably quite incomplete
and/or buggy.
* Doesn't daemonize. Maybe use