disable closingTracked on OSX

Don't trust OSX FSEvents's eventFlagItemModified to be called when the last
writer of a file closes it; apparently that sometimes does not happen,
which prevented files from being quickly added.

This commit was sponsored by John Peloquin on Patreon.

4 files changed, 58 insertions, 6 deletions

diff --git a/CHANGELOG b/CHANGELOG
index d84b3cb8c..305970f84 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -27,6 +27,9 @@ git-annex (6.20170520) UNRELEASED; urgency=medium
     into a repository watched by the assistant auto-merge.
   * Makefile: Install completions for the fish and zsh shells
     when git-annex is built with optparse-applicative-0.14.
+  * assistant: Don't trust OSX FSEvents's eventFlagItemModified to be called
+    when the last writer of a file closes it; apparently that sometimes
+    does not happen, which prevented files from being quickly added.
 
  -- Joey Hess <id@joeyh.name>  Wed, 24 May 2017 14:03:40 -0400
 
diff --git a/Utility/DirWatcher.hs b/Utility/DirWatcher.hs
index bde710626..892841f93 100644
--- a/Utility/DirWatcher.hs
+++ b/Utility/DirWatcher.hs
@@ -64,18 +64,18 @@ eventsCoalesce = error "eventsCoalesce not defined"
 {- With inotify, file closing is tracked to some extent, so an add event
  - will always be received for a file once its writer closes it, and
  - (typically) not before. This may mean multiple add events for the same file.
- - 
- - fsevents behaves similarly, although different event types are used for
- - creating and modification of the file.
  -
  - OTOH, with kqueue, add events will often be received while a file is
  - still being written to, and then no add event will be received once the
- - writer closes it. -}
+ - writer closes it.
+ - 
+ - fsevents sometimes behaves similarly, but has sometimes been 
+ - seen to behave like kqueue. -}
 closingTracked :: Bool
-#if (WITH_INOTIFY || WITH_FSEVENTS || WITH_WIN32NOTIFY)
+#if (WITH_INOTIFY || WITH_WIN32NOTIFY)
 closingTracked = True
 #else
-#if WITH_KQUEUE
+#if (WITH_KQUEUE || WITH_FSEVENTS)
 closingTracked = False
 #else
 closingTracked = error "closingTracked not defined"
diff --git a/doc/bugs/Git-annex_and_Microsoft_Office_files_on_OS_X/comment_1_0cf11096ceeb6cf93db5609a42a70641._comment b/doc/bugs/Git-annex_and_Microsoft_Office_files_on_OS_X/comment_1_0cf11096ceeb6cf93db5609a42a70641._comment
new file mode 100644
index 000000000..ae27704b2
--- /dev/null
+++ b/doc/bugs/Git-annex_and_Microsoft_Office_files_on_OS_X/comment_1_0cf11096ceeb6cf93db5609a42a70641._comment
@@ -0,0 +1,41 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2017-06-09T17:52:00Z"
+ content="""
+This must have to do with the fsevents interface used on OSX.
+
+In Assistant.Threads.Committer.safeToAdd, when lsof detects
+a file is still open for write by some process, it cancels
+the add. This relies on events being received when files
+get closed (closingTracked).
+
+In Utility.DirWatcher.FSEvents.watchDir, when an event has
+eventFlagItemModified set, it treats that as a file add event.
+The intent is to emulate inotify's handling of file add events when
+files are closed.
+
+So, two theories:
+
+1. Perhaps eventFlagItemModified only gets set if the file
+   is actually modified. Ie, if MS office writes the file
+   and while it's being written another process opens it to read
+   it (perhaps to index the content), then if the other process
+   doesn't modify it, eventFlagItemModified is not set.
+
+2. Perhaps the way the assistant hard links/moves the file around
+   confuses the FSEvents handling. Perhaps there is an event with
+   eventFlagItemModified, but it's for the locked down file, or
+   something like that, so git-annex ignores it.
+
+In any case, I'm leaning toward thinking that closingTracked should
+not be True for FSEvents. This bug report seems to show, conclusively,
+that FSEvents does not have that property. If closingTracked was False,
+as it is for KQueue, the assistant would postpone adding the file,
+and keep retrying, around once per second, until it no longer had
+any writers, and then add it.
+
+So, I've made that change. I suspect it fixes the bug, but it would
+be pretty hard for me to test it. Could you please download tomorrow's
+daily build of git-annex for OSX, and see if it fixes the problem?
+"""]]
diff --git a/doc/forum/Git-annex_and_Microsoft_Office_files_on_OS_X/comment_1_71fdd5e7061d6deb357290057804cc27._comment b/doc/forum/Git-annex_and_Microsoft_Office_files_on_OS_X/comment_1_71fdd5e7061d6deb357290057804cc27._comment
new file mode 100644
index 000000000..3cd1c5943
--- /dev/null
+++ b/doc/forum/Git-annex_and_Microsoft_Office_files_on_OS_X/comment_1_71fdd5e7061d6deb357290057804cc27._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2017-06-09T17:51:35Z"
+ content="""
+A bug was opened about this:
+[[bugs/Git-annex_and_Microsoft_Office_files_on_OS_X]]
+"""]]