diff options
author | Joey Hess <joey@kitenet.net> | 2012-06-15 22:59:32 -0400 |
---|---|---|
committer | Joey Hess <joey@kitenet.net> | 2012-06-15 22:59:32 -0400 |
commit | bd8319e78ccf13d52ecf14b4f0c86ebf141671ab (patch) | |
tree | eda188443fb35bd15c28d97c3d4f29ac36b06135 /doc/design/assistant/inotify.mdwn | |
parent | af7b6319d725e8090b32f74b397a9eba79e22978 (diff) |
update and blog for the day
the last of the bad bugs is fixed!
Diffstat (limited to 'doc/design/assistant/inotify.mdwn')
-rw-r--r-- | doc/design/assistant/inotify.mdwn | 74 |
1 files changed, 37 insertions, 37 deletions
diff --git a/doc/design/assistant/inotify.mdwn b/doc/design/assistant/inotify.mdwn index baa420b4e..0b0eb430c 100644 --- a/doc/design/assistant/inotify.mdwn +++ b/doc/design/assistant/inotify.mdwn @@ -5,43 +5,6 @@ There is a `watch` branch in git that adds the command. ## known bugs -* A process has a file open for write, another one closes it, - and so it's added. Then the first process modifies it. - - Or, a process has a file open for write when `git annex watch` starts - up, it will be added to the annex. If the process later continues - writing, it will change content in the annex. - - This changes content in the annex, and fsck will later catch - the inconsistency. - - Possible fixes: - - * Somehow track or detect if a file is open for write by any processes. - `lsof` could be used, although it would be a little slow. - - Here's one way to avoid the slowdown: When a file is being added, - set it read-only, and hard-link it into a quarantine directory, - remembering both filenames. - Then use the batch change mode code to detect batch adds and bundle - them together. - Just before committing, lsof the quarantine directory. Any files in - it that are still open for write can just have their write bit turned - back on and be deleted from quarantine, to be handled when their writer - closes. Files that pass quarantine get added as usual. This avoids - repeated lsof calls slowing down adds, but does add a constant factor - overhead (0.25 seconds lsof call) before any add gets committed. - - * Or, when possible, making a copy on write copy before adding the file - would avoid this. - * Or, as a last resort, make an expensive copy of the file and add that. - * Tracking file opens and closes with inotify could tell if any other - processes have the file open. But there are problems.. It doesn't - seem to differentiate between files opened for read and for write. - And there would still be a race after the last close and before it's - injected into the annex, where it could be opened for write again. - Would need to detect that and undo the annex injection or something. - * If a file is checked into git as a normal file and gets modified (or merged, etc), it will be converted into an annexed file. See [[blog/day_7__bugfixes]] @@ -140,3 +103,40 @@ Many races need to be dealt with by this code. Here are some of them. - coleasce related add/rm events for speed and less disk IO **done** - don't annex `.gitignore` and `.gitattributes` files **done** - run as a daemon **done** +- A process has a file open for write, another one closes it, + and so it's added. Then the first process modifies it. + + Or, a process has a file open for write when `git annex watch` starts + up, it will be added to the annex. If the process later continues + writing, it will change content in the annex. + + This changes content in the annex, and fsck will later catch + the inconsistency. + + Possible fixes: + + * Somehow track or detect if a file is open for write by any processes. + `lsof` could be used, although it would be a little slow. + + Here's one way to avoid the slowdown: When a file is being added, + set it read-only, and hard-link it into a quarantine directory, + remembering both filenames. + Then use the batch change mode code to detect batch adds and bundle + them together. + Just before committing, lsof the quarantine directory. Any files in + it that are still open for write can just have their write bit turned + back on and be deleted from quarantine, to be handled when their writer + closes. Files that pass quarantine get added as usual. This avoids + repeated lsof calls slowing down adds, but does add a constant factor + overhead (0.25 seconds lsof call) before any add gets committed. **done** + + * Or, when possible, making a copy on write copy before adding the file + would avoid this. + * Or, as a last resort, make an expensive copy of the file and add that. + * Tracking file opens and closes with inotify could tell if any other + processes have the file open. But there are problems.. It doesn't + seem to differentiate between files opened for read and for write. + And there would still be a race after the last close and before it's + injected into the annex, where it could be opened for write again. + Would need to detect that and undo the annex injection or something. + |