diff --git a/CHANGELOG b/CHANGELOG index d84b3cb8c4..305970f841 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -27,6 +27,9 @@ git-annex (6.20170520) UNRELEASED; urgency=medium into a repository watched by the assistant auto-merge. * Makefile: Install completions for the fish and zsh shells when git-annex is built with optparse-applicative-0.14. + * assistant: Don't trust OSX FSEvents's eventFlagItemModified to be called + when the last writer of a file closes it; apparently that sometimes + does not happen, which prevented files from being quickly added. -- Joey Hess Wed, 24 May 2017 14:03:40 -0400 diff --git a/Utility/DirWatcher.hs b/Utility/DirWatcher.hs index bde7106265..892841f93f 100644 --- a/Utility/DirWatcher.hs +++ b/Utility/DirWatcher.hs @@ -64,18 +64,18 @@ eventsCoalesce = error "eventsCoalesce not defined" {- With inotify, file closing is tracked to some extent, so an add event - will always be received for a file once its writer closes it, and - (typically) not before. This may mean multiple add events for the same file. - - - - fsevents behaves similarly, although different event types are used for - - creating and modification of the file. - - OTOH, with kqueue, add events will often be received while a file is - still being written to, and then no add event will be received once the - - writer closes it. -} + - writer closes it. + - + - fsevents sometimes behaves similarly, but has sometimes been + - seen to behave like kqueue. -} closingTracked :: Bool -#if (WITH_INOTIFY || WITH_FSEVENTS || WITH_WIN32NOTIFY) +#if (WITH_INOTIFY || WITH_WIN32NOTIFY) closingTracked = True #else -#if WITH_KQUEUE +#if (WITH_KQUEUE || WITH_FSEVENTS) closingTracked = False #else closingTracked = error "closingTracked not defined" diff --git a/doc/bugs/Git-annex_and_Microsoft_Office_files_on_OS_X/comment_1_0cf11096ceeb6cf93db5609a42a70641._comment b/doc/bugs/Git-annex_and_Microsoft_Office_files_on_OS_X/comment_1_0cf11096ceeb6cf93db5609a42a70641._comment new file mode 100644 index 0000000000..ae27704b28 --- /dev/null +++ b/doc/bugs/Git-annex_and_Microsoft_Office_files_on_OS_X/comment_1_0cf11096ceeb6cf93db5609a42a70641._comment @@ -0,0 +1,41 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 1""" + date="2017-06-09T17:52:00Z" + content=""" +This must have to do with the fsevents interface used on OSX. + +In Assistant.Threads.Committer.safeToAdd, when lsof detects +a file is still open for write by some process, it cancels +the add. This relies on events being received when files +get closed (closingTracked). + +In Utility.DirWatcher.FSEvents.watchDir, when an event has +eventFlagItemModified set, it treats that as a file add event. +The intent is to emulate inotify's handling of file add events when +files are closed. + +So, two theories: + +1. Perhaps eventFlagItemModified only gets set if the file + is actually modified. Ie, if MS office writes the file + and while it's being written another process opens it to read + it (perhaps to index the content), then if the other process + doesn't modify it, eventFlagItemModified is not set. + +2. Perhaps the way the assistant hard links/moves the file around + confuses the FSEvents handling. Perhaps there is an event with + eventFlagItemModified, but it's for the locked down file, or + something like that, so git-annex ignores it. + +In any case, I'm leaning toward thinking that closingTracked should +not be True for FSEvents. This bug report seems to show, conclusively, +that FSEvents does not have that property. If closingTracked was False, +as it is for KQueue, the assistant would postpone adding the file, +and keep retrying, around once per second, until it no longer had +any writers, and then add it. + +So, I've made that change. I suspect it fixes the bug, but it would +be pretty hard for me to test it. Could you please download tomorrow's +daily build of git-annex for OSX, and see if it fixes the problem? +"""]] diff --git a/doc/forum/Git-annex_and_Microsoft_Office_files_on_OS_X/comment_1_71fdd5e7061d6deb357290057804cc27._comment b/doc/forum/Git-annex_and_Microsoft_Office_files_on_OS_X/comment_1_71fdd5e7061d6deb357290057804cc27._comment new file mode 100644 index 0000000000..3cd1c5943c --- /dev/null +++ b/doc/forum/Git-annex_and_Microsoft_Office_files_on_OS_X/comment_1_71fdd5e7061d6deb357290057804cc27._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 1""" + date="2017-06-09T17:51:35Z" + content=""" +A bug was opened about this: +[[bugs/Git-annex_and_Microsoft_Office_files_on_OS_X]] +"""]]