From 0419f241a4ad4a3646b8b3378de611da92891fce Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Tue, 17 Dec 2013 14:07:37 -0400 Subject: [PATCH] I think I've convinced myself that the assistant is safe on windows despite the lack of lsof --- doc/design/assistant/inotify.mdwn | 45 +++++++++++++++++++++++++++++++ doc/todo/windows_support.mdwn | 3 --- 2 files changed, 45 insertions(+), 3 deletions(-) diff --git a/doc/design/assistant/inotify.mdwn b/doc/design/assistant/inotify.mdwn index d1afe63c5f..5e1b4496c2 100644 --- a/doc/design/assistant/inotify.mdwn +++ b/doc/design/assistant/inotify.mdwn @@ -64,8 +64,53 @@ I'd also like to support OSX and if possible the BSDs. * (good example program) * (good example program) + *hfsevents is now supported* + * Windows has a Win32 ReadDirectoryChangesW, and perhaps other things. + It was easy to get watching to work in windows. But there is no lsof, + to check if a file can safely be added. So, need to carefully consider + how to make adding a file safe in windows. + + Without lsof, an InodeCache is generated in "lockdown" (which doesn't + do anything to prevent new writers), and is compared with the stat of the + file after it's ingested (and checksummed). This will detect many changes + to files, which change the size or mtime. + + So, we have 2 cases to worry about. + + 1. A process has the file open for write as it's added, does not change + it until the add is done. + + As long as an event is generated once the file does get closed, this + is fine -- the modified version will be re-added. And such events are + indeed generated on windows. + + 2. A process has the file open for write as it's added, and changes it + in some way that does not affect size or mtime. + + If an event is generated when the file does get closed, this is the + same as a scenario where a process opens the file after it's added, + makes such a change, and closes it. In either case, a file closed + event is generated, and the Watcher will not detect any change + using the inode cache, so will not re-add the file. + + So, this scenario is a potential problem, but it seems at least + unlikely that a program would modify a file without affecting its + mtime. Note that this same scenario can happen even with lsof, and + even on linux (although on linux the InodeCache includes an actual + inode, which might detect the change too). + + Conclusion: It's probably ok to run without lsof on Windows. + + Corrolary: lsof might not generally be needed in direct mode, on + systems that do generate file close events (but not when + eventsCoalesce). + The same arguments given above seem to apply to !Windows. Note that lsof + is needed in indirect mode, as discussed below. + + **windows is now supported** + ## the races Many races need to be dealt with by this code. Here are some of them. diff --git a/doc/todo/windows_support.mdwn b/doc/todo/windows_support.mdwn index 0cfd919b29..fea8241cc4 100644 --- a/doc/todo/windows_support.mdwn +++ b/doc/todo/windows_support.mdwn @@ -12,9 +12,6 @@ now! --[[Joey]] * Ssh connection caching does not work on Windows, so `git annex get` has to connect twice to the remote system over ssh per file, which is much slower than on systems supporting connection caching. -* `git annex watch` works, but does not detect when files are still open - for write as they're being added (no lsof). So watch/webapp/assistant - may be unsafe. * `git annex assistant` has not been tested, is probably quite incomplete and/or buggy. * Doesn't daemonize. Maybe use