mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

I think I've convinced myself that the assistant is safe on windows despite the lack of lsof

Browse source
This commit is contained in:
Joey Hess 2013-12-17 14:07:37 -04:00
parent cc24f1a376
commit 0419f241a4
2 changed files with 45 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

45
doc/design/assistant/inotify.mdwn
View file

@ -64,8 +64,53 @@ I'd also like to support OSX and if possible the BSDs.
* <http://pypi.python.org/pypi/MacFSEvents/0.2.8> (good example program)
* <https://github.com/gorakhargosh/watchdog/blob/master/src/watchdog/observers/fsevents.py> (good example program)
*hfsevents is now supported*
* Windows has a Win32 ReadDirectoryChangesW, and perhaps other things.
It was easy to get watching to work in windows. But there is no lsof,
to check if a file can safely be added. So, need to carefully consider
how to make adding a file safe in windows.
Without lsof, an InodeCache is generated in "lockdown" (which doesn't
do anything to prevent new writers), and is compared with the stat of the
file after it's ingested (and checksummed). This will detect many changes
to files, which change the size or mtime.
So, we have 2 cases to worry about.
1. A process has the file open for write as it's added, does not change
it until the add is done.
As long as an event is generated once the file does get closed, this
is fine -- the modified version will be re-added. And such events are
indeed generated on windows.
2. A process has the file open for write as it's added, and changes it
in some way that does not affect size or mtime.
If an event is generated when the file does get closed, this is the
same as a scenario where a process opens the file after it's added,
makes such a change, and closes it. In either case, a file closed
event is generated, and the Watcher will not detect any change
using the inode cache, so will not re-add the file.
So, this scenario is a potential problem, but it seems at least
unlikely that a program would modify a file without affecting its
mtime. Note that this same scenario can happen even with lsof, and
even on linux (although on linux the InodeCache includes an actual
inode, which might detect the change too).
Conclusion: It's probably ok to run without lsof on Windows.
Corrolary: lsof might not generally be needed in direct mode, on
systems that do generate file close events (but not when
eventsCoalesce).
The same arguments given above seem to apply to !Windows. Note that lsof
is needed in indirect mode, as discussed below.
**windows is now supported**
## the races
Many races need to be dealt with by this code. Here are some of them.

3
doc/todo/windows_support.mdwn
View file

@ -12,9 +12,6 @@ now! --[[Joey]]
* Ssh connection caching does not work on Windows, so `git annex get`
has to connect twice to the remote system over ssh per file, which
is much slower than on systems supporting connection caching.
* `git annex watch` works, but does not detect when files are still open
for write as they're being added (no lsof). So watch/webapp/assistant
may be unsafe.
* `git annex assistant` has not been tested, is probably quite incomplete
and/or buggy.
* Doesn't daemonize. Maybe use