Merge branch 'master' into assistant

doc/design/assistant/inotify.mdwn

@@ -18,6 +18,17 @@ available!
   I may need to fork off multiple watcher processes to handle this.
   See [[bugs/Issue_on_OSX_with_some_system_limits]].
 
+## todo
+
+* Run niced and ioniced? Seems to make sense, this is a background job.
+* configurable option to only annex files meeting certian size or
+  filename criteria
+* option to check files not meeting annex criteria into git directly,
+  automatically
+* honor .gitignore, not adding files it excludes (difficult, probably
+  needs my own .gitignore parser to avoid excessive running of git commands
+  to check for ignored files)
+
 ## beyond Linux
 
 I'd also like to support OSX and if possible the BSDs.
@@ -65,17 +76,6 @@ I'd also like to support OSX and if possible the BSDs.
 
 * Windows has a Win32 ReadDirectoryChangesW, and perhaps other things.
 
-## todo
-
-- Run niced and ioniced? Seems to make sense, this is a background job.
-- configurable option to only annex files meeting certian size or
-  filename criteria
-- option to check files not meeting annex criteria into git directly,
-  automatically
-- honor .gitignore, not adding files it excludes (difficult, probably
-  needs my own .gitignore parser to avoid excessive running of git commands
-  to check for ignored files)
-
 ## the races
 
 Many races need to be dealt with by this code. Here are some of them.

doc/design/assistant/syncing.mdwn

@@ -1,23 +1,25 @@
 Once files are added (or removed or moved), need to send those changes to
 all the other git clones, at both the git level and the key/value level.
 
-## action items
+## immediate action items
 
 * Check that download transfer triggering code works (when a symlink appears
   and the remote does *not* upload to us.
-* Investigate why transfers seem to block other git-annex assistant work.
 * At startup, and possibly periodically, look for files we have that
   location tracking indicates remotes do not, and enqueue Uploads for
   them. Also, enqueue Downloads for any files we're missing.
-* Find a way to probe available outgoing bandwidth, to throttle so
-  we don't bufferbloat the network to death.
-* git-annex needs a simple speed control knob, which can be plumbed
-  through to, at least, rsync. A good job for an hour in an
-  airport somewhere.
-* file transfer processes are not waited for, contain the zombies.
+* The TransferWatcher does not notice ongoing transfers, because inotify is
+  waiting for the info file to be closed, but that never happens, it's left
+  open to keep it locked. May need to separate the transfer info files
+  into an info file, and a lock file.
 
 ## longer-term TODO
 
+* git-annex needs a simple speed control knob, which can be plumbed
+  through to, at least, rsync. A good job for an hour in an
+  airport somewhere.
+* Find a way to probe available outgoing bandwidth, to throttle so
+  we don't bufferbloat the network to death.
 * Investigate the XMPP approach like dvcs-autosync does, or other ways of
    signaling a change out of band.
 * Add a hook, so when there's a change to sync, a program can be run