gitignore support for the assistant and watcher

Requires git 1.8.4 or newer. When it's installed, a background
git check-ignore process is run, and used to efficiently check ignores
whenever a new file is added.

Thanks to Adam Spiers, for getting the necessary support into git for this.

A complication is what to do about files that are gitignored but have
been checked into git anyway. git commands assume the ignore has been
overridden in this case, and not need any more overriding to commit a
changed version.

However, for the assistant to do the same, it would have to run git ls-files
to check if the ignored file is in git. This is somewhat expensive. Or it
could use the running git-cat-file process to query the file that way,
but that requires transferring the whole file content over a pipe, so it
can be quite expensive too, for files that are not git-annex
symlinks.

Now imagine if the user knows that a file or directory tree will be getting
frequent changes, and doesn't want the assistant to sync it, so gitignores
it. The assistant could overload the system with repeated ls-files checks!

So, I've decided that the assistant will not automatically commit changes
to files that are gitignored. This is a tradeoff. Hopefully it won't be a
problem to adjust .gitignore settings to not ignore files you want the
assistant to autocommit, or to manually git annex add files that are listed
in .gitignore.

(This could be revisited if git-annex gets access to an interface to check
the content of the index w/o forking a git command. This could be libgit2,
or perhaps a separate git cat-file --batch-check process, so it wouldn't
need to ship over the whole file content.)

This commit was sponsored by Francois Marier. Thanks!

Assistant/Threads/Watcher.hs

@@ -1,11 +1,11 @@
 {- git-annex assistant tree watcher
  -
- - Copyright 2012 Joey Hess <joey@kitenet.net>
+ - Copyright 2012-2013 Joey Hess <joey@kitenet.net>
  -
  - Licensed under the GNU GPL version 3 or higher.
  -}
 
-{-# LANGUAGE DeriveDataTypeable, CPP #-}
+{-# LANGUAGE DeriveDataTypeable, BangPatterns, CPP #-}
 
 module Assistant.Threads.Watcher (
 	watchThread,
@@ -33,6 +33,7 @@ import qualified Backend
 import Annex.Direct
 import Annex.Content.Direct
 import Annex.CatFile
+import Annex.CheckIgnore
 import Annex.Link
 import Annex.FileMatcher
 import Annex.ReplaceFile
@@ -141,6 +142,8 @@ startupScan scanner = do
 
 		return (True, r)
 
+{- Hardcoded ignores, passed to the DirWatcher so it can avoid looking
+ - at the entire .git directory. Does not include .gitignores. -}
 ignored :: FilePath -> Bool
 ignored = ig . takeFileName
   where
@@ -152,6 +155,12 @@ ignored = ig . takeFileName
 #endif
 	ig _ = False
 
+unlessIgnored :: FilePath -> Assistant (Maybe Change) -> Assistant (Maybe Change)
+unlessIgnored file a = ifM (liftAnnex $ checkIgnored file)
+	( noChange
+	, a
+	)
+
 type Handler = FilePath -> Maybe FileStatus -> Assistant (Maybe Change)
 
 {- Runs an action handler, and if there was a change, adds it to the ChangeChan.
@@ -186,7 +195,9 @@ add bigfilematcher file = ifM (liftAnnex $ checkFileMatcher bigfilematcher file)
 
 onAdd :: FileMatcher -> Handler
 onAdd matcher file filestatus
-	| maybe False isRegularFile filestatus = add matcher file
+	| maybe False isRegularFile filestatus =
+		unlessIgnored file $
+			add matcher file
 	| otherwise = noChange
 
 {- In direct mode, add events are received for both new files, and
@@ -214,9 +225,10 @@ onAddDirect symlinkssupported matcher file fs = do
 					liftAnnex $ changedDirect key file
 					add matcher file
 				)
-		_ -> guardSymlinkStandin Nothing $ do
-			debug ["add direct", file]
-			add matcher file
+		_ -> unlessIgnored file $
+			guardSymlinkStandin Nothing $ do
+				debug ["add direct", file]
+				add matcher file
   where
  	{- On a filesystem without symlinks, we'll get changes for regular
 	 - files that git uses to stand-in for symlinks. Detect when
@@ -240,7 +252,7 @@ onAddDirect symlinkssupported matcher file fs = do
  - before adding it.
  -}
 onAddSymlink :: Bool -> Handler
-onAddSymlink isdirect file filestatus = do
+onAddSymlink isdirect file filestatus = unlessIgnored file $ do
 	linktarget <- liftIO (catchMaybeIO $ readSymbolicLink file)
 	kv <- liftAnnex (Backend.lookupFile file)
 	onAddSymlink' linktarget (fmap fst kv) isdirect file filestatus