Keys marked as dead are now skipped by --all.

fsck already special-cased dead keys to make --all not report errors with
them, and it makes sense to also expand that to whereis. I think it makes
sense for dead keys to be skipped by all uses of --all, so mistakes can be
completely forgotten about and not come back to haunt us.

The speed impact of testing if the key is dead is negligible for fsck and
whereis, since they use the location log anyway and it gets cached.
This does slow down a few commands that support --all, in particular
metadata --all runs around 2x as slow. I don't think metadata
--all is often used though. It might slow down copy/move/mirror
--all and get --all.
log --all is not affected (does not use the normal --all machinery).

Dead keys will still be processed by --incomplete, --branch,
--failed, and --key. Although it would be unlikely for a dead key to
ave in incomplete or failed transfer. It seems to make perfect sense for
--branch to process keys on the branch, even if dead.

(fsck's special-casing of dead keys was left in, so if one of these options
causes a dead key to be fscked, there will be a nice message.)

This commit was supported by the NSF-funded DataLad project.

CHANGELOG

@@ -21,6 +21,7 @@ git-annex (6.20170322) UNRELEASED; urgency=medium
     This was never supported before.
   * git annex add -u now supported, analagous to git add -u
   * version: Added "dependency versions" line.
+  * Keys marked as dead are now skipped by --all.
 
  -- Joey Hess <id@joeyh.name>  Wed, 29 Mar 2017 12:41:46 -0400
 

Logs/Location.hs

@@ -86,7 +86,7 @@ checkDead :: Key -> Annex Bool
 checkDead key = do
 	config <- Annex.getGitConfig
 	ls <- compactLog <$> readLog (locationLogFile config key)
-	return $ all (\l -> status l == InfoDead) ls
+	return $! all (\l -> status l == InfoDead) ls
 
 {- Updates the log to say that a key is dead. 
  - 
@@ -111,17 +111,24 @@ setDead' l = l
 	}
 
 {- Finds all keys that have location log information.
- - (There may be duplicate keys in the list.) -}
+ - (There may be duplicate keys in the list.)
+ -
+ - Keys that have been marked as dead are not included.
+ -}
 loggedKeys :: Annex [Key]
-loggedKeys = mapMaybe locationLogFileKey <$> Annex.Branch.files
+loggedKeys = loggedKeys' (not <$$> checkDead)
+
+{- Note that sel should be strict, to avoid the filterM building many
+ - thunks. -} 
+loggedKeys' :: (Key -> Annex Bool) -> Annex [Key]
+loggedKeys' sel = filterM sel =<<
+	(mapMaybe locationLogFileKey <$> Annex.Branch.files)
 
 {- Finds all keys that have location log information indicating
  - they are present for the specified repository. -}
 loggedKeysFor :: UUID -> Annex [Key]
-loggedKeysFor u = filterM isthere =<< loggedKeys
+loggedKeysFor u = loggedKeys' isthere
   where
-	{- This should run strictly to avoid the filterM
-	 - building many thunks containing keyLocations data. -}
 	isthere k = do
 		us <- loggedLocations k
 		let !there = u `elem` us

doc/git-annex-dead.mdwn

@@ -17,8 +17,9 @@ Repositories can be specified using their remote name, their
 description, or their UUID. (To undo, use `git-annex semitrust`.)
 
 When a key is specified, indicates that the content of that key has been
-irretrievably lost. This prevents `git annex fsck --all` from complaining
-about it. (To undo, add the key's content back to the repository,
+irretrievably lost. This prevents commands like `git annex fsck --all`
+from complaining about it; `--all` will not operate on the key anymore.
+(To undo, add the key's content back to the repository, 
 by using eg, `git-annex reinject`.)
 
 # SEE ALSO