fix fsck --from --all to not fall over trying to check required content

fsck: When --from is used in combination with --all or similar options, do
not verify required content, which can't be checked properly when operating
on keys.

This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.

CHANGELOG

@@ -16,6 +16,9 @@ git-annex (8.20210311) UNRELEASED; urgency=medium
     when it is claimed by a special remote other than the web.
   * Work around some buggy webdav server behavior involving renaming files.
   * Make --debug also enable debugging in child git-annex processes.
+  * fsck: When --from is used in combination with --all or similar options,
+    do not verify required content, which can't be checked properly when
+    operating on keys.
 
  -- Joey Hess <id@joeyh.name>  Fri, 12 Mar 2021 12:06:37 -0400
 

Command/Fsck.hs

@@ -306,25 +306,30 @@ verifyLocationLog' key ai present u updatestatus = do
 {- Verifies that all repos that are required to contain the content do,
  - checking against the location log. -}
 verifyRequiredContent :: Key -> ActionItem -> Annex Bool
-verifyRequiredContent key ai@(ActionItemAssociatedFile afile _) = do
-	requiredlocs <- S.fromList . M.keys <$> requiredContentMap
-	if S.null requiredlocs
-		then return True
-		else do
-			presentlocs <- S.fromList <$> loggedLocations key
-			missinglocs <- filterM
-				(\u -> isRequiredContent (Just u) S.empty (Just key) afile False)
-				(S.toList $ S.difference requiredlocs presentlocs)
-			if null missinglocs
-				then return True
-				else do
-					missingrequired <- Remote.prettyPrintUUIDs "missingrequired" missinglocs
-					warning $
-						"** Required content " ++
-						decodeBS' (actionItemDesc ai) ++
-						" is missing from these repositories:\n" ++
-						missingrequired
-					return False
+verifyRequiredContent key ai@(ActionItemAssociatedFile afile _) = case afile of
+	-- Can't be checked if there's no associated file.
+	AssociatedFile Nothing -> return True
+	AssociatedFile (Just _) -> do
+		requiredlocs <- S.fromList . M.keys <$> requiredContentMap
+		if S.null requiredlocs
+			then return True
+			else go requiredlocs
+  where
+	go requiredlocs = do
+		presentlocs <- S.fromList <$> loggedLocations key
+		missinglocs <- filterM
+			(\u -> isRequiredContent (Just u) S.empty (Just key) afile False)
+			(S.toList $ S.difference requiredlocs presentlocs)
+		if null missinglocs
+			then return True
+			else do
+				missingrequired <- Remote.prettyPrintUUIDs "missingrequired" missinglocs
+				warning $
+					"** Required content " ++
+					decodeBS' (actionItemDesc ai) ++
+					" is missing from these repositories:\n" ++
+					missingrequired
+				return False
 verifyRequiredContent _ _ = return True
 
 {- Verifies the associated file records. -}

doc/bugs/git-annex-fsck__fails_for_missing_dead_keys.mdwn

@@ -39,3 +39,5 @@ NU/Linux
 ### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
 
 Thanks again for git-annex, I keep finding new uses for it.
+
+> [[fixed|done]] --[[Joey]]

doc/bugs/git-annex-fsck__fails_for_missing_dead_keys/comment_1_3fc0d0c2dda1930601a683e16055cf89._comment

@@ -0,0 +1,45 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2021-03-22T18:47:00Z"
+ content="""
+Your transcript does not show the bug, but I was able to reproduce it eventually.
+
+	# git annex required here anything
+	# git annex add foo
+	# git annex drop --force foo
+	# git rm foo
+	# git annex fsck --all
+	fsck SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 
+	  ** No known copies exist of SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
+	
+	  (Avoid this check by running: git annex dead --key )
+	# git-annex dead --key SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
+	# git annex fsck --all
+	fsck SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 
+	  This key is dead, skipping.
+	ok
+	# 
+
+So far this is operating as expected. But then clone this repo to another
+one, and in the other one:
+
+	# git annex fsck --all --from origin
+	fsck SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 
+	  ** Required content SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 is missing from these repositories:
+	  	56b59085-83b0-4b98-82c3-1baacd93d3e2 -- joey@darkstar:/tmp/test [origin]
+	
+	  This key is dead, skipping.
+	failed
+	(recording state in git...)
+	git-annex: fsck: 1 failed
+
+So fsck --from --all is verifying required content when it should not,
+because a) matching required content with --all mostly doesn't make sense
+(considering things like "include=" can be in it and can't be matched)
+and b) it's not done without --all.
+
+(I don't think dead keys are actually involved, I mean if a particular file
+is set as required content of a repo, and is not present in it, fsck should
+complain about that, even if the key is dead.)
+"""]]