assistant: Committing a whole lot of files at once could overflow command-line length limits and cause the commit to fail. This only happened when using the assistant in an indirect mode repository.

Assistant/Threads/Committer.hs

@@ -290,8 +290,12 @@ handleAdds havelsof delayadd cs = returnWhen (null incomplete) $ do
 	-- files. The ls-files is run on a batch of files.
 	findnew [] = return ([], noop)
 	findnew pending@(exemplar:_) = do
-		(newfiles, cleanup) <- liftAnnex $
-			inRepo (Git.LsFiles.notInRepo False $ map changeFile pending)
+		let segments = segmentXargs $ map changeFile pending
+		rs <- liftAnnex $ forM segments $ \fs ->
+			inRepo (Git.LsFiles.notInRepo False fs)
+		let (newfiles, cleanup) = foldl'
+			(\(l1, a1) (l2, a2) -> (l1 ++ l2, a1 >> a2))
+			([], return True) rs
 		-- note: timestamp info is lost here
 		let ts = changeTime exemplar
 		return (map (PendingAddChange ts) newfiles, void $ liftIO cleanup)

debian/changelog

@@ -14,6 +14,9 @@ git-annex (5.20150318) UNRELEASED; urgency=medium
     command that ignored it.)
   * Improve error message when --in @date is used and there is no
     reflog for the git-annex branch.
+  * assistant: Committing a whole lot of files at once could overflow
+    command-line length limits and cause the commit to fail. This
+    only happened when using the assistant in an indirect mode repository.
 
  -- Joey Hess <id@joeyh.name>  Thu, 19 Mar 2015 17:05:32 -0400
 

doc/bugs/assistant_committer_crash_on_OSX_when_adding_lots_of_files.mdwn

@@ -8,3 +8,15 @@ long)
 </pre>
 
 Probably need to tune the command length limit for !linux. --[[Joey]]
+
+> Investigation suggests this is the problem:
+
+	inRepo (Git.LsFiles.notInRepo False $ map changeFile pending)
+
+> If a lot of new files have been added, the `pending` list can be
+> arbitrarily large, and this passes it to git ls-files as parameters.a
+> 
+> It's not the actual commit that fails; that uses Git.Queue and xargs.
+> --[[Joey]] 
+
+>> [[fixed|done]] --[[Joey]]