Fix truncation of parameters that could occur when using xargs git-annex.

This will only ever result in a few more git-ls-files being run than were run
before. (Only 1 more is really needed, but around 10 more are currently run
for a max length command line.)

So, no need to worry about the extra zombie, or lost laziness due to concat.
This commit is contained in:
Joey Hess 2015-04-02 01:18:00 -04:00
parent bf563d5df0
commit 8aa6b5f2a6
3 changed files with 5 additions and 2 deletions

View file

@ -215,8 +215,8 @@ seekActions gen = do
seekHelper :: ([FilePath] -> Git.Repo -> IO ([FilePath], IO Bool)) -> [FilePath] -> Annex [FilePath] seekHelper :: ([FilePath] -> Git.Repo -> IO ([FilePath], IO Bool)) -> [FilePath] -> Annex [FilePath]
seekHelper a params = do seekHelper a params = do
ll <- inRepo $ \g -> ll <- inRepo $ \g -> concat <$> forM (segmentXargsOrdered params)
runSegmentPaths (\fs -> Git.Command.leaveZombie <$> a fs g) params (runSegmentPaths (\fs -> Git.Command.leaveZombie <$> a fs g))
forM_ (map fst $ filter (null . snd) $ zip params ll) $ \p -> forM_ (map fst $ filter (null . snd) $ zip params ll) $ \p ->
unlessM (isJust <$> liftIO (catchMaybeIO $ getSymbolicLinkStatus p)) $ unlessM (isJust <$> liftIO (catchMaybeIO $ getSymbolicLinkStatus p)) $
error $ p ++ " not found" error $ p ++ " not found"

1
debian/changelog vendored
View file

@ -24,6 +24,7 @@ git-annex (5.20150328) UNRELEASED; urgency=medium
for distributed fsck. for distributed fsck.
* Significantly sped up processing of large numbers of directories * Significantly sped up processing of large numbers of directories
passed to a single git-annex command. passed to a single git-annex command.
* Fix truncation of parameters that could occur when using xargs git-annex.
-- Joey Hess <id@joeyh.name> Fri, 27 Mar 2015 16:04:43 -0400 -- Joey Hess <id@joeyh.name> Fri, 27 Mar 2015 16:04:43 -0400

View file

@ -5,6 +5,8 @@ Feeding git-annex a long list off directories, eg with xargs can have
ls-files command is longer than the git-annex command often, so it gets ls-files command is longer than the git-annex command often, so it gets
truncated and some files are not processed. truncated and some files are not processed.
> [[fixed|done]] --[[Joey]]
* It can take a really long time for git-annex to chew through the * It can take a really long time for git-annex to chew through the
git-ls-files results. There is probably an exponential blowup in the time git-ls-files results. There is probably an exponential blowup in the time
relative to the number of parameters. Some of the stuff being done to relative to the number of parameters. Some of the stuff being done to