cat-file resource pool

Avoid running a large number of git cat-file child processes when run with
a large -J value.

This implementation takes care to avoid adding any overhead to git-annex
when run without -J. When run with -J, there is a small bit of added
overhead, to manipulate the resource pool. That optimisation added a
fair bit of complexity.
This commit is contained in:
Joey Hess 2020-04-20 13:53:27 -04:00
parent 87b7b0f202
commit cee6b344b4
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
14 changed files with 243 additions and 47 deletions

View file

@ -199,8 +199,7 @@ startConcurrency :: UsedStages -> Annex a -> Annex a
startConcurrency usedstages a = do
fromcmdline <- Annex.getState Annex.concurrency
fromgitcfg <- annexJobs <$> Annex.getGitConfig
let usegitcfg = Annex.changeState $
\c -> c { Annex.concurrency = fromgitcfg }
let usegitcfg = setConcurrency fromgitcfg
case (fromcmdline, fromgitcfg) of
(NonConcurrent, NonConcurrent) -> a
(Concurrent n, _) ->