use queue when upgrading, flushing every so often

Added a cheap way to query the size of a queue.

runQueueAt is not the default yet only because there may be some code that
expects to be able to queue some suff, do something else, and run the whole
queue at the end.

10240 is an arbitrary size for the queue. If we assume annexed
filenames are between 10 and 255 characters long, then the queue will
build up between 100kb and 2550kb long commands. The max command line
length on linux is somewhere above 20k, so this is a fairly good balance --
the queue will buffer only a few megabytes of stuff and a minimal number
of commands will be run by xargs.

Also, insert queue items strictly, this should save memory.
This commit is contained in:
Joey Hess 2011-03-16 15:10:15 -04:00
parent 0f8edc99ee
commit bc21502b9a
4 changed files with 34 additions and 18 deletions

View file

@ -9,6 +9,7 @@ module GitQueue (
Queue,
empty,
add,
size,
run
) where
@ -31,22 +32,28 @@ data Action = Action {
{- A queue of actions to perform (in any order) on a git repository,
- with lists of files to perform them on. This allows coalescing
- similar git commands. -}
type Queue = M.Map Action [FilePath]
data Queue = Queue Integer (M.Map Action [FilePath])
deriving (Show, Eq)
{- Constructor for empty queue. -}
empty :: Queue
empty = M.empty
empty = Queue 0 M.empty
{- Adds an action to a queue. -}
add :: Queue -> String -> [CommandParam] -> FilePath -> Queue
add queue subcommand params file = M.insertWith (++) action [file] queue
add (Queue n m) subcommand params file = Queue (n + 1) m'
where
action = Action subcommand params
m' = M.insertWith' (++) action [file] m
{- Number of items in a queue. -}
size :: Queue -> Integer
size (Queue n _) = n
{- Runs a queue on a git repository. -}
run :: Git.Repo -> Queue -> IO ()
run repo queue = do
forM_ (M.toList queue) $ uncurry $ runAction repo
run repo (Queue _ m) = do
forM_ (M.toList m) $ uncurry $ runAction repo
return ()
{- Runs an Action on a list of files in a git repository.