fix git command queue to be concurrency safe

Probably not noticed until now because the queue is large enough that two
threads each filling theirs at the same time and flushing is unlikely to
happen.

Also made explicit that each worker thread gets its own queue.
I think that was the case before, but if something was put in the queue
before worker threads were forked off, they could have each inherited the
same queue.

Could have gone with a single shared queue, but per-worker queues is more
efficient, because a worker can add lots of stuff to its own queue without
any locking.

This commit was sponsored by Ole-Morten Duesund on Patreon.
This commit is contained in:
Joey Hess 2018-08-28 13:14:44 -04:00
parent 3a71f3a4a9
commit 759a87ad70
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
6 changed files with 31 additions and 13 deletions

View file

@ -23,6 +23,8 @@ import Annex hiding (new)
import qualified Git.Queue
import qualified Git.UpdateIndex
import qualified Control.Concurrent.SSem as SSem
{- Adds a git command to the queue. -}
addCommand :: String -> [CommandParam] -> [FilePath] -> Annex ()
addCommand command params files = do
@ -56,10 +58,23 @@ flush = do
unless (0 == Git.Queue.size q) $ do
store =<< flush' q
{- When there are multiple worker threads, each has its own queue.
-
- But, flushing two queues at the same time could lead to failures due to
- git locking files. So, only one queue is allowed to flush at a time.
- The repoqueuesem is shared between threads.
-}
flush' :: Git.Queue.Queue -> Annex Git.Queue.Queue
flush' q = do
showStoringStateAction
inRepo $ Git.Queue.flush q
flush' q = bracket lock unlock go
where
lock = do
s <- getState repoqueuesem
liftIO $ SSem.wait s
return s
unlock = liftIO . SSem.signal
go _ = do
showStoringStateAction
inRepo $ Git.Queue.flush q
{- Gets the size of the queue. -}
size :: Annex Int