git-annex/doc/bugs/jobs/comment_1_2f37448b1ab618cf485d5335d3b2977d._comment
Joey Hess 759a87ad70
fix git command queue to be concurrency safe
Probably not noticed until now because the queue is large enough that two
threads each filling theirs at the same time and flushing is unlikely to
happen.

Also made explicit that each worker thread gets its own queue.
I think that was the case before, but if something was put in the queue
before worker threads were forked off, they could have each inherited the
same queue.

Could have gone with a single shared queue, but per-worker queues is more
efficient, because a worker can add lots of stuff to its own queue without
any locking.

This commit was sponsored by Ole-Morten Duesund on Patreon.
2018-08-28 13:16:33 -04:00

17 lines
495 B
Text

[[!comment format=mdwn
username="joey"
subject="""comment 1"""
date="2018-08-28T16:16:03Z"
content="""
Reproduced by creating 10000 files and running
`git -c annex.queuesize=100 annex add -J8`
Apparently it runs two concurrent git adds
when flushing the queue in some circumstances.
The smaller queue size must make it easier to reproduce;
without it all 10000 files get added ok here.
It's not specific to v6 at all.
Two worker threads are flushing their queues at the same time.
"""]]