git-annex/doc/bugs/jobs/comment_1_2f37448b1ab618cf485d5335d3b2977d._comment

[[!comment format=mdwn
 username="joey"
 subject="""comment 1"""
 date="2018-08-28T16:16:03Z"
 content="""
Reproduced by creating 10000 files and running 
`git -c annex.queuesize=100 annex add -J8`

Apparently it runs two concurrent git adds
when flushing the queue in some circumstances.
The smaller queue size must make it easier to reproduce;
without it all 10000 files get added ok here.

It's not specific to v6 at all.

Two worker threads are flushing their queues at the same time.
"""]]
followup 2018-08-28 16:21:23 +00:00			`[[!comment format=mdwn`
			`username="joey"`
			`subject="""comment 1"""`
			`date="2018-08-28T16:16:03Z"`
			`content="""`
reproduced 2018-08-28 16:35:15 +00:00			`Reproduced by creating 10000 files and running`
			`git -c annex.queuesize=100 annex add -J8`
followup 2018-08-28 16:21:23 +00:00
reproduced 2018-08-28 16:35:15 +00:00			`Apparently it runs two concurrent git adds`
			`when flushing the queue in some circumstances.`
			`The smaller queue size must make it easier to reproduce;`
			`without it all 10000 files get added ok here.`
followup 2018-08-28 16:21:23 +00:00
reproduced 2018-08-28 16:35:15 +00:00			`It's not specific to v6 at all.`

fix git command queue to be concurrency safe Probably not noticed until now because the queue is large enough that two threads each filling theirs at the same time and flushing is unlikely to happen. Also made explicit that each worker thread gets its own queue. I think that was the case before, but if something was put in the queue before worker threads were forked off, they could have each inherited the same queue. Could have gone with a single shared queue, but per-worker queues is more efficient, because a worker can add lots of stuff to its own queue without any locking. This commit was sponsored by Ole-Morten Duesund on Patreon. 2018-08-28 17:14:44 +00:00			`Two worker threads are flushing their queues at the same time.`
followup 2018-08-28 16:21:23 +00:00			`"""]]`