reproduced

doc/bugs/jobs.mdwn

@@ -78,4 +78,4 @@ operating system: linux x86_64
 
 ### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
 
-[[!meta title="v6 git annex add -J crash due to index somehow getting locked"]]
+[[!meta title="git annex add -J crash due to index somehow getting locked"]]

doc/bugs/jobs/comment_1_2f37448b1ab618cf485d5335d3b2977d._comment

@@ -3,14 +3,23 @@
  subject="""comment 1"""
  date="2018-08-28T16:16:03Z"
  content="""
-Does the .git/index.lock file still exist now that git-annex has exited?
+Reproduced by creating 10000 files and running 
+`git -c annex.queuesize=100 annex add -J8`
 
-Can you provide a log with --debug of the git-annex add command?
+Apparently it runs two concurrent git adds
-Can trim it to only the lines containing xargs.
+when flushing the queue in some circumstances.
+The smaller queue size must make it easier to reproduce;
+without it all 10000 files get added ok here.
 
-I am pretty sure git-annex only runs one git add at a time, no matter how
+It's not specific to v6 at all.
-many jobs you configure. It seems quite likely that some other git command
+
-was operating in the repository at the same time and locked the index,
+Hmm, worker threads don't actually get separate
-or like the error message says, git may have somehow crashed and left
+Annex.repoqueue; a single queue is inherited
-the index.lock behind.
+from the parent. Two worker threads could try
+to flush the shared queue at the same time.
+
+Since the queue updates are also done non-atomically,
+which looks susceptable to races too,
+I think each thread needs to get its own queue, but
+only one thread ought to be allowed to flush its queue at a time.
 """]]