From 3a71f3a4a955c0334f5f68defecd5995a424ee2d Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Tue, 28 Aug 2018 12:35:15 -0400 Subject: [PATCH] reproduced --- doc/bugs/jobs.mdwn | 2 +- ..._2f37448b1ab618cf485d5335d3b2977d._comment | 25 +++++++++++++------ 2 files changed, 18 insertions(+), 9 deletions(-) diff --git a/doc/bugs/jobs.mdwn b/doc/bugs/jobs.mdwn index 49763134f2..d59574fabc 100644 --- a/doc/bugs/jobs.mdwn +++ b/doc/bugs/jobs.mdwn @@ -78,4 +78,4 @@ operating system: linux x86_64 ### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) -[[!meta title="v6 git annex add -J crash due to index somehow getting locked"]] +[[!meta title="git annex add -J crash due to index somehow getting locked"]] diff --git a/doc/bugs/jobs/comment_1_2f37448b1ab618cf485d5335d3b2977d._comment b/doc/bugs/jobs/comment_1_2f37448b1ab618cf485d5335d3b2977d._comment index 11175b2ecd..5e4b320220 100644 --- a/doc/bugs/jobs/comment_1_2f37448b1ab618cf485d5335d3b2977d._comment +++ b/doc/bugs/jobs/comment_1_2f37448b1ab618cf485d5335d3b2977d._comment @@ -3,14 +3,23 @@ subject="""comment 1""" date="2018-08-28T16:16:03Z" content=""" -Does the .git/index.lock file still exist now that git-annex has exited? +Reproduced by creating 10000 files and running +`git -c annex.queuesize=100 annex add -J8` -Can you provide a log with --debug of the git-annex add command? -Can trim it to only the lines containing xargs. +Apparently it runs two concurrent git adds +when flushing the queue in some circumstances. +The smaller queue size must make it easier to reproduce; +without it all 10000 files get added ok here. -I am pretty sure git-annex only runs one git add at a time, no matter how -many jobs you configure. It seems quite likely that some other git command -was operating in the repository at the same time and locked the index, -or like the error message says, git may have somehow crashed and left -the index.lock behind. +It's not specific to v6 at all. + +Hmm, worker threads don't actually get separate +Annex.repoqueue; a single queue is inherited +from the parent. Two worker threads could try +to flush the shared queue at the same time. + +Since the queue updates are also done non-atomically, +which looks susceptable to races too, +I think each thread needs to get its own queue, but +only one thread ought to be allowed to flush its queue at a time. """]]