sync: Fix locking problems during merge when annex.pidlock is set
Presumably git merge sometimes needs to verifiy if a worktree file is
modified, and so will then run git-annex filter-process which would try to
take the pid lock. And for whatever reason, git-annex sync already had the
pidlock held. I have not replicated that, but it does make enough sense to
deploy the workaround.
Like I said back in commit 7bdb0cdc0d
,
Arguably, it would be better to have a way to make any process git-annex
runs have the env var set. But then it would need to take the pid lock
when running any and all processes, and that would be a problem when
git-annex runs two processes concurrently. So, I'm left doing it ad-hoc
in places where git-annex really does run a child process, directly
or indirectly via a particular git command.
Sponsored-by: KDM on Patreon
This commit is contained in:
parent
37ff9b6401
commit
0485dd3161
4 changed files with 41 additions and 9 deletions
|
@ -3,6 +3,7 @@ git-annex (10.20231130) UNRELEASED; urgency=medium
|
||||||
* Make git-annex get/copy/move --from foo override configuration of
|
* Make git-annex get/copy/move --from foo override configuration of
|
||||||
remote.foo.annex-ignore, as documented.
|
remote.foo.annex-ignore, as documented.
|
||||||
* Support git-annex copy/move --from-anywhere --to remote.
|
* Support git-annex copy/move --from-anywhere --to remote.
|
||||||
|
* sync: Fix locking problems during merge when annex.pidlock is set.
|
||||||
|
|
||||||
-- Joey Hess <id@joeyh.name> Thu, 30 Nov 2023 14:48:12 -0400
|
-- Joey Hess <id@joeyh.name> Thu, 30 Nov 2023 14:48:12 -0400
|
||||||
|
|
||||||
|
|
|
@ -76,6 +76,7 @@ import Annex.TaggedPush
|
||||||
import Annex.CurrentBranch
|
import Annex.CurrentBranch
|
||||||
import Annex.Import
|
import Annex.Import
|
||||||
import Annex.CheckIgnore
|
import Annex.CheckIgnore
|
||||||
|
import Annex.PidLock
|
||||||
import Types.FileMatcher
|
import Types.FileMatcher
|
||||||
import Types.GitConfig
|
import Types.GitConfig
|
||||||
import Types.Availability
|
import Types.Availability
|
||||||
|
@ -352,15 +353,16 @@ mergeConfig mergeunrelated = do
|
||||||
]
|
]
|
||||||
|
|
||||||
merge :: CurrBranch -> [Git.Merge.MergeConfig] -> SyncOptions -> Git.Branch.CommitMode -> [Git.Branch] -> Annex Bool
|
merge :: CurrBranch -> [Git.Merge.MergeConfig] -> SyncOptions -> Git.Branch.CommitMode -> [Git.Branch] -> Annex Bool
|
||||||
merge currbranch mergeconfig o commitmode tomergel = do
|
merge currbranch mergeconfig o commitmode tomergel =
|
||||||
canresolvemerge <- if resolveMergeOverride o
|
runsGitAnnexChildProcessViaGit $ do
|
||||||
then getGitConfigVal annexResolveMerge
|
canresolvemerge <- if resolveMergeOverride o
|
||||||
else return False
|
then getGitConfigVal annexResolveMerge
|
||||||
and <$> case currbranch of
|
else return False
|
||||||
(Just b, Just adj) -> forM tomergel $ \tomerge ->
|
and <$> case currbranch of
|
||||||
mergeToAdjustedBranch tomerge (b, adj) mergeconfig canresolvemerge commitmode
|
(Just b, Just adj) -> forM tomergel $ \tomerge ->
|
||||||
(b, _) -> forM tomergel $ \tomerge ->
|
mergeToAdjustedBranch tomerge (b, adj) mergeconfig canresolvemerge commitmode
|
||||||
autoMergeFrom tomerge b mergeconfig commitmode canresolvemerge
|
(b, _) -> forM tomergel $ \tomerge ->
|
||||||
|
autoMergeFrom tomerge b mergeconfig commitmode canresolvemerge
|
||||||
|
|
||||||
syncBranch :: Git.Branch -> Git.Branch
|
syncBranch :: Git.Branch -> Git.Branch
|
||||||
syncBranch = Git.Ref.underBase "refs/heads/synced" . origBranch
|
syncBranch = Git.Ref.underBase "refs/heads/synced" . origBranch
|
||||||
|
|
|
@ -55,3 +55,5 @@ Could this be caused by e.g. git annex running git merge which runs git annex fi
|
||||||
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
|
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
|
||||||
|
|
||||||
Lots! This problem popped up during our regular use of git-annex in plant genomic research, where we use git annex to manage and move our analyses between the many clusters we must use for computation. Git annex is indispensable for this use case!!
|
Lots! This problem popped up during our regular use of git-annex in plant genomic research, where we use git annex to manage and move our analyses between the many clusters we must use for computation. Git annex is indispensable for this use case!!
|
||||||
|
|
||||||
|
> [[fixed|done]] (for the merge case at least) --[[Joey]]
|
||||||
|
|
|
@ -0,0 +1,27 @@
|
||||||
|
[[!comment format=mdwn
|
||||||
|
username="joey"
|
||||||
|
subject="""comment 1"""
|
||||||
|
date="2023-12-04T17:06:11Z"
|
||||||
|
content="""
|
||||||
|
> Could this be caused by e.g. git annex running git merge which runs git annex
|
||||||
|
> filterprocess (directly or via git status), and git-annex-filterprocess tries
|
||||||
|
> to take the pidlock that git-annex-sync already has?
|
||||||
|
|
||||||
|
If so you should be able to see the two git-annex's running in ps when this hang occurs
|
||||||
|
with the git process in between them.
|
||||||
|
That would be very helpful to verify since it would narrow down the git command
|
||||||
|
that runs this git-annex child process.
|
||||||
|
|
||||||
|
git-annex does deal with exactly this kind of scenario when running eg
|
||||||
|
`git update-index`, by setting an environment variable to tell the child git-annex
|
||||||
|
process not to re-lock the pid lock. But not in eg the case of a git merge.
|
||||||
|
|
||||||
|
So, if you can narrow it down specifically to `git merge` running when this
|
||||||
|
happens, or whatever other command, it should be easy to fix it.
|
||||||
|
|
||||||
|
Hmm, I suppose that it makes sense `git merge` will sometimes need to check if
|
||||||
|
a worktree file is modified, and so run git-annex. I'm pretty convinced by your
|
||||||
|
transcript that it is the git merge that's hanging. So I've gone ahead and put
|
||||||
|
in the workaround for that. There's of course still the possibility there's
|
||||||
|
some other git command I've not anticipated that still needs the workaround.
|
||||||
|
"""]]
|
Loading…
Reference in a new issue