drop: Avoid redundant object directory thawing.

Sponsored-by: Dartmouth College's DANDI project
This commit is contained in:
Joey Hess 2025-03-26 11:25:35 -04:00
parent 83163ae08a
commit c39f8a168a
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
3 changed files with 48 additions and 2 deletions

View file

@ -368,8 +368,16 @@ lockContentUsing contentlocker key fallback a = withContentLockFile key $ \mlock
cleanuplockfile lockfile
#endif
cleanuplockfile lockfile = void $ tryNonAsync $ do
thawContentDir lockfile
cleanuplockfile lockfile =
-- Often the content directory will be thawed already,
-- so avoid re-thawing, unless cleanup fails.
tryNonAsync (cleanuplockfile' lockfile) >>= \case
Right () -> return ()
Left _ -> void $ tryNonAsync $ do
thawContentDir lockfile
cleanuplockfile' lockfile
cleanuplockfile' lockfile = do
liftIO $ removeWhenExistsWith removeFile lockfile
cleanObjectDirs lockfile

View file

@ -5,6 +5,7 @@ git-annex (10.20250321) UNRELEASED; urgency=medium
the git-annex branch.
* Fix build without the assistant.
* fsck: Avoid complaining about required content of dead repositories.
* drop: Avoid redundant object directory thawing.
-- Joey Hess <id@joeyh.name> Fri, 21 Mar 2025 12:27:11 -0400

View file

@ -0,0 +1,37 @@
[[!comment format=mdwn
username="joey"
subject="""comment 1"""
date="2025-03-26T14:45:38Z"
content="""
Do you have `annex.freezecontent-command` / `annex.thawcontent-command`
configured for this repo? Asking because I know you use it somewhere,
and some expensive command might explain why thawing takes so long.
As to the content directory perms, `drwxr-sr-x` is weird and the really
weird part of it is the `s`. I don't see how git-annex would ever
set that bit itself.
Here's what's going on with the repeated thawing:
* Before it can drop the content, it has to take the drop lock, so thaws
the directory in order to write the lock file.
* After taking the drop lock, it re-freezes. Of course this is not
necessary, but it avoids a complicated error unwind if it is later unable
to drop.
* Then it thaws in order to delete the object file.
* The final thaw is to delete the drop lock file. While the thaw is in
this situation unncessary, since it's left thawed after deleting the
object file, if the drop had failed it would get to this point with the
directory frozen, but would still want to delete the drop lock file,
and so would need to thaw it.
I have now optimized away that final thaw. I don't think it makes sense to
optimise away the first thaw/freeze cycle, because it would complicate
error unwinding and there is anyway no indication it's slow.
I don't know though, that the 2 minutes between the 2nd and 3rd thawing
lines are caused by the actual chmod calls. Seems unlikely, unless you do
have a thawcontent-command. It seems more likely that deleting the object
file and whatever else happens at that point is what is taking 2 minutes.
I suggest strace.
"""]]