Make git-annex copy --from --to --fast actually fast

Eg when the destination is logged as containing a file, skip
actively checking that it does contain it.

Note that --fast does not prevent other verifications of content
location that are done in a copy --from --to. Perhaps it could, but this
change will already avoid the real unnecessary work of operating on
files that are already in the remote.

And avoiding other verifications
might cause it to fail if the location log thinks that --to does not
contain the content but does. Such complications with `git-annex copy
--to remote --fast` led to commit d006586cd0
which added a note that gets displayed when that fails, mentioning it
might be due to --fast being enabled.

copy --from --to is already complicated enough without needing to worry
about such edge cases, so continuing to doing some verification of
content location after the initial --fast filtering seems ok.

Sponsored-by: Dartmouth College's DANDI project
This commit is contained in:
Joey Hess 2023-11-17 17:27:58 -04:00
parent 96c0e720c5
commit 6e3bcbf4dd
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
5 changed files with 30 additions and 1 deletions

View file

@ -19,6 +19,7 @@ git-annex (10.20230927) UNRELEASED; urgency=medium
to tune the output of the above added options. to tune the output of the above added options.
* Fix bug in git-annex copy --from --to that skipped files that were * Fix bug in git-annex copy --from --to that skipped files that were
locally present. locally present.
* Make git-annex copy --from --to --fast actually fast.
-- Joey Hess <id@joeyh.name> Tue, 10 Oct 2023 13:17:31 -0400 -- Joey Hess <id@joeyh.name> Tue, 10 Oct 2023 13:17:31 -0400

View file

@ -345,7 +345,13 @@ fromToStart removewhen afile key ai si src dest =
starting (describeMoveAction removewhen) (OnlyActionOn key ai) si $ starting (describeMoveAction removewhen) (OnlyActionOn key ai) si $
fromToPerform src dest removewhen key afile fromToPerform src dest removewhen key afile
where where
somethingtodo = pure (Remote.uuid src /= Remote.uuid dest) somethingtodo
| Remote.uuid src == Remote.uuid dest = return False
| otherwise = do
fast <- Annex.getRead Annex.fast
if fast && removewhen == RemoveNever
then not <$> expectedPresent dest key
else return True
{- When there is a local copy, transfer it to the dest, and drop from the src. {- When there is a local copy, transfer it to the dest, and drop from the src.
- -

View file

@ -84,3 +84,5 @@ and then in conda with `10.20230626-g801c4b7`
[[!meta author=yoh]] [[!meta author=yoh]]
[[!tag projects/dandi]] [[!tag projects/dandi]]
> [[fixed|done]] --[[Joey]]

View file

@ -0,0 +1,13 @@
[[!comment format=mdwn
username="joey"
subject="""comment 1"""
date="2023-11-17T20:57:19Z"
content="""
> but it doesn't works out correctly whenever there are some files to actually copy
I think that was due to the bug you linked, which is now fixed.
I've confirmed that `--fast` is not actually implemented for `git-annex
copy --from --to`. Explicitly specifying `--not --in destremote` is a
fine workaround. But I've gone ahead and implemented `--fast` for it too.
"""]]

View file

@ -0,0 +1,7 @@
[[!comment format=mdwn
username="joey"
subject="""comment 2"""
date="2023-11-17T21:16:35Z"
content="""
BTW `git-annex find --print0` is the output eqivilant of -z.
"""]]