copy/move --from-anywhere --to remote

Implementation was simple because it's equivilant to
--from=foo --to remote for each other remote, followed by
--to remote when there's a local copy.

(Or, in the edge case of --from-anywhere --to=here,
it's the same as --to=here.)

Note that, when the local repo does not have a copy,
fromToPerform gets it from a remote, sends it to the destination,
and drops the local copy. Another call to that for a second remote
will notice that the dest now has a copy, and simply drop from the
second remote, avoiding a second transfer.

Also note that, when numcopies doesn't allow dropping it from
everywhere, it will drop it from the cheapest remotes first
(maybe not ideal) up to more expensive remotes, and finally from the local
repo. So the local repo will generally end up holding a copy. Maybe not
ideal in all cases either, but it seems no worse to do that than to end up
with a copy undropped from a remote.

And I'm not entirely happy with the output, eg:

	copy bigfile (from r3...) ok
	copy bigfile ok

That makes sense if you think of the second line as being
the same as what is output by `git-annex copy bigfile --to bar`,
but it's less clear in this context. Maybe add "(from here...)"?
Also the --json output doesn't have a machine-readable field for
the "from" uuid, and maybe it should?

Sponsored-by: Dartmouth College's DANDI project
This commit is contained in:
Joey Hess 2023-11-30 16:32:32 -04:00
parent 1654572bc1
commit 1e31bf8122
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
7 changed files with 88 additions and 15 deletions

View file

@ -0,0 +1,16 @@
[[!comment format=mdwn
username="joey"
subject="""comment 6"""
date="2023-11-30T18:26:30Z"
content="""
I like the idea of `copy --from-anywhere --to=remote` and just
use the lowest cost remote (when not in local repo). Like `git-annex get`
and `git-annex copy --to=here`.
Hmm, if there's a remote that is too expensive to want to use in such a
copy, it would be possible to use `-c remote.foo.annex-ignore=true`
to make it avoid using that remote. As can also be done in the case of
`git-annex get`, although that was not documented well.
I've implemented --from-anywhere..
"""]]

View file

@ -41,6 +41,11 @@ Paths of files or directories to operate on can be specified.
then deleting the content from the local repository (if it was not present
to start with).
* `--from-anywhere --to=remote`
Copy to the remote files from the local repository as well as from any reachable
remotes.
* `--jobs=N` `-JN`
Enables parallel transfers with up to the specified number of jobs

View file

@ -38,6 +38,11 @@ Paths of files or directories to operate on can be specified.
then deleting the content from the local repository (if it was not present
to start with).
* `--from-anywhere --to=remote`
Move to the remote files from the local repository and from all
reachable remotes.
* `--force`
Override numcopies and required content checking, and always remove