7c7af82578
Make a pass over the whole exported tree, and upload anything that has not yet reached the export. Update location log when exporting. Note that the synthesized keys for non-annexed files are stored in the location log too. Some cases involving files in the tree with the same content are not handled correctly yet. This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
39 lines
1.7 KiB
Markdown
39 lines
1.7 KiB
Markdown
`git annex export` corresponding to import. This might be useful for eg,
|
|
datalad. There are some requests to make eg a S3 bucket mirror the
|
|
filenames in the git annex repository with incremental updates,
|
|
which seem out of scope (and there are many tools to do stuff like that
|
|
search "deploy files to S3 bucket"),
|
|
but something simpler like `git annex export` could be worth doing.
|
|
|
|
`git annex export --to remote files` would copy the files to the remote,
|
|
using the names in the working tree. For remotes like S3, it could add the
|
|
url of the exported file, so that another clone of the repo could use the
|
|
exported data.
|
|
|
|
Would this be able to reuse the existing `storeKey` interface, or would
|
|
there need to be a new interface in supported remotes?
|
|
|
|
--[[Joey]]
|
|
|
|
Work is in progress. Todo list:
|
|
|
|
* Remember the previously exported tree (in git-annex branch, see design)
|
|
and use to make next export more efficient.
|
|
* Only export to remotes that were initialized to support it.
|
|
* Prevent using export remotes for key/value storage.
|
|
* When exporting, update location tracking to allow getting from exports,
|
|
* Use retrieveExport when getting from export remotes.
|
|
* Efficient handling of renames.
|
|
* Detect export conflicts (see design)
|
|
* Support export to aditional special remotes (S3 etc)
|
|
* Support export to external special remotes.
|
|
* If the same content is present in two different files, export
|
|
location tracking can be messed up.
|
|
|
|
When one of the files is deleted and
|
|
that tree is exported, the location log for the key will be updated
|
|
to say it's not present, even though the other file is still present.
|
|
|
|
And, once one of the files is uploaded, the location log will
|
|
say the content is present, so the pass over the tree won't try to
|
|
upload the other file.
|