mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

protocol design

Browse source
This commit is contained in:
Joey Hess 2017-07-12 12:43:46 -04:00
parent 905b1108b7
commit aa7cc67a3d
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
1 changed files with 48 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

50
doc/design/exporting_trees_to_special_remotes.mdwn
View file

@ -4,6 +4,8 @@ and content from the tree.
(See also [[todo/export]] and [[todo/dumb, unsafe, human-readable_backend]])
[[!toc ]]
## configuring a special remote for tree export
If a special remote already has files stored in it, switching it to be a
@ -62,6 +64,11 @@ To efficiently update an export, git-annex can diff the tree
that was exported with the new tree. The naive approach is to upload
new and modified files and remove deleted files.
Note that a file may have been partially uploaded to an export, and then
the export updated to a tree without that file. So, need to try to delete
all removed files, even if location tracking does not say that the special
remote contains them.
With rename detection, if the special remote supports moving files,
more efficient updates can be done. It gets complicated; consider two files
that swap names.
@ -93,7 +100,45 @@ if the file instead still has the old key's content. Instead, the whole
file needs to be re-uploaded.
Alternative: Keep an index file that's the current state of the export.
See comment #4 of [[todo/export]]. Not sure if that works?
See comment #4 of [[todo/export]]. Not sure if that works? Perhaps it
would be overkill if it's only used to support resuming partial uploads.
## changes to special remote interface
This needs some additional methods added to special remotes, and to
the [[external_special_remote_protocol]].
* `TRANSFEREXPORT STORE|RETRIEVE Key File Name`
Requests the transfer of a File on local disk to or from a given
Name on the special remote.
The Name will be in the form of a relative path, and may contain
path separators, whitespace, and other special characters.
The Key is provided in case the special remote wants to use eg
`SETURIPRESENT`.
The remote responds with either `TRANSFER-SUCCESS` or
`TRANSFER-FAILURE`, and a remote where exports do not make sense
may always fail.
* `CHECKPRESENTEXPORT Key Name`
Requests the remote to check if a Name is present in it.
The remote responds with `CHECKPRESENT-SUCCESS`, `CHECKPRESENT-FAILURE`,
or `CHECKPRESENT-UNKNOWN`.
* `REMOVEEXPORT Key Name`
Requests the remote to remove content stored by `TRANSFEREXPORT`.
The Key is provided in case the remote wants to use eg
`SETURIMISSING`.
The remote responds with either `REMOVE-SUCCESS` or
`REMOVE-FAILURE`.
* `RENAMEEXPORT Key OldName NewName`
Requests the remote rename a file stored on it from OldName to NewName.
The Key is provided in case the remote wants to use eg
`SETURIMISSING` and `SETURIPRESENT`.
The remote responds with `RENAMEEXPORT-SUCCESS,
`RENAMEEXPORT-FAILURE`, or with `RENAMEEXPORT-UNSUPPORTED` if an efficient
rename cannot be done.
To support old external special remote programs that have not been updated
to support exports, git-annex will need to handle an `ERROR` response
when using any of the above.
## location tracking
@ -141,7 +186,8 @@ find what changes need to be made to the special remote.
If the filenames are stored in the location tracking log, the exported tree
could be reconstructed, but it would take O(N) queries to git, where N is
the total number of keys git-annex knows about; updating exports of small
subsets of large repositories would be expensive.
subsets of large repositories would be expensive. So grafting in the
exported tree seems the better approach.
## export conflicts