mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

upload fanout resume seems free of fenceposts

Browse source 
Tested it with small chunk sizes (like 2) and resumes that were
eg 1 byte from the end of the file or beginning of file.

Also, git-annex testremote passes now against a cluster!
This commit is contained in:
Joey Hess 2024-06-23 16:22:39 -04:00
parent 8a341cd195
commit 5d8bdac38e
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
1 changed files with 10 additions and 11 deletions
Download patch file Download diff file Expand all files Collapse all files

21
doc/todo/git-annex_proxies.mdwn
View file

@ -55,18 +55,12 @@ For June's work on [[design/passthrough_proxy]], implementation plan:
* Getting a key from a cluster should proxy from one of the nodes that has * Getting a key from a cluster should proxy from one of the nodes that has
it. (done) it. (done)
* Implement upload with fanout and reporting back additional UUIDs over P2P
protocol. (done)
* Implement cluster drops, trying to remove from all nodes, and returning * Implement cluster drops, trying to remove from all nodes, and returning
which UUIDs it was dropped from. (done) which UUIDs it was dropped from. (done)
* Getting a key from a cluster currently always selects the lowest cost
remote, and always the same remote if cost is the same. Should
round-robin amoung remotes, and prefer to avoid using remotes that
other git-annex processes are currently using.
* Implement upload with fanout and reporting back additional UUIDs over P2P
protocol. (done, but need to check for fencepost errors on resume of
incomplete upload with remotes at different points)
* `git-annex sync --content` does not send content to clusters, or to * `git-annex sync --content` does not send content to clusters, or to
proxied remotes. Seems that the second call to Remote.list somehow proxied remotes. Seems that the second call to Remote.list somehow
is failing to listProxies. is failing to listProxies.
@ -77,6 +71,11 @@ For June's work on [[design/passthrough_proxy]], implementation plan:
* On upload to cluster, send to nodes where it's preferred content, and not * On upload to cluster, send to nodes where it's preferred content, and not
to other nodes. to other nodes.
* Getting a key from a cluster currently always selects the lowest cost
remote, and always the same remote if cost is the same. Should
round-robin amoung remotes, and prefer to avoid using remotes that
other git-annex processes are currently using.
* Support annex.jobs for clusters. * Support annex.jobs for clusters.
* On upload to a cluster, as well as fanout to nodes, if the key is * On upload to a cluster, as well as fanout to nodes, if the key is
@ -108,5 +107,5 @@ For June's work on [[design/passthrough_proxy]], implementation plan:
* Support using a proxy when its url is a P2P address. * Support using a proxy when its url is a P2P address.
(Eg tor-annex remotes.) (Eg tor-annex remotes.)
* `viconfig` support for setting preferred content and description of * `viconfig` support for setting preferred content, group,
clusters and description of clusters