From 5d8bdac38e9c1c36a4fc3523804fd5fae43e0db9 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Sun, 23 Jun 2024 16:22:39 -0400 Subject: [PATCH] upload fanout resume seems free of fenceposts Tested it with small chunk sizes (like 2) and resumes that were eg 1 byte from the end of the file or beginning of file. Also, git-annex testremote passes now against a cluster! --- doc/todo/git-annex_proxies.mdwn | 21 ++++++++++----------- 1 file changed, 10 insertions(+), 11 deletions(-) diff --git a/doc/todo/git-annex_proxies.mdwn b/doc/todo/git-annex_proxies.mdwn index cf62bd6783..2b779b5c17 100644 --- a/doc/todo/git-annex_proxies.mdwn +++ b/doc/todo/git-annex_proxies.mdwn @@ -55,18 +55,12 @@ For June's work on [[design/passthrough_proxy]], implementation plan: * Getting a key from a cluster should proxy from one of the nodes that has it. (done) +* Implement upload with fanout and reporting back additional UUIDs over P2P + protocol. (done) + * Implement cluster drops, trying to remove from all nodes, and returning which UUIDs it was dropped from. (done) -* Getting a key from a cluster currently always selects the lowest cost - remote, and always the same remote if cost is the same. Should - round-robin amoung remotes, and prefer to avoid using remotes that - other git-annex processes are currently using. - -* Implement upload with fanout and reporting back additional UUIDs over P2P - protocol. (done, but need to check for fencepost errors on resume of - incomplete upload with remotes at different points) - * `git-annex sync --content` does not send content to clusters, or to proxied remotes. Seems that the second call to Remote.list somehow is failing to listProxies. @@ -77,6 +71,11 @@ For June's work on [[design/passthrough_proxy]], implementation plan: * On upload to cluster, send to nodes where it's preferred content, and not to other nodes. +* Getting a key from a cluster currently always selects the lowest cost + remote, and always the same remote if cost is the same. Should + round-robin amoung remotes, and prefer to avoid using remotes that + other git-annex processes are currently using. + * Support annex.jobs for clusters. * On upload to a cluster, as well as fanout to nodes, if the key is @@ -108,5 +107,5 @@ For June's work on [[design/passthrough_proxy]], implementation plan: * Support using a proxy when its url is a P2P address. (Eg tor-annex remotes.) -* `viconfig` support for setting preferred content and description of - clusters +* `viconfig` support for setting preferred content, group, + and description of clusters