dropping from clusters
Dropping from a cluster drops from every node of the cluster. Including nodes that the cluster does not think have the content. This is different from GET and CHECKPRESENT, which do trust the cluster's location log. The difference is that removing from a cluster should make 100% the content is gone from every node. So doing extra work is ok. Compare with CHECKPRESENT where checking every node could make it very expensive, and the worst that can happen in a false negative is extra work being done. Extended the P2P protocol with FAILURE-PLUS to handle the case where a drop from one node succeeds, but a drop from another node fails. In that case the entire cluster drop has failed. Note that SUCCESS-PLUS is returned when dropping from a proxied remote that is not a cluster, when the protocol version supports it. This is because P2P.Proxy does not know when it's proxying for a single node cluster vs for a remote that is not a cluster.
This commit is contained in:
parent
a6a04b7e5e
commit
5b332a87be
14 changed files with 144 additions and 59 deletions
|
@ -55,6 +55,9 @@ For June's work on [[design/passthrough_proxy]], implementation plan:
|
|||
* Getting a key from a cluster should proxy from one of the nodes that has
|
||||
it. (done)
|
||||
|
||||
* Implement cluster drops, trying to remove from all nodes, and returning
|
||||
which UUIDs it was dropped from. (done)
|
||||
|
||||
* Getting a key from a cluster currently always selects the lowest cost
|
||||
remote, and always the same remote if cost is the same. Should
|
||||
round-robin amoung remotes, and prefer to avoid using remotes that
|
||||
|
@ -67,13 +70,16 @@ For June's work on [[design/passthrough_proxy]], implementation plan:
|
|||
* On upload to cluster, send to nodes where it's preferred content, and not
|
||||
to other nodes.
|
||||
|
||||
* Implement cluster drops, trying to remove from all nodes, and returning
|
||||
which UUIDs it was dropped from.
|
||||
|
||||
Problem: `move --from cluster` in "does this make it worse"
|
||||
* Problem: `move --from cluster` in "does this make it worse"
|
||||
check may fail to realize that dropping from multiple nodes does in fact
|
||||
make it worse.
|
||||
|
||||
* Bug: When a cluster has one node, copying a file to it does not update
|
||||
location log to say the content is present on it. It's returning SUCCESS
|
||||
rather than SUCCESS-PLUS.
|
||||
|
||||
* Support annex.jobs for clusters.
|
||||
|
||||
* On upload to a cluster, as well as fanout to nodes, if the key is
|
||||
preferred content of the proxy repository, store it there.
|
||||
(But not when preferred content is not configured.)
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue