git-annex-shell: proxy nodes located beyond remote cluster gateways

Walking a tightrope between security and convenience here, because
git-annex-shell needs to only proxy for things when there has been
an explicit, local action to configure them.

In this case, the user has to have run `git-annex extendcluster`,
which now sets annex-cluster-gateway on the remote.

Note that any repositories that the gateway is recorded to
proxy for will be proxied onward. This is not limited to cluster nodes,
because checking the node log would not add any security; someone could
add any uuid to it. The gateway of course then does its own
checking to determine if it will allow proxying for the remote.
This commit is contained in:
Joey Hess 2024-06-26 12:56:16 -04:00
parent 1ec2fecf3f
commit 07e899c9d3
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
10 changed files with 90 additions and 44 deletions

View file

@ -1398,11 +1398,12 @@ repository, using [[git-annex-config]]. See its man page for a list.)
* `annex.cluster.<name>`
[[git-annex-updatecluster]] sets this to the UUID of a cluster
based on `remote.<name>.annex-cluster-node` configuration.
This is set to make the repository be a gateway to a cluster.
The value is the cluster UUID. Note that cluster UUIDs are not
the same as repository UUIDs, and a repository UUID cannot be used here.
Note that cluster UUIDs are not the same as repository UUIDs,
and a repository UUID cannot be used here.
Usually this is set up by running [[git-annex-initcluster]] or
[[git-annex-extendcluster]].
# CONFIGURATION OF REMOTES
@ -1681,14 +1682,11 @@ Remotes are configured using these settings in `.git/config`.
After configuring this, run [[git-annex-updateproxy](1) to store
the new configuration in the git-annex branch.
* `remote.<name>.annex-proxied`
Setting this to "true" indicates that a remote is proxied via the
git-annex repository that its remote points to. That prevents commands
like `git-annex sync` from pulling and pushing the remote.
* `remote.<name>.annex-proxied-by`
Usually this is used internally, when git-annex sets up proxied remotes,
and will not need to be set.
and will not need to be configured. The value is the UUID of the
git-annex repository that proxies access to this remote.
* `remote.<name>.annex-cluster-node`
@ -1699,6 +1697,15 @@ Remotes are configured using these settings in `.git/config`.
After configuring this, run [[git-annex-updatecluster](1) to store
the new configuration in the git-annex branch.
* `remote.<name>.annex-cluster-gateway`
Set to the UUID of a cluster that this remote serves as a gateway for.
Multiple UUIDs can be listed, separated by whitespace. When the local
repository is also a gateway for that cluster, it will proxy for the
nodes of the remote gateway.
Usually this is set up by running [[git-annex-extendcluster]].
* `remote.<name>.annex-private`
When this is set to true, no information about the remote will be

View file

@ -33,8 +33,11 @@ For June's work on [[design/passthrough_proxy]], remaining todos:
protocol messages on to any remotes that have the same UUID as
the cluster. Needs VIA extension to P2P protocol to avoid cycles.
* `git-annex updatecluster` needs changes to support a distributed cluster.
Currently it will remove nodes that are behind another gateway.
Current status: Distributed cluster nodes are visible,
and can be accessed directly, but trying to GET from a cluster
fails when the content is located behind a remote gateway.
And PUT only sends to the immediate nodes
of the cluster, not on to other gateways.
* Getting a key from a cluster currently always selects the lowest cost
remote, and always the same remote if cost is the same. Should
@ -106,3 +109,8 @@ For June's work on [[design/passthrough_proxy]], remaining todos:
* Support annex.jobs for clusters. (done)
* Add `git-annex extendcluster` command and extend `git-annex updatecluster`
to support clusters with multiple gateways. (done)
* Support proxying for a remote that is proxied by another gateway of
a cluster. (done)