support multi-gateway clusters

VIA extension still needed otherwise a copy to a cluster can loop
forever.
This commit is contained in:
Joey Hess 2024-06-26 15:02:10 -04:00
parent 8b6708e745
commit 4172109c8d
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
2 changed files with 13 additions and 11 deletions

View file

@ -65,8 +65,9 @@ clusterProxySelector clusteruuid protocolversion = do
nodeuuids <- (fromMaybe S.empty . M.lookup clusteruuid . clusterUUIDs) nodeuuids <- (fromMaybe S.empty . M.lookup clusteruuid . clusterUUIDs)
<$> getClusters <$> getClusters
clusternames <- annexClusters <$> Annex.getGitConfig clusternames <- annexClusters <$> Annex.getGitConfig
remotes <- filter (isnode nodeuuids clusternames) <$> remoteList allremotes <- remoteList
nodes <- mapM (proxySshRemoteSide protocolversion) remotes let clusterremotes = filter (isnode allremotes nodeuuids clusternames) allremotes
nodes <- mapM (proxySshRemoteSide protocolversion) clusterremotes
return $ ProxySelector return $ ProxySelector
{ proxyCHECKPRESENT = nodecontaining nodes { proxyCHECKPRESENT = nodecontaining nodes
, proxyGET = nodecontaining nodes , proxyGET = nodecontaining nodes
@ -94,15 +95,22 @@ clusterProxySelector clusteruuid protocolversion = do
} }
where where
-- Nodes of the cluster have remote.name.annex-cluster-node -- Nodes of the cluster have remote.name.annex-cluster-node
-- containing its name. -- containing its name. Or they are proxied by a remote
isnode nodeuuids clusternames r = -- that has remote.name.annex-cluster-node containing the cluster's
-- UUID.
isnode rs nodeuuids clusternames r =
case remoteAnnexClusterNode (Remote.gitconfig r) of case remoteAnnexClusterNode (Remote.gitconfig r) of
Nothing -> False
Just names Just names
| any (isclustername clusternames) names -> | any (isclustername clusternames) names ->
flip S.member nodeuuids $ flip S.member nodeuuids $
ClusterNodeUUID $ Remote.uuid r ClusterNodeUUID $ Remote.uuid r
| otherwise -> False | otherwise -> False
Nothing -> case remoteAnnexProxiedBy (Remote.gitconfig r) of
Just proxyuuid -> not $ null $
filter (== clusteruuid) $
concatMap (remoteAnnexClusterGateway . Remote.gitconfig) $
filter (\p -> Remote.uuid p == proxyuuid) rs
Nothing -> False
isclustername clusternames name = isclustername clusternames name =
M.lookup name clusternames == Just clusteruuid M.lookup name clusternames == Just clusteruuid

View file

@ -33,12 +33,6 @@ For June's work on [[design/passthrough_proxy]], remaining todos:
protocol messages on to any remotes that have the same UUID as protocol messages on to any remotes that have the same UUID as
the cluster. Needs VIA extension to P2P protocol to avoid cycles. the cluster. Needs VIA extension to P2P protocol to avoid cycles.
Current status: Distributed cluster nodes are visible,
and can be accessed directly, but trying to GET from a cluster
fails when the content is located behind a remote gateway.
And PUT only sends to the immediate nodes
of the cluster, not on to other gateways.
* Getting a key from a cluster currently always selects the lowest cost * Getting a key from a cluster currently always selects the lowest cost
remote, and always the same remote if cost is the same. Should remote, and always the same remote if cost is the same. Should
round-robin amoung remotes, and prefer to avoid using remotes that round-robin amoung remotes, and prefer to avoid using remotes that