fix git-annex sync --content with proxied remotes
Loading the remote list a second time was removing all proxied remotes. That happened because setting up the proxied remote added some config fields to the in-memory git config, and on the second load, it saw those configs and decided not to overwrite them with the proxy. Now on the second load, that still happens. But now, the proxied git configs are used to generate a remote same as if those configs were all set. The reason that didn't happen before was twofold, the gitremotes cache was not dropped, and the remote's url field was not set correctly. The problem with the remote's url field is that while it was marked as proxy inherited, all other proxy inherited fields are annex- configs. And the code to inherit didn't work for the url field. Now it all works, but git-annex sync is left running git push/pull on the proxied remote, which doesn't work. That still needs to be fixed.
This commit is contained in:
parent
6f94062c53
commit
0c111fc96a
4 changed files with 14 additions and 11 deletions
|
@ -448,8 +448,8 @@ claimingUrl' remotefilter url = do
|
||||||
where
|
where
|
||||||
checkclaim = maybe (pure False) (`id` url) . claimUrl
|
checkclaim = maybe (pure False) (`id` url) . claimUrl
|
||||||
|
|
||||||
{- Is this a remote of a type we can sync with, or a special remote
|
{- Is this a remote of a type that git pull and push work with?
|
||||||
- with an annex:: url configured? -}
|
- That includes special remotes with an annex:: url configured. -}
|
||||||
gitSyncableRemote :: Remote -> Bool
|
gitSyncableRemote :: Remote -> Bool
|
||||||
gitSyncableRemote r
|
gitSyncableRemote r
|
||||||
| gitSyncableRemoteType (remotetype r)
|
| gitSyncableRemoteType (remotetype r)
|
||||||
|
|
|
@ -811,14 +811,14 @@ listProxied proxies rs = concat <$> mapM go rs
|
||||||
| any isconfig (M.keys (Git.config g)) = pure Nothing
|
| any isconfig (M.keys (Git.config g)) = pure Nothing
|
||||||
| otherwise = do
|
| otherwise = do
|
||||||
-- Not using addGitConfigOverride for inherited
|
-- Not using addGitConfigOverride for inherited
|
||||||
-- configs other than the uuid, because child
|
-- configs, because child git processes do not
|
||||||
-- git processes do not need them to be provided
|
-- need them to be provided with -c.
|
||||||
-- with -c.
|
|
||||||
Annex.adjustGitRepo (pure . annexconfigadjuster)
|
Annex.adjustGitRepo (pure . annexconfigadjuster)
|
||||||
return $ Just $ renamedr
|
return $ Just $ renamedr
|
||||||
where
|
where
|
||||||
renamedr =
|
renamedr =
|
||||||
let c = adduuid configkeyUUID $ Git.fullconfig r
|
let c = adduuid configkeyUUID $
|
||||||
|
Git.fullconfig r
|
||||||
in r
|
in r
|
||||||
{ Git.remoteName = Just proxyname
|
{ Git.remoteName = Just proxyname
|
||||||
, Git.config = M.map Prelude.head c
|
, Git.config = M.map Prelude.head c
|
||||||
|
@ -827,6 +827,7 @@ listProxied proxies rs = concat <$> mapM go rs
|
||||||
|
|
||||||
annexconfigadjuster r' =
|
annexconfigadjuster r' =
|
||||||
let c = adduuid (configRepoUUID renamedr) $
|
let c = adduuid (configRepoUUID renamedr) $
|
||||||
|
addurl (remoteConfig renamedr "url") $
|
||||||
inheritconfigs $ Git.fullconfig r'
|
inheritconfigs $ Git.fullconfig r'
|
||||||
in r'
|
in r'
|
||||||
{ Git.config = M.map Prelude.head c
|
{ Git.config = M.map Prelude.head c
|
||||||
|
@ -836,6 +837,9 @@ listProxied proxies rs = concat <$> mapM go rs
|
||||||
adduuid ck = M.insert ck
|
adduuid ck = M.insert ck
|
||||||
[Git.ConfigValue $ fromUUID $ proxyRemoteUUID p]
|
[Git.ConfigValue $ fromUUID $ proxyRemoteUUID p]
|
||||||
|
|
||||||
|
addurl ck = M.insert ck
|
||||||
|
[Git.ConfigValue $ encodeBS $ Git.repoLocation r]
|
||||||
|
|
||||||
inheritconfigs c = foldl' inheritconfig c proxyInheritedFields
|
inheritconfigs c = foldl' inheritconfig c proxyInheritedFields
|
||||||
|
|
||||||
inheritconfig c k = case (M.lookup dest c, M.lookup src c) of
|
inheritconfig c k = case (M.lookup dest c, M.lookup src c) of
|
||||||
|
|
|
@ -620,7 +620,7 @@ remoteGitConfigField = \case
|
||||||
-- Allow proxy chains.
|
-- Allow proxy chains.
|
||||||
ProxyField -> inherited "proxy"
|
ProxyField -> inherited "proxy"
|
||||||
ClusterNodeField -> uninherited "cluster-node"
|
ClusterNodeField -> uninherited "cluster-node"
|
||||||
UrlField -> inherited "url"
|
UrlField -> uninherited "url"
|
||||||
ShellField -> inherited "shell"
|
ShellField -> inherited "shell"
|
||||||
SshOptionsField -> inherited "ssh-options"
|
SshOptionsField -> inherited "ssh-options"
|
||||||
RsyncOptionsField -> inherited "rsync-options"
|
RsyncOptionsField -> inherited "rsync-options"
|
||||||
|
|
|
@ -26,15 +26,14 @@ In development on the `proxy` branch.
|
||||||
|
|
||||||
For June's work on [[design/passthrough_proxy]], remaining todos:
|
For June's work on [[design/passthrough_proxy]], remaining todos:
|
||||||
|
|
||||||
* `git-annex sync --content` does not send content to clusters, or to
|
* `git-annex sync` etc should not treat clusters as git syncable remotes.
|
||||||
proxied remotes. Seems that the second call to Remote.list somehow
|
|
||||||
is failing to listProxies.
|
|
||||||
|
|
||||||
* `git-annex sync` etc, when operating on clusters, should first
|
* `git-annex sync` etc, when operating on clusters, should first
|
||||||
operate on the cluster as a whole, to take advantages of fanout on upload
|
operate on the cluster as a whole, to take advantages of fanout on upload
|
||||||
and mass drop. Only operate on individual cluster nodes afterwards,
|
and mass drop. Only operate on individual cluster nodes afterwards,
|
||||||
to handle cases such as a cluster containing a key, but some node
|
to handle cases such as a cluster containing a key, but some node
|
||||||
wanting and lacking the key.
|
wanting and lacking the key. Perhaps just setting cost for nodes slightly
|
||||||
|
higher than the cluster cost will be enough?
|
||||||
|
|
||||||
* On upload to cluster, send to nodes where it's preferred content, and not
|
* On upload to cluster, send to nodes where it's preferred content, and not
|
||||||
to other nodes.
|
to other nodes.
|
||||||
|
|
Loading…
Add table
Add a link
Reference in a new issue