p2p ssh connection pools

Much like Remote.P2P, there's a pool of connections to a peer, in order
to support concurrent operations.

Deals with old git-annex-ssh on the remote that does not support p2pstdio,
by only trying once to use it, and remembering if it's not supported.

Made p2pstdio send an AUTH_SUCCESS with its uuid, which serves the dual
purposes of something to detect to see that the connection is working,
and a way to verify that it's connected to the right uuid.
(There's a redundant uuid check since the uuid field is sent
by git_annex_shell, but I anticipate that being removed later when
the legacy git-annex-shell stuff gets removed.)

Not entirely happy with Remote.Git.runSsh's behavior
when the proto action fails. Running the fallback will work ok, but what
will we do when the fallbacks later get removed? It might be better to
try to reconnect, in case the connection got closed.

This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
This commit is contained in:
Joey Hess 2018-03-08 14:02:18 -04:00
parent 978078f0fe
commit c036a380b2
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
7 changed files with 150 additions and 16 deletions

View file

@ -55,6 +55,9 @@ import qualified Remote.Helper.Ssh as Ssh
import qualified Remote.GCrypt
import qualified Remote.P2P
import P2P.Address
import qualified P2P.Protocol as P2P
import qualified P2P.Annex as P2P
import qualified P2P.IO as P2P
import Annex.Path
import Creds
import Messages.Progress
@ -729,10 +732,11 @@ mkCopier remotewanthardlink rsyncparams = do
, return copier
)
{- Normally the UUID is checked at startup, but annex-checkuuid config
- can prevent that. To avoid getting confused, a deferred
- check is done just before the repository is used. This returns False
- when the repository UUID is not as expected. -}
{- Normally the UUID of a local repository is checked at startup,
- but annex-checkuuid config can prevent that. To avoid getting
- confused, a deferred check is done just before the repository
- is used.
- This returns False when the repository UUID is not as expected. -}
type DeferredUUIDCheck = Annex Bool
mkDeferredUUIDCheck :: Git.Repo -> UUID -> RemoteGitConfig -> Annex DeferredUUIDCheck
@ -751,3 +755,32 @@ mkDeferredUUIDCheck r u gc
return ok
, liftIO $ readMVar v
)
-- Runs a P2P Proto action on a remote when it supports that,
-- otherwise the fallback action.
runSsh :: Remote -> Ssh.P2PSshConnectionPool -> P2P.Proto a -> Annex a -> Annex a
runSsh r connpool proto fallback =
Ssh.getP2PSshConnection r connpool >>= maybe fallback go
where
go c = do
(c', v) <- runSsh' proto c
case v of
Just res -> do
liftIO $ Ssh.storeP2PSshConnection connpool c'
return res
-- Running the proto failed, either due to a protocol
-- error or a network error, so discard the
-- connection, and run the fallback.
Nothing -> fallback
runSsh' :: P2P.Proto a -> Ssh.P2PSshConnection -> Annex (Ssh.P2PSshConnection, Maybe a)
runSsh' _ P2P.ClosedConnection = return (P2P.ClosedConnection, Nothing)
runSsh' a conn@(P2P.OpenConnection (c, _pid)) =
P2P.runFullProto P2P.Client c a >>= \case
Right r -> return (conn, Just r)
-- When runFullProto fails, the connection is no longer
-- usable, so close it.
Left e -> do
warning $ "Lost connection (" ++ e ++ ")"
conn' <- liftIO $ Ssh.closeP2PSshConnection conn
return (conn', Nothing)