toward SafeDropProof expiry checking

Added Maybe POSIXTime to SafeDropProof, which gets set when the proof is
based on a LockedCopy. If there are several LockedCopies, it uses the
closest expiry time. That is not optimal, it may be that the proof
expires based on one LockedCopy but another one has not expired. But
that seems unlikely to really happen, and anyway the user can just
re-run a drop if it fails due to expiry.

Pass the SafeDropProof to removeKey, which is responsible for checking
it for expiry in situations where that could be a problem. Which really
only means in Remote.Git.

Made Remote.Git check expiry when dropping from a local remote.

Checking expiry when dropping from a P2P remote is not yet implemented.
P2P.Protocol.remove has SafeDropProof plumbed through to it for that
purpose.

Fixing the remaining 2 build warnings should complete this work.

Note that the use of a POSIXTime here means that if the clock gets set
forward while git-annex is in the middle of a drop, it may say that
dropping took too long. That seems ok. Less ok is that if the clock gets
turned back a sufficient amount (eg 5 minutes), proof expiry won't be
noticed. It might be better to use the Monotonic clock, but that doesn't
advance when a laptop is suspended, and while there is the linux
Boottime clock, that is not available on other systems. Perhaps a
combination of POSIXTime and the Monotonic clock could detect laptop
suspension and also detect clock being turned back?

There is a potential future flag day where
p2pDefaultLockContentRetentionDuration is not assumed, but is probed
using the P2P protocol, and peers that don't support it can no longer
produce a LockedCopy. Until that happens, when git-annex is
communicating with older peers there is a risk of data loss when
a ssh connection closes during LOCKCONTENT.
This commit is contained in:
Joey Hess 2024-07-04 12:23:46 -04:00
parent 98dbfb6bbd
commit 1243af4a18
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
39 changed files with 274 additions and 123 deletions

View file

@ -21,8 +21,10 @@ import Types.NumCopies
import Annex.Verify
import Logs.Location
import Utility.SafeOutput
import Utility.HumanTime
import Control.Concurrent
import Data.Time.Clock.POSIX
-- Runs a Proto action using a connection it sets up.
type ProtoRunner a = P2P.Proto a -> Annex (Maybe a)
@ -60,8 +62,8 @@ retrieve gc runner k af dest p verifyconfig = do
Just (False, _) -> giveup "Transfer failed"
Nothing -> remoteUnavail
remove :: UUID -> ProtoRunner (Either String Bool, Maybe [UUID]) -> Key -> Annex ()
remove remoteuuid runner k = runner (P2P.remove k) >>= \case
remove :: UUID -> ProtoRunner (Either String Bool, Maybe [UUID]) -> Maybe SafeDropProof -> Key -> Annex ()
remove remoteuuid runner proof k = runner (P2P.remove proof k) >>= \case
Just (Right True, alsoremoveduuids) -> note alsoremoveduuids
Just (Right False, alsoremoveduuids) -> do
note alsoremoveduuids
@ -94,18 +96,23 @@ checkpresent runner k =
-}
lock :: WithConn a c -> ProtoConnRunner c -> UUID -> Key -> (VerifiedCopy -> Annex a) -> Annex a
lock withconn connrunner u k callback = withconn $ \conn -> do
starttime <- liftIO getPOSIXTime
connv <- liftIO $ newMVar conn
let runproto d p = do
c <- liftIO $ takeMVar connv
(c', mr) <- connrunner p c
liftIO $ putMVar connv c'
return (fromMaybe d mr)
r <- P2P.lockContentWhile runproto k go
r <- P2P.lockContentWhile runproto k (go starttime)
conn' <- liftIO $ takeMVar connv
return (conn', r)
where
go False = giveup "can't lock content"
go True = withVerifiedCopy LockedCopy u (return True) callback
go _ False = giveup "can't lock content"
go starttime True = do
let check = return $ Left $ starttime + retentionduration
withVerifiedCopy LockedCopy u check callback
retentionduration = fromIntegral $
durationSeconds p2pDefaultLockContentRetentionDuration
remoteUnavail :: a
remoteUnavail = giveup "can't connect to remote"