toward SafeDropProof expiry checking

Added Maybe POSIXTime to SafeDropProof, which gets set when the proof is
based on a LockedCopy. If there are several LockedCopies, it uses the
closest expiry time. That is not optimal, it may be that the proof
expires based on one LockedCopy but another one has not expired. But
that seems unlikely to really happen, and anyway the user can just
re-run a drop if it fails due to expiry.

Pass the SafeDropProof to removeKey, which is responsible for checking
it for expiry in situations where that could be a problem. Which really
only means in Remote.Git.

Made Remote.Git check expiry when dropping from a local remote.

Checking expiry when dropping from a P2P remote is not yet implemented.
P2P.Protocol.remove has SafeDropProof plumbed through to it for that
purpose.

Fixing the remaining 2 build warnings should complete this work.

Note that the use of a POSIXTime here means that if the clock gets set
forward while git-annex is in the middle of a drop, it may say that
dropping took too long. That seems ok. Less ok is that if the clock gets
turned back a sufficient amount (eg 5 minutes), proof expiry won't be
noticed. It might be better to use the Monotonic clock, but that doesn't
advance when a laptop is suspended, and while there is the linux
Boottime clock, that is not available on other systems. Perhaps a
combination of POSIXTime and the Monotonic clock could detect laptop
suspension and also detect clock being turned back?

There is a potential future flag day where
p2pDefaultLockContentRetentionDuration is not assumed, but is probed
using the P2P protocol, and peers that don't support it can no longer
produce a LockedCopy. Until that happens, when git-annex is
communicating with older peers there is a risk of data loss when
a ssh connection closes during LOCKCONTENT.
This commit is contained in:
Joey Hess 2024-07-04 12:23:46 -04:00
parent 98dbfb6bbd
commit 1243af4a18
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
39 changed files with 274 additions and 123 deletions

View file

@ -61,6 +61,7 @@ import P2P.Address
import Annex.Path
import Creds
import Types.NumCopies
import Annex.SafeDropProof
import Types.ProposedAccepted
import Annex.Action
import Messages.Progress
@ -437,26 +438,43 @@ keyUrls gc repo r key = map tourl locs'
#endif
remoteconfig = gitconfig r
dropKey :: Remote -> State -> Key -> Annex ()
dropKey r st key = do
dropKey :: Remote -> State -> Maybe SafeDropProof -> Key -> Annex ()
dropKey r st proof key = do
repo <- getRepo r
dropKey' repo r st key
dropKey' repo r st proof key
dropKey' :: Git.Repo -> Remote -> State -> Key -> Annex ()
dropKey' repo r st@(State connpool duc _ _ _) key
dropKey' :: Git.Repo -> Remote -> State -> Maybe SafeDropProof -> Key -> Annex ()
dropKey' repo r st@(State connpool duc _ _ _) proof key
| not $ Git.repoIsUrl repo = ifM duc
( guardUsable repo (giveup "cannot access remote") $
commitOnCleanup repo r st $ onLocalFast st $ do
whenM (Annex.Content.inAnnex key) $ do
let cleanup = logStatus key InfoMissing
Annex.Content.lockContentForRemoval key cleanup $ \lock -> do
Annex.Content.removeAnnex lock
cleanup
( guardUsable repo (giveup "cannot access remote") removelocal
, giveup "remote does not have expected annex.uuid value"
)
| Git.repoIsHttp repo = giveup "dropping from http remote not supported"
| otherwise = P2PHelper.remove (uuid r)
(Ssh.runProto r connpool (return (Right False, Nothing))) key
| otherwise = P2PHelper.remove (uuid r) p2prunner proof key
where
p2prunner = Ssh.runProto r connpool (return (Right False, Nothing))
-- It could take a long time to eg, automount a drive containing
-- the repo, so check the proof for expiry again after locking the
-- content for removal.
removelocal = do
proofunexpired <- commitOnCleanup repo r st $ onLocalFast st $ do
ifM (Annex.Content.inAnnex key)
( do
let cleanup = do
logStatus key InfoMissing
return True
Annex.Content.lockContentForRemoval key cleanup $ \lock ->
ifM (liftIO $ checkSafeDropProofEndTime proof)
( do
Annex.Content.removeAnnex lock
cleanup
, return False
)
, return True
)
unless proofunexpired
safeDropProofExpired
lockKey :: Remote -> State -> Key -> (VerifiedCopy -> Annex r) -> Annex r
lockKey r st key callback = do