incremental checksum on download from ssh or p2p

Checksum as content is received from a remote git-annex repository, rather
than doing it in a second pass.

Not tested at all yet, but I imagine it will work!

Not implemented for any special remotes, and also not implemented for
copies from local remotes. It may be that, for local remotes, it will
suffice to use rsync, rely on its checksumming, and simply return Verified.
(It would still make a checksumming pass when cp is used for COW, I guess.)
This commit is contained in:
Joey Hess 2021-02-09 17:03:27 -04:00
parent ed684f651e
commit 62e152f210
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
17 changed files with 118 additions and 52 deletions

View file

@ -26,10 +26,10 @@ module Annex.Content.Presence (
import Annex.Common
import qualified Annex
import Annex.Verify
import Annex.LockPool
import Annex.WorkerPool
import Types.Remote (unVerified, Verification(..), RetrievalSecurityPolicy(..))
import qualified Types.Remote
import qualified Types.Backend
import qualified Backend
import qualified Database.Keys
@ -231,16 +231,3 @@ warnUnverifiableInsecure k = warning $ unwords
]
where
kv = decodeBS (formatKeyVariety (fromKey keyVariety k))
data VerifyConfig = AlwaysVerify | NoVerify | RemoteVerify Remote | DefaultVerify
shouldVerify :: VerifyConfig -> Annex Bool
shouldVerify AlwaysVerify = return True
shouldVerify NoVerify = return False
shouldVerify DefaultVerify = annexVerify <$> Annex.getGitConfig
shouldVerify (RemoteVerify r) =
(shouldVerify DefaultVerify
<&&> pure (remoteAnnexVerify (Types.Remote.gitconfig r)))
-- Export remotes are not key/value stores, so always verify
-- content from them even when verification is disabled.
<||> Types.Remote.isExportSupported r