add equivilant key log for VURL keys

When downloading a VURL from the web, make sure that the equivilant key
log is populated.

Unfortunately, this does not hash the content while it's being
downloaded from the web. There is not an interface in Backend currently
for incrementally hash generation, only for incremental verification of an
existing hash. So this might add a noticiable delay, and it has to show
a "(checksum...") message. This could stand to be improved.

But, that separate hashing step only has to happen on the first download
of new content from the web. Once the hash is known, the VURL key can have
its hash verified incrementally while downloading except when the
content in the web has changed. (Doesn't happen yet because
verifyKeyContentIncrementally is not implemented yet for VURL keys.)

Note that the equivilant key log file is formatted as a presence log.
This adds a tiny bit of overhead (eg "1 ") per line over just listing the
urls. The reason I chose to use that format is it seems possible that
there will need to be a way to remove an equivilant key at some point in
the future. I don't know why that would be necessary, but it seemed wise
to allow for the possibility.

Downloads of VURL keys from other special remotes that claim urls,
like bittorrent for example, does not popilate the equivilant key log.
So for now, no checksum verification will be done for those.

Sponsored-by: Nicholas Golder-Manning on Patreon
This commit is contained in:
Joey Hess 2024-02-29 15:41:57 -04:00
parent 0f7143d226
commit 55bf01b788
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
9 changed files with 125 additions and 14 deletions

View file

@ -1,6 +1,6 @@
{- git-annex key/value backends
-
- Copyright 2010-2021 Joey Hess <id@joeyh.name>
- Copyright 2010-2024 Joey Hess <id@joeyh.name>
-
- Licensed under the GNU AGPL version 3 or higher.
-}
@ -10,6 +10,7 @@
module Backend (
builtinList,
defaultBackend,
defaultHashBackend,
genKey,
getBackend,
chooseBackend,
@ -18,6 +19,7 @@ module Backend (
maybeLookupBackendVariety,
isStableKey,
isCryptographicallySecure,
isCryptographicallySecure',
) where
import Annex.Common
@ -40,7 +42,13 @@ import qualified Data.Map as M
builtinList :: [Backend]
builtinList = Backend.Hash.backends ++ Backend.WORM.backends ++ Backend.URL.backends
{- Backend to use by default when generating a new key. -}
{- The default hashing backend. This must use a cryptographically secure
- hash. -}
defaultHashBackend :: Backend
defaultHashBackend = Prelude.head builtinList
{- Backend to use by default when generating a new key. Takes git config
- and --backend option into account. -}
defaultBackend :: Annex Backend
defaultBackend = maybe cache return =<< Annex.getState Annex.backend
where
@ -49,7 +57,7 @@ defaultBackend = maybe cache return =<< Annex.getState Annex.backend
=<< Annex.getRead Annex.forcebackend
b <- case n of
Just name | valid name -> lookupname name
_ -> pure (Prelude.head builtinList)
_ -> pure defaultHashBackend
Annex.changeState $ \s -> s { Annex.backend = Just b }
return b
valid name = not (null name)
@ -116,5 +124,8 @@ isStableKey k = maybe False (`B.isStableKey` k)
<$> maybeLookupBackendVariety (fromKey keyVariety k)
isCryptographicallySecure :: Key -> Annex Bool
isCryptographicallySecure k = maybe False B.isCryptographicallySecure
isCryptographicallySecure k = maybe False isCryptographicallySecure'
<$> maybeLookupBackendVariety (fromKey keyVariety k)
isCryptographicallySecure' :: Backend -> Bool
isCryptographicallySecure' = B.isCryptographicallySecure