2024-05-07 17:42:12 +00:00
|
|
|
{- Backends for git-remote-annex.
|
|
|
|
-
|
|
|
|
- GITBUNDLE keys store git bundles
|
|
|
|
- GITMANIFEST keys store ordered lists of GITBUNDLE keys
|
|
|
|
-
|
|
|
|
- Copyright 2024 Joey Hess <id@joeyh.name>
|
|
|
|
-
|
|
|
|
- Licensed under the GNU AGPL version 3 or higher.
|
|
|
|
-}
|
|
|
|
|
|
|
|
{-# LANGUAGE OverloadedStrings #-}
|
|
|
|
|
|
|
|
module Backend.GitRemoteAnnex (
|
|
|
|
backends,
|
|
|
|
genGitBundleKey,
|
|
|
|
genManifestKey,
|
2024-05-21 13:51:19 +00:00
|
|
|
genBackupManifestKey,
|
git-annex unused --from remote skips its git-remote-annex keys
This turns out to only be necessary is edge cases. Most of the
time, git-annex unused --from remote doesn't see git-remote-annex keys
at all, because it does not record a location log for them.
On the other hand, git-annex unused does find them, since it does not
rely on the location log. And that's good because they're a local cache
that the user should be able to drop.
If, however, the user ran git-annex unused and then git-annex move
--unused --to remote, the keys would have a location log for that
remote. Then git-annex unused --from remote would see them, and would
consider them unused. Even when they are present on the special remote
they belong to. And that risks losing data if they drop the keys from
the special remote, but didn't expect it would delete git branches they
had pushed to it.
So, make git-annex unused --from skip git-remote-annex keys whose uuid
is the same as the remote.
2024-05-14 19:12:07 +00:00
|
|
|
isGitRemoteAnnexKey,
|
2024-05-07 17:42:12 +00:00
|
|
|
) where
|
|
|
|
|
|
|
|
import Annex.Common
|
|
|
|
import Types.Key
|
|
|
|
import Types.Backend
|
|
|
|
import Utility.Hash
|
|
|
|
import Utility.Metered
|
|
|
|
import qualified Backend.Hash as Hash
|
|
|
|
|
|
|
|
import qualified Data.ByteString.Short as S
|
git-annex unused --from remote skips its git-remote-annex keys
This turns out to only be necessary is edge cases. Most of the
time, git-annex unused --from remote doesn't see git-remote-annex keys
at all, because it does not record a location log for them.
On the other hand, git-annex unused does find them, since it does not
rely on the location log. And that's good because they're a local cache
that the user should be able to drop.
If, however, the user ran git-annex unused and then git-annex move
--unused --to remote, the keys would have a location log for that
remote. Then git-annex unused --from remote would see them, and would
consider them unused. Even when they are present on the special remote
they belong to. And that risks losing data if they drop the keys from
the special remote, but didn't expect it would delete git branches they
had pushed to it.
So, make git-annex unused --from skip git-remote-annex keys whose uuid
is the same as the remote.
2024-05-14 19:12:07 +00:00
|
|
|
import qualified Data.ByteString.Char8 as B8
|
2024-05-07 17:42:12 +00:00
|
|
|
|
|
|
|
backends :: [Backend]
|
git-annex unused --from remote skips its git-remote-annex keys
This turns out to only be necessary is edge cases. Most of the
time, git-annex unused --from remote doesn't see git-remote-annex keys
at all, because it does not record a location log for them.
On the other hand, git-annex unused does find them, since it does not
rely on the location log. And that's good because they're a local cache
that the user should be able to drop.
If, however, the user ran git-annex unused and then git-annex move
--unused --to remote, the keys would have a location log for that
remote. Then git-annex unused --from remote would see them, and would
consider them unused. Even when they are present on the special remote
they belong to. And that risks losing data if they drop the keys from
the special remote, but didn't expect it would delete git branches they
had pushed to it.
So, make git-annex unused --from skip git-remote-annex keys whose uuid
is the same as the remote.
2024-05-14 19:12:07 +00:00
|
|
|
backends = [gitbundle, gitmanifest]
|
2024-05-07 17:42:12 +00:00
|
|
|
|
|
|
|
gitbundle :: Backend
|
|
|
|
gitbundle = Backend
|
|
|
|
{ backendVariety = GitBundleKey
|
|
|
|
, genKey = Nothing
|
|
|
|
-- ^ Not provided because these keys can only be generated by
|
|
|
|
-- git-remote-annex.
|
|
|
|
, verifyKeyContent = Just $ Hash.checkKeyChecksum sameCheckSum hash
|
|
|
|
, verifyKeyContentIncrementally = Just (liftIO . incrementalVerifier)
|
|
|
|
, canUpgradeKey = Nothing
|
|
|
|
, fastMigrate = Nothing
|
|
|
|
, isStableKey = const True
|
|
|
|
, isCryptographicallySecure = Hash.cryptographicallySecure hash
|
|
|
|
, isCryptographicallySecureKey = const $ pure $
|
|
|
|
Hash.cryptographicallySecure hash
|
|
|
|
}
|
|
|
|
|
git-annex unused --from remote skips its git-remote-annex keys
This turns out to only be necessary is edge cases. Most of the
time, git-annex unused --from remote doesn't see git-remote-annex keys
at all, because it does not record a location log for them.
On the other hand, git-annex unused does find them, since it does not
rely on the location log. And that's good because they're a local cache
that the user should be able to drop.
If, however, the user ran git-annex unused and then git-annex move
--unused --to remote, the keys would have a location log for that
remote. Then git-annex unused --from remote would see them, and would
consider them unused. Even when they are present on the special remote
they belong to. And that risks losing data if they drop the keys from
the special remote, but didn't expect it would delete git branches they
had pushed to it.
So, make git-annex unused --from skip git-remote-annex keys whose uuid
is the same as the remote.
2024-05-14 19:12:07 +00:00
|
|
|
gitmanifest :: Backend
|
|
|
|
gitmanifest = Backend
|
|
|
|
{ backendVariety = GitManifestKey
|
|
|
|
, genKey = Nothing
|
|
|
|
, verifyKeyContent = Nothing
|
|
|
|
, verifyKeyContentIncrementally = Nothing
|
|
|
|
, canUpgradeKey = Nothing
|
|
|
|
, fastMigrate = Nothing
|
|
|
|
, isStableKey = const True
|
|
|
|
, isCryptographicallySecure = False
|
|
|
|
, isCryptographicallySecureKey = const $ pure False
|
|
|
|
}
|
|
|
|
|
2024-05-07 17:42:12 +00:00
|
|
|
-- git bundle keys use the sha256 hash.
|
|
|
|
hash :: Hash.Hash
|
|
|
|
hash = Hash.SHA2Hash (HashSize 256)
|
|
|
|
|
|
|
|
incrementalVerifier :: Key -> IO IncrementalVerifier
|
|
|
|
incrementalVerifier =
|
|
|
|
mkIncrementalVerifier sha2_256_context "checksum" . sameCheckSum
|
|
|
|
|
|
|
|
sameCheckSum :: Key -> String -> Bool
|
|
|
|
sameCheckSum key s = s == expected
|
|
|
|
where
|
|
|
|
-- The checksum comes after a UUID.
|
|
|
|
expected = reverse $ takeWhile (/= '-') $ reverse $
|
|
|
|
decodeBS $ S.fromShort $ fromKey keyName key
|
|
|
|
|
|
|
|
genGitBundleKey :: UUID -> RawFilePath -> MeterUpdate -> Annex Key
|
|
|
|
genGitBundleKey remoteuuid file meterupdate = do
|
|
|
|
filesize <- liftIO $ getFileSize file
|
|
|
|
s <- Hash.hashFile hash file meterupdate
|
|
|
|
return $ mkKey $ \k -> k
|
|
|
|
{ keyName = S.toShort $ fromUUID remoteuuid <> "-" <> encodeBS s
|
|
|
|
, keyVariety = GitBundleKey
|
|
|
|
, keySize = Just filesize
|
|
|
|
}
|
|
|
|
|
2024-05-21 13:51:19 +00:00
|
|
|
genManifestKey :: UUID -> Key
|
|
|
|
genManifestKey = genManifestKey' Nothing
|
|
|
|
|
|
|
|
genBackupManifestKey :: UUID -> Key
|
|
|
|
genBackupManifestKey = genManifestKey' (Just ".bak")
|
|
|
|
|
|
|
|
genManifestKey' :: Maybe S.ShortByteString -> UUID -> Key
|
|
|
|
genManifestKey' extension u = mkKey $ \kd -> kd
|
2024-05-20 19:41:09 +00:00
|
|
|
{ keyName = S.toShort (fromUUID u) <>
|
2024-05-21 13:51:19 +00:00
|
|
|
fromMaybe mempty extension
|
git-annex unused --from remote skips its git-remote-annex keys
This turns out to only be necessary is edge cases. Most of the
time, git-annex unused --from remote doesn't see git-remote-annex keys
at all, because it does not record a location log for them.
On the other hand, git-annex unused does find them, since it does not
rely on the location log. And that's good because they're a local cache
that the user should be able to drop.
If, however, the user ran git-annex unused and then git-annex move
--unused --to remote, the keys would have a location log for that
remote. Then git-annex unused --from remote would see them, and would
consider them unused. Even when they are present on the special remote
they belong to. And that risks losing data if they drop the keys from
the special remote, but didn't expect it would delete git branches they
had pushed to it.
So, make git-annex unused --from skip git-remote-annex keys whose uuid
is the same as the remote.
2024-05-14 19:12:07 +00:00
|
|
|
, keyVariety = GitManifestKey
|
2024-05-07 17:42:12 +00:00
|
|
|
}
|
git-annex unused --from remote skips its git-remote-annex keys
This turns out to only be necessary is edge cases. Most of the
time, git-annex unused --from remote doesn't see git-remote-annex keys
at all, because it does not record a location log for them.
On the other hand, git-annex unused does find them, since it does not
rely on the location log. And that's good because they're a local cache
that the user should be able to drop.
If, however, the user ran git-annex unused and then git-annex move
--unused --to remote, the keys would have a location log for that
remote. Then git-annex unused --from remote would see them, and would
consider them unused. Even when they are present on the special remote
they belong to. And that risks losing data if they drop the keys from
the special remote, but didn't expect it would delete git branches they
had pushed to it.
So, make git-annex unused --from skip git-remote-annex keys whose uuid
is the same as the remote.
2024-05-14 19:12:07 +00:00
|
|
|
|
|
|
|
{- Is the key a manifest or bundle key that belongs to the special remote
|
|
|
|
- with this uuid? -}
|
|
|
|
isGitRemoteAnnexKey :: UUID -> Key -> Bool
|
|
|
|
isGitRemoteAnnexKey u k =
|
|
|
|
case fromKey keyVariety k of
|
2024-05-16 13:52:34 +00:00
|
|
|
GitBundleKey -> sameuuid $ \b ->
|
git-annex unused --from remote skips its git-remote-annex keys
This turns out to only be necessary is edge cases. Most of the
time, git-annex unused --from remote doesn't see git-remote-annex keys
at all, because it does not record a location log for them.
On the other hand, git-annex unused does find them, since it does not
rely on the location log. And that's good because they're a local cache
that the user should be able to drop.
If, however, the user ran git-annex unused and then git-annex move
--unused --to remote, the keys would have a location log for that
remote. Then git-annex unused --from remote would see them, and would
consider them unused. Even when they are present on the special remote
they belong to. And that risks losing data if they drop the keys from
the special remote, but didn't expect it would delete git branches they
had pushed to it.
So, make git-annex unused --from skip git-remote-annex keys whose uuid
is the same as the remote.
2024-05-14 19:12:07 +00:00
|
|
|
-- Remove the checksum that comes after the UUID.
|
2024-05-16 13:52:34 +00:00
|
|
|
let b' = B8.dropWhileEnd (/= '-') b
|
|
|
|
in B8.take (B8.length b' - 1) b'
|
2024-05-20 19:41:09 +00:00
|
|
|
GitManifestKey -> sameuuid $ \b ->
|
|
|
|
-- Remove an optional extension after the UUID.
|
|
|
|
-- (A UUID never contains '.')
|
|
|
|
if '.' `B8.elem` b
|
|
|
|
then
|
|
|
|
let b' = B8.dropWhileEnd (/= '.') b
|
|
|
|
in B8.take (B8.length b' - 1) b'
|
|
|
|
else b
|
git-annex unused --from remote skips its git-remote-annex keys
This turns out to only be necessary is edge cases. Most of the
time, git-annex unused --from remote doesn't see git-remote-annex keys
at all, because it does not record a location log for them.
On the other hand, git-annex unused does find them, since it does not
rely on the location log. And that's good because they're a local cache
that the user should be able to drop.
If, however, the user ran git-annex unused and then git-annex move
--unused --to remote, the keys would have a location log for that
remote. Then git-annex unused --from remote would see them, and would
consider them unused. Even when they are present on the special remote
they belong to. And that risks losing data if they drop the keys from
the special remote, but didn't expect it would delete git branches they
had pushed to it.
So, make git-annex unused --from skip git-remote-annex keys whose uuid
is the same as the remote.
2024-05-14 19:12:07 +00:00
|
|
|
_ -> False
|
|
|
|
where
|
|
|
|
sameuuid f = fromUUID u == f (S.fromShort (fromKey keyName k))
|