try retrieval from more than one export location when the first fails

Combined with commit 0ffc59d341, this
fixes the case where there are duplicate files on the special remote,
and the first gets modified/deleted, while the second is still present.

directory, adb: Fixed a bug when importtree=yes, and multiple files in the
special remote have the same content, that caused it to refuse to get a
file from the special remote, incorrectly complaining that it had changed,
due to only accepting the inode+mtime of one file (that was since modified
or deleted) and not accepting the inode+mtime of other duplicate files.

Sponsored-by: Max Thoursie on Patreon
This commit is contained in:
Joey Hess 2022-09-20 13:33:57 -04:00
parent 0ffc59d341
commit 0756f4453d
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
2 changed files with 19 additions and 11 deletions

View file

@ -17,6 +17,12 @@ git-annex (10.20220823) UNRELEASED; urgency=medium
ignoreinode setting has been changed. When getting a file from such a ignoreinode setting has been changed. When getting a file from such a
remote, accept the content that would have been accepted with the remote, accept the content that would have been accepted with the
previous ignoreinode setting. previous ignoreinode setting.
* directory, adb: Fixed a bug when importtree=yes, and multiple files
in the special remote have the same content, that caused it to
refuse to get a file from the special remote, incorrectly complaining
that it had changed, due to only accepting the inode+mtime of one file
(that was since modified or deleted) and not accepting the inode+mtime
of other duplicate files.
-- Joey Hess <id@joeyh.name> Mon, 29 Aug 2022 15:03:04 -0400 -- Joey Hess <id@joeyh.name> Mon, 29 Aug 2022 15:03:04 -0400

View file

@ -327,11 +327,6 @@ adjustExportImport' isexport isimport r rs = do
db <- getexportdb dbv db <- getexportdb dbv
liftIO $ Export.getExportTree db k liftIO $ Export.getExportTree db k
getfirstexportloc dbv k = do
getexportlocs dbv k >>= \case
[] -> giveup "unknown export location"
(l:_) -> return l
getexportlocs dbv k = do getexportlocs dbv k = do
db <- getexportdb dbv db <- getexportdb dbv
liftIO $ Export.getExportTree db k >>= \case liftIO $ Export.getExportTree db k >>= \case
@ -340,6 +335,15 @@ adjustExportImport' isexport isimport r rs = do
, return [] , return []
) )
ls -> return ls ls -> return ls
tryexportlocs dbv k a =
go Nothing =<< getexportlocs dbv k
where
go Nothing [] = giveup "unknown export location"
go (Just ex) [] = throwM ex
go _ (l:ls) = tryNonAsync (a l) >>= \case
Right v -> return v
Left e -> go (Just e) ls
getkeycids ciddbv k = do getkeycids ciddbv k = do
db <- getciddb ciddbv db <- getciddb ciddbv
@ -350,9 +354,8 @@ adjustExportImport' isexport isimport r rs = do
-- have replaced with content not of the requested key, the content -- have replaced with content not of the requested key, the content
-- has to be strongly verified. -- has to be strongly verified.
retrieveKeyFileFromExport dbv k _af dest p = ifM (isVerifiable k) retrieveKeyFileFromExport dbv k _af dest p = ifM (isVerifiable k)
( do ( tryexportlocs dbv k $ \loc ->
l <- getfirstexportloc dbv k retrieveExport (exportActions r) k loc dest p >>= return . \case
retrieveExport (exportActions r) k l dest p >>= return . \case
UnVerified -> MustVerify UnVerified -> MustVerify
IncompleteVerify iv -> MustFinishIncompleteVerify iv IncompleteVerify iv -> MustFinishIncompleteVerify iv
v -> v v -> v
@ -362,9 +365,8 @@ adjustExportImport' isexport isimport r rs = do
retrieveKeyFileFromImport dbv ciddbv k af dest p = do retrieveKeyFileFromImport dbv ciddbv k af dest p = do
cids <- getkeycids ciddbv k cids <- getkeycids ciddbv k
if not (null cids) if not (null cids)
then do then tryexportlocs dbv k $ \loc ->
l <- getfirstexportloc dbv k snd <$> retrieveExportWithContentIdentifier (importActions r) loc cids dest (Left k) p
snd <$> retrieveExportWithContentIdentifier (importActions r) l cids dest (Left k) p
-- In case a content identifier is somehow missing, -- In case a content identifier is somehow missing,
-- try this instead. -- try this instead.
else if isexport else if isexport