mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

export retrieval fallback to handle S3 remote with partially missing version IDs

Browse source 
When key-based retrieval from a S3 remote with exporttree=yes
appendonly=yes fails, fall back to trying to retrieve from the exported
tree. This allows downloads of files that were exported to such a remote
before versioning was enabled on it.

This is useful at least for a transition for users who got into that
situation, so they can download content from their S3 remote. May want to
remove this in the future though, since normally trying to download the
second time is only extra work.

This commit was sponsored by Brock Spratlen on Patreon.
This commit is contained in:
Joey Hess 2019-01-30 13:23:03 -04:00
parent 9216718fa0
commit 720e5fda5c
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
2 changed files with 16 additions and 5 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
CHANGELOG
View file

@ -3,6 +3,10 @@ git-annex (7.20190130) UNRELEASED; urgency=medium
* Fix false positive in export conflict detection, that occurred
when the same tree was exported by multiple clones. Previous fix was
incomplete.
* When key-based retrieval from a S3 remote with exporttree=yes appendonly=yes
fails, fall back to trying to retrieve from the exported tree.
This allows downloads of files that were exported to such a remote
before versioning was enabled on it.
-- Joey Hess <id@joeyh.name> Wed, 30 Jan 2019 12:30:22 -0400

17
Remote/Helper/Export.hs
View file

@ -141,11 +141,18 @@ adjustExportable r = case M.lookup "exporttree" (config r) of
-- with content not of the requested key,
-- the content has to be strongly verified.
--
-- But, appendonly remotes have a key/value store,
-- so don't need to use retrieveExport.
, retrieveKeyFile = if appendonly r
then retrieveKeyFile r
else retrieveKeyFileFromExport getexportlocs exportinconflict
-- appendonly remotes have a key/value store,
-- so don't need to use retrieveExport. However,
-- fall back to it if retrieveKeyFile fails.
, retrieveKeyFile = \k af dest p ->
let retrieveexport = retrieveKeyFileFromExport getexportlocs exportinconflict k af dest p
in if appendonly r
then do
ret@(ok, _v) <- retrieveKeyFile r k af dest p
if ok
then return ret
else retrieveexport
else retrieveexport
, retrieveKeyFileCheap = if appendonly r
then retrieveKeyFileCheap r
else \_ _ _ -> return False