export appendonly support

Make `git annex export` check appendonly when removing a file from an
export, and not update the location log, since the remote still contains
the content.

This commit was supported by the NSF-funded DataLad project.
This commit is contained in:
Joey Hess 2018-08-30 11:18:20 -04:00
parent 02630b39ee
commit 8b39db20b5
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
4 changed files with 15 additions and 11 deletions

View file

@ -285,11 +285,15 @@ cleanupUnexport r ea db eks loc = do
removeExportedLocation db (asKey ek) loc
flushDbQueue db
remaininglocs <- liftIO $
concat <$> forM eks (\ek -> getExportedLocation db (asKey ek))
when (null remaininglocs) $
forM_ eks $ \ek ->
logChange (asKey ek) (uuid r) InfoMissing
-- A readonly remote can support removeExportLocation to remove
-- the file from the exported tree, but still retain the content
-- and allow retrieving it.
unless (Remote.readonly) $ do
remaininglocs <- liftIO $
concat <$> forM eks (\ek -> getExportedLocation db (asKey ek))
when (null remaininglocs) $
forM_ eks $ \ek ->
logChange (asKey ek) (uuid r) InfoMissing
removeEmptyDirectories ea db loc (map asKey eks)

View file

@ -79,7 +79,7 @@ gen r u c gc = do
, remotetype = remote
, availability = if bupLocal buprepo then LocallyAvailable else GloballyAvailable
, readonly = False
, appendonly = True
, appendonly = False
, mkUnavailable = return Nothing
, getInfo = return [("repo", buprepo)]
, claimUrl = Nothing

View file

@ -123,7 +123,9 @@ data RemoteA a = Remote
-- a Remote can be known to be readonly
, readonly :: Bool
-- a Remote can allow writes but not have a way to delete content
-- from it
-- from it. Note that an export remote that supports removeExport
-- to remove a file from the exported tree, but still retains the
-- content in accessible form should set this to True.
, appendonly :: Bool
-- a Remote can be globally available. (Ie, "in the cloud".)
, availability :: Availability

View file

@ -31,7 +31,7 @@ won't be updated, as discussed above.
## final plan
Add an "appendOnly" field to Remote, indicating it retains all content stored
in it.
in it. done
Let S3 remotes be configured with versioned=yes or something like that
(what does S3 call the feature?) which enables appendOnly.
@ -43,7 +43,7 @@ Make S3 refuse to removeKey when configured appendOnly, failing with an error.
Make `git annex export` check appendOnly when removing a file from an
export, and not update the location log, since the remote still contains
the content.
the content. done
Make git-annex sync and the assistant skip trying to drop from appendOnly
remotes since it's just going to fail.
@ -52,8 +52,6 @@ Make exporttree=yes remotes that are appendOnly be trusted, and not force
verification of content, since the usual concerns about losing data when an
export is updated by someone else don't apply.
Make bup an appendOnly remote.
When a file was deleted from an exported tree, and then put back
in a later exported tree, it might get re-uploaded even though the content
is still retained in the versioned remote. S3 might have a way to avoid