stop warning about removals from IA

In a test, I uploaded a pdf, and several files were derived from it.
After removing the pdf, the derived files went away after approximatly
half an hour. This window does not seem worth warning about every time.
Documented it in the tip.
This commit is contained in:
Joey Hess 2017-09-12 12:45:33 -04:00
parent da23dec7d3
commit 1332e6cec0
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
3 changed files with 7 additions and 15 deletions

View file

@ -9,8 +9,7 @@ git-annex (6.20170819) UNRELEASED; urgency=medium
* Support building with feed-1.0, while still supporting older versions. * Support building with feed-1.0, while still supporting older versions.
* init: Display an additional message when it detects a filesystem that * init: Display an additional message when it detects a filesystem that
allows writing to files whose write bit is not set. allows writing to files whose write bit is not set.
* S3: Allow removing files from IA, but warn about derived versions * S3: Allow removing files from IA.
potentially still existing there.
-- Joey Hess <id@joeyh.name> Mon, 28 Aug 2017 12:20:59 -0400 -- Joey Hess <id@joeyh.name> Mon, 28 Aug 2017 12:20:59 -0400

View file

@ -278,18 +278,11 @@ retrieveCheap _ _ _ = return False
- While it may remove the file, there are generally other files - While it may remove the file, there are generally other files
- derived from it that it does not remove. -} - derived from it that it does not remove. -}
remove :: S3Info -> S3Handle -> Remover remove :: S3Info -> S3Handle -> Remover
remove info h k = warnIARemoval info $ do remove info h k = do
res <- tryNonAsync $ sendS3Handle h $ res <- tryNonAsync $ sendS3Handle h $
S3.DeleteObject (T.pack $ bucketObject info k) (bucket info) S3.DeleteObject (T.pack $ bucketObject info k) (bucket info)
return $ either (const False) (const True) res return $ either (const False) (const True) res
warnIARemoval :: S3Info -> Annex a -> Annex a
warnIARemoval info a
| isIA info = do
warning "Derived versions of removed file may still be present in the Internet Archive"
a
| otherwise = a
checkKey :: Remote -> S3Info -> Maybe S3Handle -> CheckPresent checkKey :: Remote -> S3Info -> Maybe S3Handle -> CheckPresent
checkKey r info Nothing k = case getpublicurl info of checkKey r info Nothing k = case getpublicurl info of
Nothing -> do Nothing -> do
@ -345,7 +338,7 @@ retrieveExportS3 r info _k loc f p =
return True return True
removeExportS3 :: Remote -> S3Info -> Key -> ExportLocation -> Annex Bool removeExportS3 :: Remote -> S3Info -> Key -> ExportLocation -> Annex Bool
removeExportS3 r info _k loc = warnIARemoval info $ removeExportS3 r info _k loc =
catchNonAsync go (\e -> warning (show e) >> return False) catchNonAsync go (\e -> warning (show e) >> return False)
where where
go = withS3Handle (config r) (gitconfig r) (uuid r) $ \h -> do go = withS3Handle (config r) (gitconfig r) (uuid r) $ \h -> do

View file

@ -51,15 +51,15 @@ Then you can annex files and copy them to the remote as usual:
# git annex copy photo1.jpeg --fast --to archive-panama # git annex copy photo1.jpeg --fast --to archive-panama
copy (to archive-panama...) ok copy (to archive-panama...) ok
## update lag
It may take a while for archive.org to make files publically visible after It may take a while for archive.org to make files publically visible after
they've been uploaded. they've been uploaded.
## removing files
While files can be removed from the Internet Archive, While files can be removed from the Internet Archive,
[derived versions](https://archive.org/help/derivatives.php) [derived versions](https://archive.org/help/derivatives.php)
of some files may continued to be stored there after the originals of some files may continued to be stored there for a while
were removed. git-annex warns about this problem. after the originals were removed.
## exporting trees ## exporting trees