mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

export: re-send lost exported files after fsck notices they're gone

Browse source 
When content has been lost from an export remote and  git-annex fsck --from
remote has noticed it's gone, re-running git-annex export or git-annex sync
--content will re-upload it.

Note that normally there's no way to remove a single file from an export.
doc/design/exporting_trees_to_special_remotes.mdwn talks about this
in the section "dropping from exports and copying to exports". But, if
a file is somehow deleted or corrupted on the export, and fsck notices
this, it will update the location log to say it's missing.

So, checking the location log when determining if a file needs to be sent
to the export will let such missing files be added back in. There's
otherwise no way to do so. It does not fall afoul of the races documented
in the abovementioned section, I think.

This commit was sponsored by Ryan Newton on Patreon.
This commit is contained in:
Joey Hess 2018-06-14 12:22:12 -04:00
parent 4a3f1a15c5
commit b6e4ed9aa7
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
2 changed files with 10 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
CHANGELOG
View file

@ -14,6 +14,9 @@ git-annex (6.20180530) UNRELEASED; urgency=medium
* Display error messages that come from git-annex-shell when the p2p
protocol is used, so that diskreserve messages, IO errors, etc from
the remote side are visible again.
* When content has been lost from an export remote and
git-annex fsck --from remote has noticed it's gone, re-running
git-annex export or git-annex sync --content will re-upload it.
-- Joey Hess <id@joeyh.name> Wed, 30 May 2018 11:49:08 -0400

8
Command/Export.hs
View file

@ -201,7 +201,8 @@ fillExport r ea db new = do
startExport :: Remote -> ExportActions Annex -> ExportHandle -> MVar Bool -> Git.LsTree.TreeItem -> CommandStart
startExport r ea db cvar ti = do
ek <- exportKey (Git.LsTree.sha ti)
stopUnless (liftIO $ notElem loc <$> getExportedLocation db (asKey ek)) $ do
np <- notpresent ek
stopUnless (notpresent ek) $ do
showStart ("export " ++ name r) f
liftIO $ modifyMVar_ cvar (pure . const True)
next $ performExport r ea db ek af (Git.LsTree.sha ti) loc
@ -209,6 +210,11 @@ startExport r ea db cvar ti = do
loc = mkExportLocation f
f = getTopFilePath (Git.LsTree.file ti)
af = AssociatedFile (Just f)
notpresent ek = (||)
<$> liftIO (notElem loc <$> getExportedLocation db (asKey ek))
-- If content was removed from the remote, the export db
-- will still list it, so also check location tracking.
<*> (notElem (uuid r) <$> loggedLocations (asKey ek))
performExport :: Remote -> ExportActions Annex -> ExportHandle -> ExportKey -> AssociatedFile -> Sha -> ExportLocation -> CommandPerform
performExport r ea db ek af contentsha loc = do