storeExportWithContentIdentifierM for directory special remote

Not sure if my reasoning about the races really holds.

It would certianly be possible to better guard against races by using
Linux-specific renameat2 with RENAME_EXCHANGE or RENAME_NOREPLACE.

Or by using link and relying on it not overwriting existing files -- but
that would need a filesystem that supports hard links and directory can
be used in filesystems that don't.
This commit is contained in:
Joey Hess 2019-03-04 14:46:25 -04:00
parent 1ec9e1494c
commit 88ccfaa78c
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
2 changed files with 49 additions and 4 deletions

View file

@ -387,4 +387,50 @@ retrieveExportWithContentIdentifierM dir loc cid dest mkkey p =
| otherwise = return Nothing
storeExportWithContentIdentifierM :: FilePath -> FilePath -> Key -> ExportLocation -> [ContentIdentifier] -> MeterUpdate -> Annex (Maybe ContentIdentifier)
storeExportWithContentIdentifierM dir = error "TODO"
storeExportWithContentIdentifierM dir src _k loc overwritablecids p =
liftIO $ catchDefaultIO Nothing $ do
createDirectoryIfMissing True destdir
docopy checkoverwrite
where
dest = dir </> fromExportLocation loc
(destdir, base) = splitFileName dest
template = relatedTemplate (base ++ ".tmp")
docopy cont = withTmpFileIn destdir template $ \tmpf tmph -> do
withMeteredFile src p (L.hPut tmph)
hFlush tmph
getFileStatus tmpf >>= mkContentIdentifier tmpf >>= \case
Nothing -> return Nothing
Just newcid -> cont newcid $ do
rename tmpf dest
return (Just newcid)
-- If the destination file already exists, it should only
-- be overwritten when its ContentIdentifier is in overwritablecids
-- or is the same as the ContentIdentifier of the replacement.
--
-- This should avoid races to the extent possible. However,
-- if something has the destination file open for write,
-- it could write to it after it's been overwritten with the new
-- content, and its write would be lost, and we don't need to
-- detect that. (In similar situations, git doesn't either!)
--
-- It follows that if something is written to the destination file
-- shortly before, it's acceptable to overwrite anyway, as that's
-- nearly indistinguishable from the above case.
--
-- So, it suffices to check if the destination file's current
-- content can be overwritten, and immediately overwrite it.
checkoverwrite newcid finalize = do
destst <- getFileStatus dest
if isRegularFile destst
then catchDefaultIO Nothing (mkContentIdentifier dest destst) >>= \case
Just destcid
| destcid `elem` overwritablecids ->
finalize
| destcid == newcid -> finalize
-- dest exists with other content
| otherwise -> return Nothing
-- dest does not exist, not overwriting
Nothing -> finalize
else return Nothing

View file

@ -274,9 +274,8 @@ data ImportActions a = ImportActions
--
-- Since other things can modify the same file on the special
-- remote, this must take care to not overwrite such modifications,
-- and only overwrite a file that has the same ContentIdentifier
-- as was passed to it, unless listContents can recover an
-- overwritten file.
-- and only overwrite a file that has one of the ContentIdentifiers
-- passed to it, unless listContents can recover an overwritten file.
--
-- Also, since there can be concurrent writers, the implementation
-- needs to make sure that the ContentIdentifier it returns