disable buggy concurrency in Command.Export

Fix a crash or potentially not all files being exported when sync -J
--content is used with an export remote.

Crash as described in fixed bug report.

waitForAllRunningCommandActions inserted in several points where all the
commandActions started before need to have finished before moving on to
the next stage of the export. A race across those points could have
maybe resulted in not all files being exported, or a wrong tree being
export.

For example, changeExport starting up an action like
a rename of A to B. Then, with that action still running, fillExport
uploading a new A, *before* the rename occurred. That race seems
unlikely to have happened. There are some other ones that this also
fixes.
This commit is contained in:
Joey Hess 2020-05-26 13:44:43 -04:00
parent 51432a6704
commit 864ba4ecaa
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
4 changed files with 52 additions and 8 deletions

View file

@ -136,6 +136,7 @@ changeExport r db (PreferredFiltered new) = do
(Git.DiffTree.file diff)
forM_ (incompleteExportedTreeishes old) $ \incomplete ->
mapdiff recover incomplete new
waitForAllRunningCommandActions
-- Diff the old and new trees, and delete or rename to new name all
-- changed files in the export. After this, every file that remains
@ -158,12 +159,14 @@ changeExport r db (PreferredFiltered new) = do
(Just oldf, Nothing) ->
startUnexport' r db oldf ek
_ -> stop
waitForAllRunningCommandActions
-- Rename from temp to new files.
seekdiffmap $ \(ek, (moldf, mnewf)) ->
case (moldf, mnewf) of
(Just _oldf, Just newf) ->
startMoveFromTempName r db ek newf
_ -> stop
waitForAllRunningCommandActions
ts -> do
warning "Resolving export conflict.."
forM_ ts $ \oldtreesha -> do
@ -181,6 +184,7 @@ changeExport r db (PreferredFiltered new) = do
(\diff -> commandAction $ startUnexport r db (Git.DiffTree.file diff) (unexportboth diff))
oldtreesha new
updateExportTree db emptyTree new
waitForAllRunningCommandActions
liftIO $ recordExportTreeCurrent db new
-- Waiting until now to record the export guarantees that,
@ -239,6 +243,7 @@ fillExport r db (PreferredFiltered newtree) mtbcommitsha = do
allfilledvar <- liftIO $ newMVar (AllFilled True)
commandActions $ map (startExport r db cvar allfilledvar) l
void $ liftIO $ cleanup
waitForAllRunningCommandActions
case mtbcommitsha of
Nothing -> noop
@ -484,3 +489,4 @@ filterPreferredContent r tree = logExportExcluded (uuid r) $ \logwriter -> do
)
-- Always export non-annexed files.
Nothing -> return (Just ti)