better handling of finishing up incomplete incremental verify

Now it's run in VerifyStage.

I thought about keeping the file handle open, and resuming reading where
tailVerify left off. But that risks leaking open file handles, until the
GC closes them, if the deferred verification does not get resumed. Since
that could perhaps happen if there's an exception somewhere, I decided
that was too unsafe.

Instead, re-open the file, seek, and resume.

Sponsored-by: Dartmouth College's DANDI project
This commit is contained in:
Joey Hess 2021-08-16 14:50:21 -04:00
parent e0b7f391bd
commit c4aba8e032
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
7 changed files with 114 additions and 88 deletions

View file

@ -113,7 +113,8 @@ byteRetriever a k _m _miv callback = a k (callback . ByteContent)
-- A Retriever that writes the content of a Key to a provided file.
-- The action is responsible for updating the progress meter as it
-- retrieves data. The incremental verifier is updated in the background as
-- the action writes to the file.
-- the action writes to the file, but may not be updated with the entire
-- content of the file.
fileRetriever :: (FilePath -> Key -> MeterUpdate -> Annex ()) -> Retriever
fileRetriever a k m miv callback = do
f <- prepTmp k
@ -123,13 +124,10 @@ fileRetriever a k m miv callback = do
Just iv -> do
finished <- liftIO newEmptyTMVarIO
t <- liftIO $ async $ tailVerify iv f finished
retrieve
liftIO $ atomically $ putTMVar finished ()
liftIO (wait t) >>= \case
Nothing -> noop
Just deferredverify -> do
showAction (descVerify iv)
liftIO deferredverify
let finishtail = do
liftIO $ atomically $ putTMVar finished ()
liftIO (wait t)
retrieve `finally` finishtail
pruneTmpWorkDirBefore f (callback . FileContent . fromRawFilePath)
{- The base Remote that is provided to specialRemote needs to have