better handling of finishing up incomplete incremental verify

Now it's run in VerifyStage.

I thought about keeping the file handle open, and resuming reading where
tailVerify left off. But that risks leaking open file handles, until the
GC closes them, if the deferred verification does not get resumed. Since
that could perhaps happen if there's an exception somewhere, I decided
that was too unsafe.

Instead, re-open the file, seek, and resume.

Sponsored-by: Dartmouth College's DANDI project
This commit is contained in:
Joey Hess 2021-08-16 14:50:21 -04:00
parent e0b7f391bd
commit c4aba8e032
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
7 changed files with 114 additions and 88 deletions

View file

@ -6,6 +6,7 @@
-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE BangPatterns #-}
module Backend.Hash (
backends,
@ -280,19 +281,25 @@ md5Hasher = mkHasher md5 md5_context
mkIncrementalVerifier :: HashAlgorithm h => Context h -> Key -> IO IncrementalVerifier
mkIncrementalVerifier ctx key = do
v <- newIORef (Just ctx)
v <- newIORef (Just (ctx, 0))
return $ IncrementalVerifier
{ updateIncremental = \b ->
modifyIORef' v $ \case
Just ctx' -> Just (hashUpdate ctx' b)
(Just (ctx', n)) ->
let !ctx'' = hashUpdate ctx' b
!n' = n + fromIntegral (S.length b)
in (Just (ctx'', n'))
Nothing -> Nothing
, finalizeIncremental =
readIORef v >>= \case
Just ctx' -> do
(Just (ctx', _)) -> do
let digest = hashFinalize ctx'
return $ sameCheckSum key (show digest)
Nothing -> return False
, failIncremental = writeIORef v Nothing
, positionIncremental = readIORef v >>= \case
Just (_, n) -> return (Just n)
Nothing -> return Nothing
, descVerify = descChecksum
}