mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

Improved handling of --time-limit when combined with -J

Browse source 
When concurrency is enabled, there can be worker threads still running
when the time limit is checked. Exiting right there does not
give those threads time to finish what they're doing. Instead, the seeking
is wrapped up, and git-annex then shuts down cleanly.

The whole point of --time-limit existing, rather than using timeout(1)
when running git-annex is to let git-annex finish the action(s) it is
working on when the time limit is reached, and shut down cleanly.

I noticed this problem when investigating why restagePointerFile might
not have run after get/drop of an unlocked file. With --time-limit -J,
a worker thread may have finished updating a work tree file, and be killed
by the time limit check before it can run restagePointerFile. So despite
--time-limit running the shutdown actions, the work tree file didn't get
restaged.

Sponsored-by: Dartmouth College's DANDI project
This commit is contained in:
Joey Hess 2022-09-22 12:47:40 -04:00
parent 6f0566d704
commit 66bd4f80b3
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
5 changed files with 34 additions and 26 deletions
Download patch file Download diff file Expand all files Collapse all files

18
CmdLine/Action.hs
View file

@ -32,8 +32,8 @@ import qualified System.Console.Regions as Regions
{- Runs a command, starting with the check stage, and then
- the seek stage. Finishes by running the continuation.
-
- Can exit when there was a problem or when files were skipped.
- Also shows a count of any failures when that is enabled.
- Can exit when there was a problem or when a time or size limit was
- reached. Also shows a count of any failures when that is enabled.
-}
performCommandAction :: Bool -> Command -> CommandSeek -> Annex () -> Annex ()
performCommandAction canexit (Command { cmdcheck = c, cmdname = name }) seek cont = do
@ -43,19 +43,19 @@ performCommandAction canexit (Command { cmdcheck = c, cmdname = name }) seek con
finishCommandActions
cont
st <- Annex.getState id
when canexit $ liftIO $ case (Annex.errcounter st, Annex.skippedfiles st) of
when canexit $ liftIO $ case (Annex.errcounter st, Annex.reachedlimit st) of
(0, False) -> noop
(errcnt, False) -> do
showerrcount errcnt
exitWith $ ExitFailure 1
(0, True) -> exitskipped
(0, True) -> exitreachedlimit
(errcnt, True) -> do
showerrcount errcnt
exitskipped
exitreachedlimit
where
showerrcount cnt = hPutStrLn stderr $
name ++ ": " ++ show cnt ++ " failed"
exitskipped = exitWith $ ExitFailure 101
exitreachedlimit = exitWith $ ExitFailure 101
commandActions :: [CommandStart] -> Annex ()
commandActions = mapM_ commandAction
@ -328,7 +328,7 @@ checkSizeLimit (Just sizelimitvar) startmsg a =
Nothing -> do
fsz <- catchMaybeIO $ withObjectLoc k $
liftIO . getFileSize
maybe skipped go fsz
maybe reachedlimit go fsz
Nothing -> a
where
go sz = do
@ -342,6 +342,6 @@ checkSizeLimit (Just sizelimitvar) startmsg a =
else return False
if fits
then a
else skipped
else reachedlimit
skipped = Annex.changeState $ \s -> s { Annex.skippedfiles = True }
reachedlimit = Annex.changeState $ \s -> s { Annex.reachedlimit = True }