use fine-grained WorkerStages when transferring and verifying
This means that Command.Move and Command.Get don't need to manually set the stage, and is a lot cleaner conceptually. Also, this makes Command.Sync.syncFile use the worker pool better. In the scenario where it first downloads content and then uploads it to some other remotes, it will start in TransferStage, then enter VerifyStage and then go back to TransferStage for each transfer to the remotes. Before, it entered CleanupStage after the download, and stayed in it for the upload, so too many transfer jobs could run at the same time. Note that, in Remote.Git, it uses runTransfer and also verifyKeyContent inside onLocal. That has a Annex state for the remote, with no worker pool. So the resulting calls to enteringStage won't block in there. While Remote.Git.copyToRemote does do checksum verification, I realized that should not use a verification slot in the WorkerPool to do it. Because, it's reading back from eg, a removable disk to checksum. That will contend with other writes to that disk. It's best to treat that checksum verification as just part of the transer. So, removed the todo item about that, as there's nothing needing to be done.
This commit is contained in:
parent
53882ab4a7
commit
9d36c826c0
9 changed files with 13 additions and 34 deletions
18
Command.hs
18
Command.hs
|
@ -105,24 +105,6 @@ stop = return Nothing
|
|||
stopUnless :: Annex Bool -> Annex (Maybe a) -> Annex (Maybe a)
|
||||
stopUnless c a = ifM c ( a , stop )
|
||||
|
||||
{- This can be used in the perform stage to run the action that is the bulk
|
||||
- of the work to do in that stage. If the action succeeds, then any actions
|
||||
- run after it will be scheduled as if they were run in the cleanup stage
|
||||
- instead of the perform stage.
|
||||
-
|
||||
- This is not needed for a perform stage that uses `next` to run the
|
||||
- cleanup stage action. But sometimes a larger action is being built up
|
||||
- and it's not practical to separate out the cleanup stage part from the
|
||||
- rest of the action.
|
||||
-}
|
||||
performJob :: Observable a => Annex a -> Annex a
|
||||
performJob a = do
|
||||
r <- a
|
||||
if observeBool r
|
||||
then changeStageTo CleanupStage
|
||||
else noop
|
||||
return r
|
||||
|
||||
{- When acting on a failed transfer, stops unless it was in the specified
|
||||
- direction. -}
|
||||
checkFailedTransferDirection :: ActionItem -> Direction -> Annex (Maybe a) -> Annex (Maybe a)
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue