mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

improve transfer stages for some commands

Browse source 
move --to, copy --to, mirror --to: When concurrency is enabled, run cleanup
actions in separate job pool from uploads.

transferStages was confusingly named, it's only useful when doing downloads
as then the verify actions can be run concurrently with other downloads.
For commands that upload, there will be more concurrency from running
cleanup actions in a separate job pool.

As for sync, I left it using downloadStages although that's not optimal
for the part of a sync that uploads. Perhaps it should use the union of
both?
This commit is contained in:
Joey Hess 2020-05-26 11:55:50 -04:00
parent 0d82a88742
commit e04a931439
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
6 changed files with 22 additions and 10 deletions
Download patch file Download diff file Expand all files Collapse all files

13
Types/WorkerPool.hs
View file

@ -82,12 +82,13 @@ commandStages = UsedStages
, stageSet = S.fromList [PerformStage, CleanupStage]
}
-- | When a command is transferring content, it can use this instead.
-- Transfers are often bottlenecked on the network another disk than the one
-- containing the repository, while verification bottlenecks on
-- the disk containing the repository or on the CPU.
transferStages :: UsedStages
transferStages = UsedStages
-- | When a command is downloading content, it can use this instead.
-- Downloads are often bottlenecked on the network or another disk
-- than the one containing the repository, while verification bottlenecks
-- on the disk containing the repository or on the CPU. So, run the
-- transfer and verify stage separately.
downloadStages :: UsedStages
downloadStages = UsedStages
{ initialStage = TransferStage
, stageSet = S.fromList [TransferStage, VerifyStage]
}