get -J: Download different files from different remotes when the remotes have the same costs.

Only done in -J mode because only if there's concurrency can downloading
from two remotes be faster. Without concurrency, it's likely the case that
sequential downloads from the same remote are faster than switching back
and forth between two remotes.

There is some hairy MVar code here, but basically it just keeps
the activeremotes MVar full except when deciding which remote to assign
to a thread.

Also affects gets by sync --content -J

This commit was sponsored by Jochen Bartl.
This commit is contained in:
Joey Hess 2016-09-06 12:42:50 -04:00
parent eb469bd139
commit 31289da691
No known key found for this signature in database
GPG key ID: C910D9222512E3C7
6 changed files with 135 additions and 59 deletions

View file

@ -89,16 +89,17 @@ getKey' key afile = dispatch
showNote "not available"
showlocs
return False
dispatch remotes = notifyTransfer Download afile $ trycopy remotes remotes
trycopy full [] _ = do
Remote.showTriedRemotes full
showlocs
return False
trycopy full (r:rs) witness =
ifM (probablyPresent r)
( docopy r witness <||> trycopy full rs witness
, trycopy full rs witness
dispatch remotes = notifyTransfer Download afile $ \witness -> do
ok <- pickRemote remotes $ \r -> ifM (probablyPresent r)
( docopy r witness
, return False
)
if ok
then return ok
else do
Remote.showTriedRemotes remotes
showlocs
return False
showlocs = Remote.showLocations False key []
"No other repository is known to contain the file."
-- This check is to avoid an ugly message if a remote is a