avoid potentially very long bwlimit delay at start
I first saw this getting with -J2 over ssh, but later saw it also without the -J2. It was resuming, and the calulated unboundDelay was many minutes. The first update of the meter jumped to some large value, because of the resuming, and so it thought the BW was super fast. Avoid by waiting until the second meter update. Might be a good idea to also guard for the delay being many seconds and avoid waiting. But how many? If BW is legitimately super fast, and a remote happens to read more than a 32kb or so chunk at a time, it could in theory download megabytes or gigabytes of data before the first meter update. It would actually be appropriate then to delay for a long time, if the desired BW was low. Could make up some numbers that are sane now, but tech may improve. (BTW, pleased to see bwlimit does work with -J. I had worried that it might not, if the meter update happened in a different thread than the downloading, but it's done in the same thread.) Sponsored-by: Brett Eisenberg on Patreon
This commit is contained in:
parent
94a9116175
commit
64cac1a721
2 changed files with 17 additions and 16 deletions
|
@ -397,23 +397,27 @@ bwLimitMeterUpdate bwlimit duration meterupdate
|
|||
| bwlimit <= 0 = return meterupdate
|
||||
| otherwise = do
|
||||
nowtime <- getPOSIXTime
|
||||
mv <- newMVar (nowtime, 0)
|
||||
mv <- newMVar (nowtime, Nothing)
|
||||
return (mu mv)
|
||||
where
|
||||
mu mv n@(BytesProcessed i) = do
|
||||
endtime <- getPOSIXTime
|
||||
(starttime, previ) <- takeMVar mv
|
||||
(starttime, mprevi) <- takeMVar mv
|
||||
|
||||
case mprevi of
|
||||
Just previ -> do
|
||||
let runtime = endtime - starttime
|
||||
let currbw = fromIntegral (i - previ) / runtime
|
||||
let pausescale = if currbw > bwlimit'
|
||||
then (currbw / bwlimit') - 1
|
||||
else 0
|
||||
unboundDelay (floor (runtime * pausescale * msecs))
|
||||
Nothing -> return ()
|
||||
|
||||
let runtime = endtime - starttime
|
||||
let currbw = fromIntegral (i - previ) / runtime
|
||||
let pausescale = if currbw > bwlimit'
|
||||
then (currbw / bwlimit') - 1
|
||||
else 0
|
||||
unboundDelay (floor (runtime * pausescale * msecs))
|
||||
meterupdate n
|
||||
|
||||
nowtime <- getPOSIXTime
|
||||
putMVar mv (nowtime, i)
|
||||
putMVar mv (nowtime, Just i)
|
||||
|
||||
bwlimit' = fromIntegral (bwlimit * durationSeconds duration)
|
||||
msecs = fromIntegral oneSecond
|
||||
|
|
|
@ -10,9 +10,9 @@ works, it will probably work to put the delay in there. --[[Joey]]
|
|||
|
||||
[[confirmed]]
|
||||
|
||||
> Implemented and works well.
|
||||
>
|
||||
> A local git remote, when resuming an interrupted
|
||||
> Implemented and works well. [[done]] --[[Joey]]
|
||||
|
||||
> Note: A local git remote, when resuming an interrupted
|
||||
> transfer, has to hash the file (with default annex.verify settings),
|
||||
> and that hashing updates the progress bar, and so the bwlimit can kick
|
||||
> in and slow down that initial hashing, before any data copying begins.
|
||||
|
@ -20,8 +20,7 @@ works, it will probably work to put the delay in there. --[[Joey]]
|
|||
> remote you're wanting to limit disk IO. Only reason it might not be ok
|
||||
> is if the intent is to limit IO to the disk containing the remote
|
||||
> but not the one containing the annex repo. (This also probably
|
||||
> holds for the directory special remote.)
|
||||
>
|
||||
> holds for the directory special remote.)
|
||||
> Other remotes, including git over ssh, when resuming don't have that
|
||||
> problem. Looks like chunked special remotes narrowly avoid it, just
|
||||
> because their implementation choose to not do incremental verification
|
||||
|
@ -37,5 +36,3 @@ works, it will probably work to put the delay in there. --[[Joey]]
|
|||
> tend to be 32kb or so, and the pauses a small fraction of a second. So
|
||||
> mentioning this only for completeness.) --[[Joey]]
|
||||
|
||||
[[done]]
|
||||
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue