mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

Limit retrying of failed transfers when forward progress is being made to 5

Browse source 
To avoid some unusual edge cases where too much retrying could result in
far more data transfer than makes sense.
This commit is contained in:
Joey Hess 2020-09-04 12:46:37 -04:00
parent 820d4368b3
commit 1d244bafbd
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
3 changed files with 59 additions and 37 deletions
Download patch file Download diff file Expand all files Collapse all files

14
doc/todo/limit_forwardRetry.mdwn
View file

@ -4,7 +4,8 @@ one more byte got transferred than in the previous, failed try.
Suppose that a transfer was restarting from the beginning each time, and it
just so happened that each try got a tiny little bit further before
failing. Then transferring an `N` byte object could result in `sum [1..N]`
bytes being sent. Worst case.
bytes being sent. Worst case. (Real world it involves the size of chunks
sent in a failing operation, so probably `sum [1..N/1024]` or so.)
So I think forwardRetry should cap after some amount of automatic retrying.
Ie, it could give up after 5 retries. --[[Joey]]
@ -15,3 +16,14 @@ if a remote is doing that (unless some timing heuristics were used). Around
5 retries seems fairly reasonable for that case too, it would be unlikely
for a rsync transfer to keep failing so many times while still making
forward progess. --[[Joey]]
> Or could add data to remotes about this, but it would need to be added
> for external special remotes too, and this does not really seem worth the
> complication.
>
> I think, even if a remote does not support resuming like
> rsync, it makes sense to retry a few failed transfers if it's getting
> closer to success each time, because forward progress suggests whatever
> made it fail is becoming less of a problem.
[[done]] --[[Joey]]