mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

don't rely on exception for http 416

Browse source 
Fix a bug that could make resuming a download from the web fail when the
entire content of the file is actually already present locally.

What a mess that Request can throw exceptions or not, depending on how
it's configured. Makes it very hard if you need to handle some specific
http status codes in a function like this! Implementing everything two
ways did not seem appealing, if possible at all, so I decided to
override the Request if it did come configured to throw exception on
non-2xx http status. Other exceptions, like from http-client-restricted,
or due to a redirect to a non-http url, still get thrown.

This commit was sponsored by Luke Shumaker on Patreon.
This commit is contained in:
Joey Hess 2020-11-19 14:03:00 -04:00
parent 3991c8e43d
commit b90b9b936d
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
4 changed files with 60 additions and 18 deletions
Download patch file Download diff file Expand all files Collapse all files

2
doc/bugs/download_failed__58___Requested_Range_Not_Satisfiable.mdwn
View file

@ -41,3 +41,5 @@ Stock git-annex on Fedora 32:
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
Yes! git-annex has been working great for me so far, and is powering the bioinformatics chat podcast (https://bioinformatics.chat/). Thanks!
> [[fixed|done]] --[[Joey]]

30
doc/bugs/download_failed__58___Requested_Range_Not_Satisfiable/comment_1_44bc0678a9978b62ab133d64d3614610._comment Normal file
View file

@ -0,0 +1,30 @@
[[!comment format=mdwn
username="joey"
subject="""comment 1"""
date="2020-11-19T16:56:45Z"
content="""
To reproduce this, interrupt git-annex after it downloads the whole file,
but before it moves it from the download location into the annex. (Or,
let it get the file, then move the object back to the temp object location.)
This is a tricky case, because if the total file size is not
known when resuming the download, how can it detect if it's got it all
already? And git-annex does not always know the total file size, eg when
git-annex addurl --relaxed is used, and then git-annex get is later used
to download the content.
What git-annex already tried to do to detect this is,
when it got a 416 it looks for a Content-Range header "bytes */$size"
where $size is the same as the size of the file on disk.
That relied on the http library throwing an exception for the 416.
Thing is, http may or may not throw exceptions for non-2xx
responses, depending on the input Request. IMHO that is a very bad design,
it leads to this kind of bug, rather than making it evident with the data
types what is going on.
Currently downloadConduit takes a Request, and assumes it throws exceptions
for 416, but not for 401. Both can't be right.
Ok, fixed this mess..
"""]]