mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

fix memory leak

Browse source 
Unfortunately, I don't fully understand why it was leaking using the old
method of a lazy bytestring. I just know that it was leaking, despite
neither hGetUntilMetered nor byteStringPopper seeming to leak by
themselves.

The new method avoids the lazy bytestring, and simply reads chunks from the
handle and streams them out to the http socket.
This commit is contained in:
Joey Hess 2014-11-04 15:22:08 -04:00
parent 29871e320c
commit fccdd61eec
2 changed files with 34 additions and 10 deletions
Download patch file Download diff file Expand all files Collapse all files

9
Remote/S3.hs
View file

@ -198,10 +198,11 @@ store r h = fileStorer $ \k f p -> do
let sz = if fsz - pos < partsz'
then fsz - pos
else partsz'
b <- liftIO $ hGetUntilMetered fh (< partsz') meter
let body = RequestBodyStream (fromIntegral sz) (mkPopper b)
S3.UploadPartResponse _ etag <- sendS3Handle h $
S3.uploadPart (bucket info) object partnum uploadid body
let numchunks = ceiling (fromIntegral sz / defaultChunkSize)
let popper = handlePopper numchunks defaultChunkSize p fh
let req = S3.uploadPart (bucket info) object partnum uploadid $
RequestBodyStream (fromIntegral sz) popper
S3.UploadPartResponse _ etag <- sendS3Handle h req
sendparts (offsetMeterUpdate meter (toBytesProcessed sz)) (etag:etags) (partnum + 1)
sendparts p [] 1