mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

heard from hS3 author

Browse source
This commit is contained in:
Joey Hess 2011-04-19 17:56:56 -04:00
parent 2b742dcd78
commit 936ad63cf4
1 changed files with 9 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

11
doc/bugs/S3_memory_leaks.mdwn
View file

@ -1,14 +1,21 @@
S3 has two memory leaks. S3 has two memory leaks.
## with encryption
One only occurs with encryption. It was was fixed for bup, but One only occurs with encryption. It was was fixed for bup, but
not yet for S3, in 5985acdfad8a6791f0b2fc54a1e116cee9c12479. not yet for S3, in 5985acdfad8a6791f0b2fc54a1e116cee9c12479.
(The fix I used for bup doesn't seem to work with S3.) (The fix I used for bup doesn't seem to work with S3.)
## always
The other occurs independant of encryption use. Copying a 100 mb The other occurs independant of encryption use. Copying a 100 mb
file to S3 causes an immediate sharp memory spike to 119 mb. file to S3 causes an immediate sharp memory spike to 119 mb.
Copying the file back from S3 causes a slow memory increase toward 119 mb. Copying the file back from S3 causes a slow memory increase toward 119 mb.
It's likely that this memory is used by the hS3 library, if it does not It's likely that this memory is used by the hS3 library, if it does not
construct the message to Amazon lazily. (And it may not be possible to construct the message to Amazon lazily. (And it may not be possible to
construct it lazily, if it includes checksum headers..) I have construct it lazily, if it includes checksum headers..)
emailed the hS3 author about this.
I have emailed the hS3 author about this. He wrote back quickly, seems
only getting the size of the file is causing it to be buffered, and a quick
fix should be forthcoming.
--[[Joey]] --[[Joey]]