mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

reproduced bug; started analysis

Browse source 
This commit was sponsored by Peter Hogg on Patreon.
This commit is contained in:
Joey Hess 2017-01-31 12:30:44 -04:00
parent 3300911b14
commit 5798013fb2
No known key found for this signature in database
GPG key ID: C910D9222512E3C7
1 changed files with 25 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

25
doc/todo/more_efficient_memory_usage_with_git-annex_unused/comment_1_e7811b548054d3d6851facb8d3bf8153._comment Normal file
View file

@ -0,0 +1,25 @@
[[!comment format=mdwn
username="joey"
subject="""comment 1"""
date="2017-01-31T15:36:58Z"
content="""
You need to provide at least a version number, and ideally enough
information to reproduce the bug when filing bug reports.
Anyhow, I ran git-annex (HEAD) unused in my big repo, and its memory use
got up to over 1 gb which is much more than I would expect (should be
a couple hundred mb max).
The memory growth happens in the stage when it's
constructing the bloom filter for the keys in the diff between the
index and other branches. In my big repo, those diffs are quite large;
eg I have a branch with 70k files and another with 0 files.
I replaced insertMB with noop, so the bloom filters are not really
populated, and it still uses as much memory. So
the memory is not being leaked by the bloom filters themselves, but
is instead being leaked when processing the branch diffs,
or something like that.
Need to profile to find what's leaking.
"""]]