merge: Now runs in constant space.

Before, a merge was first calculated, by running various actions that
called git and built up a list of lines, which were at the end sent
to git update-index. This necessarily used space proportional to the size
of the diff between the trees being merged.

Now, lines are streamed into git update-index from each of the actions in
turn.

Runtime size of git-annex merge when merging 50000 location log files
drops from around 100 mb to a constant 4 mb.

Presumably it runs quite a lot faster, too.
This commit is contained in:
Joey Hess 2011-11-15 23:28:01 -04:00
parent 922e9af528
commit 21a925dcf1
3 changed files with 34 additions and 25 deletions

1
debian/changelog vendored
View file

@ -13,6 +13,7 @@ git-annex (3.20111112) UNRELEASED; urgency=low
displayed)
* status: --fast is back
* Fix support for insteadOf url remapping. Closes: #644278
* merge: Now runs in constant space.
-- Joey Hess <joeyh@debian.org> Sat, 12 Nov 2011 14:50:21 -0400