mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

optimise reconcileStaged with git cat-file streaming

Browse source 
Commit 428c91606b made it need to do more
work in situations like switching between very different branches.

Compare with seekFilteredKeys which has a similar optimisation. Might be
possible to factor out the common part from these?

Sponsored-by: Dartmouth College's Datalad project
This commit is contained in:
Joey Hess 2021-06-07 14:51:38 -04:00
parent 70dbe61fc2
commit 6ceb31a30a
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
2 changed files with 84 additions and 38 deletions
Download patch file Download diff file Expand all files Collapse all files

12
doc/todo/speed_up_keys_db_update_with_git_streaming.mdwn
View file

@ -5,3 +5,15 @@ Normally it's plenty fast enough, but users who often switch between
branches that have tens to hundreds of thousands of diverged files will
find it slow, and this should speed it up by somewhere around 3x (excluding
sqlite writes). --[[Joey]]
> Implemented this. Benchmarked it in a situation where 100,000 annexed
> files were added to the index (by checking out a branch with more annexed
> files). old: 50 seconds; new: 41 seconds
> Also benchmarked when 100,000 annexed files were removed from the index.
> old: 26 seconds; new: 17 seconds.
>
> Adding associated files to the sqlite db is clearly more expensive than
> removing from it.
>
> [[done]] --[[Joey]]