mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

Added a comment: keys db optimization

Browse source
This commit is contained in:
Ilya_Shlyakhter 2021-06-02 16:53:03 +00:00 committed by admin
parent 6f3f972355
commit 6a2bfad192
1 changed files with 14 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

14
doc/todo/speed_up_keys_db_update_with_git_streaming/comment_1_0afd62cbe52eba137fbfd1907337e273._comment Normal file
View file

@ -0,0 +1,14 @@
[[!comment format=mdwn
username="Ilya_Shlyakhter"
avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
subject="keys db optimization"
date="2021-06-02T16:53:02Z"
content="""
\"users who often switch between branches that have tens to hundreds of thousands of diverged files will find it slow\" -- that's my use case ;) Could one keys-to-files db be kept per branch?
Maybe, the keys db could be split, based e.g. on prefix of md5 of the key, into separate sqlite files, and the writing to them parallelized?
It's common to be working on a many-core machine.
Is the keys-to-locked-files db used for anything besides detecting keys used by more than one files? For that one purpose there might be faster solutions.
But, if it's implemented, maybe it also be used to remove the [[limitation|git-annex-preferred-content]] that \"when a command is run with the --all option, or in a bare repository, there is no filename associated with an annexed object, and so \"include=\" and \"exclude=\" will not match\"?
"""]]