From 6a2bfad192587f7efb4069c67449a20156447bbb Mon Sep 17 00:00:00 2001 From: Ilya_Shlyakhter Date: Wed, 2 Jun 2021 16:53:03 +0000 Subject: [PATCH] Added a comment: keys db optimization --- ...ent_1_0afd62cbe52eba137fbfd1907337e273._comment | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100644 doc/todo/speed_up_keys_db_update_with_git_streaming/comment_1_0afd62cbe52eba137fbfd1907337e273._comment diff --git a/doc/todo/speed_up_keys_db_update_with_git_streaming/comment_1_0afd62cbe52eba137fbfd1907337e273._comment b/doc/todo/speed_up_keys_db_update_with_git_streaming/comment_1_0afd62cbe52eba137fbfd1907337e273._comment new file mode 100644 index 0000000000..577480bcb9 --- /dev/null +++ b/doc/todo/speed_up_keys_db_update_with_git_streaming/comment_1_0afd62cbe52eba137fbfd1907337e273._comment @@ -0,0 +1,14 @@ +[[!comment format=mdwn + username="Ilya_Shlyakhter" + avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0" + subject="keys db optimization" + date="2021-06-02T16:53:02Z" + content=""" +\"users who often switch between branches that have tens to hundreds of thousands of diverged files will find it slow\" -- that's my use case ;) Could one keys-to-files db be kept per branch? + +Maybe, the keys db could be split, based e.g. on prefix of md5 of the key, into separate sqlite files, and the writing to them parallelized? +It's common to be working on a many-core machine. + +Is the keys-to-locked-files db used for anything besides detecting keys used by more than one files? For that one purpose there might be faster solutions. +But, if it's implemented, maybe it also be used to remove the [[limitation|git-annex-preferred-content]] that \"when a command is run with the --all option, or in a bare repository, there is no filename associated with an annexed object, and so \"include=\" and \"exclude=\" will not match\"? +"""]]