mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions
git-annex/doc
History
Joey Hess f9c5aa84e0
add database benchmark 
The benchmark shows that the database access is quite fast indeed!
And, it scales linearly to the number of keys, with one exception,
getAssociatedKey.

Based on this benchmark, I don't think I need worry about optimising
for cases where all files are locked and the database is mostly empty.
In those cases, database access will be misses, and according to this
benchmark, should add only 50 milliseconds to runtime.

(NB: There may be some overhead to getting the database opened and locking
the handle that this benchmark doesn't see.)

joey@darkstar:~/src/git-annex>./git-annex benchmark
setting up database with 1000
setting up database with 10000
benchmarking keys database/getAssociatedFiles from 1000 (hit)
time                 62.77 μs   (62.70 μs .. 62.85 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 62.81 μs   (62.76 μs .. 62.88 μs)
std dev              201.6 ns   (157.5 ns .. 259.5 ns)

benchmarking keys database/getAssociatedFiles from 1000 (miss)
time                 50.02 μs   (49.97 μs .. 50.07 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 50.09 μs   (50.04 μs .. 50.17 μs)
std dev              206.7 ns   (133.8 ns .. 295.3 ns)

benchmarking keys database/getAssociatedKey from 1000 (hit)
time                 211.2 μs   (210.5 μs .. 212.3 μs)
                     1.000 R²   (0.999 R² .. 1.000 R²)
mean                 211.0 μs   (210.7 μs .. 212.0 μs)
std dev              1.685 μs   (334.4 ns .. 3.517 μs)

benchmarking keys database/getAssociatedKey from 1000 (miss)
time                 173.5 μs   (172.7 μs .. 174.2 μs)
                     1.000 R²   (0.999 R² .. 1.000 R²)
mean                 173.7 μs   (173.0 μs .. 175.5 μs)
std dev              3.833 μs   (1.858 μs .. 6.617 μs)
variance introduced by outliers: 16% (moderately inflated)

benchmarking keys database/getAssociatedFiles from 10000 (hit)
time                 64.01 μs   (63.84 μs .. 64.18 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 64.85 μs   (64.34 μs .. 66.02 μs)
std dev              2.433 μs   (547.6 ns .. 4.652 μs)
variance introduced by outliers: 40% (moderately inflated)

benchmarking keys database/getAssociatedFiles from 10000 (miss)
time                 50.33 μs   (50.28 μs .. 50.39 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 50.32 μs   (50.26 μs .. 50.38 μs)
std dev              202.7 ns   (167.6 ns .. 252.0 ns)

benchmarking keys database/getAssociatedKey from 10000 (hit)
time                 1.142 ms   (1.139 ms .. 1.146 ms)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 1.142 ms   (1.140 ms .. 1.144 ms)
std dev              7.142 μs   (4.994 μs .. 10.98 μs)

benchmarking keys database/getAssociatedKey from 10000 (miss)
time                 1.094 ms   (1.092 ms .. 1.096 ms)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 1.095 ms   (1.095 ms .. 1.097 ms)
std dev              4.277 μs   (2.591 μs .. 7.228 μs)
2016-01-12 13:07:03 -04:00
..
Android Added a comment: ssh keys, github, xmpp 2015-12-28 22:09:24 +00:00
android
assistant Added a comment: @niklaas 2015-08-17 13:42:26 +00:00
automatic_conflict_resolution
backends comment 2015-07-06 12:05:29 -04:00
bare_repositories
bugs comment 2016-01-11 12:52:11 -04:00
coding_style response 2015-09-29 11:26:42 -04:00
contact
copies
design update for v6 unlocked files 2015-12-26 14:59:06 -04:00
devblog link 2016-01-08 16:37:47 -04:00
direct_mode removed 2015-05-27 21:36:08 +00:00
download sigh @ encfs... 2015-11-30 15:22:00 -04:00
ekg
encryption response 2015-09-17 13:03:19 -04:00
footer
forum comment 2016-01-11 12:34:01 -04:00
future_proofing
git-annex-copy comment 2015-07-06 13:42:01 -04:00
git-annex-direct Added a comment: … 2015-06-13 13:07:49 +00:00
git-annex-importfeed Added a comment 2016-01-02 21:02:51 +00:00
git-annex-unused comment 2016-01-11 12:30:52 -04:00
how_it_works commet 2016-01-11 12:21:45 -04:00
install link to neurodebian standalone deb 2015-12-22 12:34:28 -04:00
internals fix filename 2015-06-09 17:51:59 -04:00
license
links
metadata
news add news item for git-annex 5.20151218 2015-12-18 14:48:04 -04:00
not
polls
preferred_content use real names used in the content expressions 2015-06-23 04:45:58 +00:00
required_content
scalability
special_remotes comment 2015-12-10 11:22:28 -04:00
stickers Added a comment 2015-11-20 08:37:39 +00:00
sync
templates add copy of note.mdwn, since Makefile's ikiwiki invocation omits the basewiki 2015-12-26 16:40:11 -04:00
testimonials
tips 2016-01-08 07:38:01 +00:00
todo add database benchmark 2016-01-12 13:07:03 -04:00
trust
tuning
upgrades
use_case
users I'm feeling.. typoish 2015-11-10 17:00:06 -04:00
videos
walkthrough update walkthrough and add tip about using v6 unlocked files 2015-12-26 16:47:23 -04:00
Android.mdwn
assistant.mdwn
automatic_conflict_resolution.mdwn
backends.mdwn backends.mdwn: Typo fix, 'SHA_512E' → 'SHA3_512E' 2015-08-13 04:19:11 +02:00
bare_repositories.mdwn
bugs.mdwn set postformtext to perhaps make it clearer that this form at the top of the page is where one reports a new bug 2015-08-02 19:20:55 -04:00
builds.mdwn add i386-ancient build 2015-09-23 13:36:45 -04:00
chunking.mdwn
coding_style.mdwn m 2015-12-05 19:52:38 +00:00
comments.mdwn
contact.mdwn
contribute.mdwn
copies.mdwn
design.mdwn
devblog.mdwn
direct_mode.mdwn don't let git-annex direct be run in a v6 repo 2015-12-04 16:33:09 -04:00
distributed_version_control.mdwn
download.mdwn clean up obsolete branches 2015-06-12 14:37:35 -04:00
ekg.mdwn
encryption.mdwn
favicon.ico
forum.mdwn
future_proofing.mdwn link to new tip about encryption 2016-01-08 11:48:53 -04:00
git-annex-add.mdwn doc update 2015-12-15 14:17:34 -04:00
git-annex-addunused.mdwn expand manpages cross-references significantly 2015-05-29 12:12:55 -04:00
git-annex-addurl.mdwn addurl: Added --with-files option. 2015-12-22 12:20:39 -04:00
git-annex-assistant.mdwn meant to be in prev commit 2015-11-02 11:22:02 -04:00
git-annex-checkpresentkey.mdwn
git-annex-contentlocation.mdwn
git-annex-copy.mdwn expand manpages cross-references significantly 2015-05-29 12:12:55 -04:00
git-annex-dead.mdwn dead --key: Can be used to mark a key as dead. 2015-06-09 14:52:05 -04:00
git-annex-describe.mdwn expand manpages cross-references significantly 2015-05-29 12:12:55 -04:00
git-annex-diffdriver.mdwn
git-annex-direct.mdwn don't let git-annex direct be run in a v6 repo 2015-12-04 16:33:09 -04:00
git-annex-drop.mdwn drop -Jn 2015-11-04 17:13:20 -04:00
git-annex-dropkey.mdwn Brought back the setkey plumbing command that was removed in 2011, since we found a use case for it. Note that the command's syntax was changed for consistency. 2015-07-02 17:44:25 -04:00
git-annex-dropunused.mdwn expand manpages cross-references significantly 2015-05-29 12:12:55 -04:00
git-annex-edit.mdwn
git-annex-enableremote.mdwn enableremote: List uuids and descriptions of remotes that can be enabled, and accept either the uuid or the description in leu if the name. 2015-10-26 14:55:40 -04:00
git-annex-examinekey.mdwn
git-annex-expire.mdwn
git-annex-find.mdwn expand manpages cross-references significantly 2015-05-29 12:12:55 -04:00
git-annex-findref.mdwn
git-annex-fix.mdwn annex.thin 2015-12-27 15:59:59 -04:00
git-annex-forget.mdwn
git-annex-fromkey.mdwn
git-annex-fsck.mdwn parallel fsck (yes, these changes are all it takes now!) 2015-11-04 16:28:14 -04:00
git-annex-fuzztest.mdwn
git-annex-get.mdwn get --incomplete: New option to resume any interrupted downloads. 2015-06-02 14:20:38 -04:00
git-annex-group.mdwn expand manpages cross-references significantly 2015-05-29 12:12:55 -04:00
git-annex-groupwanted.mdwn expand manpages cross-references significantly 2015-05-29 12:12:55 -04:00
git-annex-import.mdwn work around git check-ignore --batch bad exit status bug, and bring back import -J 2015-11-06 15:39:51 -04:00
git-annex-importfeed.mdwn doc/*.mdwn: Minor fixes (typos, letter case) 2015-07-26 04:21:06 +02:00
git-annex-indirect.mdwn don't let git-annex direct be run in a v6 repo 2015-12-04 16:33:09 -04:00
git-annex-info.mdwn
git-annex-init.mdwn Use git-annex init --version=6 to get v6 for now 2015-12-15 17:17:13 -04:00
git-annex-initremote.mdwn Special remotes configured with autoenable=true will be automatically enabled when git-annex init is run. 2015-09-14 14:49:48 -04:00
git-annex-list.mdwn expand manpages cross-references significantly 2015-05-29 12:12:55 -04:00
git-annex-lock.mdwn v6 git-annex unlock 2015-12-10 16:12:48 -04:00
git-annex-log.mdwn remove redundant options header 2015-07-06 18:22:27 -04:00
git-annex-lookupkey.mdwn doc/*.mdwn: Various typo fixes 2015-05-30 16:54:14 +02:00
git-annex-map.mdwn
git-annex-matching-options.mdwn
git-annex-merge.mdwn expand manpages cross-references significantly 2015-05-29 12:12:55 -04:00
git-annex-metadata.mdwn expand manpages cross-references significantly 2015-05-29 12:12:55 -04:00
git-annex-migrate.mdwn expand manpages cross-references significantly 2015-05-29 12:12:55 -04:00
git-annex-mirror.mdwn expand manpages cross-references significantly 2015-05-29 12:12:55 -04:00
git-annex-move.mdwn expand manpages cross-references significantly 2015-05-29 12:12:55 -04:00
git-annex-numcopies.mdwn
git-annex-pre-commit.mdwn avoid pre-commit hook messing up new-style unlocked files in v6 repo 2015-12-09 15:18:54 -04:00
git-annex-preferred-content.mdwn
git-annex-proxy.mdwn improve docs 2015-08-04 14:37:43 -04:00
git-annex-readpresentkey.mdwn
git-annex-registerurl.mdwn expand manpages cross-references significantly 2015-05-29 12:12:55 -04:00
git-annex-reinit.mdwn Special remotes configured with autoenable=true will be automatically enabled when git-annex init is run. 2015-09-14 14:49:48 -04:00
git-annex-reinject.mdwn Do verification of checksums of annex objects downloaded from remotes. 2015-10-01 15:56:39 -04:00
git-annex-rekey.mdwn
git-annex-remotedaemon.mdwn
git-annex-repair.mdwn expand manpages cross-references significantly 2015-05-29 12:12:55 -04:00
git-annex-required.mdwn add the date this was added 2015-08-18 18:53:25 +00:00
git-annex-resolvemerge.mdwn
git-annex-rmurl.mdwn expand manpages cross-references significantly 2015-05-29 12:12:55 -04:00
git-annex-schedule.mdwn expand manpages cross-references significantly 2015-05-29 12:12:55 -04:00
git-annex-semitrust.mdwn expand manpages cross-references significantly 2015-05-29 12:12:55 -04:00
git-annex-setkey.mdwn Do verification of checksums of annex objects downloaded from remotes. 2015-10-01 15:56:39 -04:00
git-annex-setpresentkey.mdwn
git-annex-shell.mdwn add unlocked flag for git-annex-shell recvkey 2015-12-26 13:59:27 -04:00
git-annex-smudge.mdwn fix docs 2015-12-16 16:00:41 -04:00
git-annex-status.mdwn status: Show added but not yet committed files. 2015-09-22 17:32:28 -04:00
git-annex-sync.mdwn 2016-01-05 22:44:15 +00:00
git-annex-test.mdwn test: Added --keep-failures option. 2016-01-06 13:44:12 -04:00
git-annex-testremote.mdwn
git-annex-transferkey.mdwn
git-annex-transferkeys.mdwn
git-annex-trust.mdwn expand manpages cross-references significantly 2015-05-29 12:12:55 -04:00
git-annex-unannex.mdwn
git-annex-undo.mdwn expand manpages cross-references significantly 2015-05-29 12:12:55 -04:00
git-annex-ungroup.mdwn expand manpages cross-references significantly 2015-05-29 12:12:55 -04:00
git-annex-uninit.mdwn expand manpages cross-references significantly 2015-05-29 12:12:55 -04:00
git-annex-unlock.mdwn annex.thin 2015-12-27 15:59:59 -04:00
git-annex-untrust.mdwn expand manpages cross-references significantly 2015-05-29 12:12:55 -04:00
git-annex-unused.mdwn fix man format 2015-07-10 16:52:53 -04:00
git-annex-upgrade.mdwn link to release notes and upgrades notes 2015-06-03 14:30:03 +00:00
git-annex-vadd.mdwn expand manpages cross-references significantly 2015-05-29 12:12:55 -04:00
git-annex-vcycle.mdwn expand manpages cross-references significantly 2015-05-29 12:12:55 -04:00
git-annex-version.mdwn
git-annex-vfilter.mdwn expand manpages cross-references significantly 2015-05-29 12:12:55 -04:00
git-annex-vicfg.mdwn
git-annex-view.mdwn expand 2015-07-20 13:48:01 -04:00
git-annex-vpop.mdwn expand manpages cross-references significantly 2015-05-29 12:12:55 -04:00
git-annex-wanted.mdwn expand manpages cross-references significantly 2015-05-29 12:12:55 -04:00
git-annex-watch.mdwn expand manpages cross-references significantly 2015-05-29 12:12:55 -04:00
git-annex-webapp.mdwn expand manpages cross-references significantly 2015-05-29 12:12:55 -04:00
git-annex-whereis.mdwn expand manpages cross-references significantly 2015-05-29 12:12:55 -04:00
git-annex-xmppgit.mdwn
git-annex.mdwn add database benchmark 2016-01-12 13:07:03 -04:00
git-union-merge.mdwn
how_it_works.mdwn
index.mdwn
install.mdwn remove obsolte note about sid package being out of date 2015-12-22 12:31:58 -04:00
internals.mdwn doc improvements 2015-12-27 16:06:11 -04:00
license.mdwn
location_tracking.mdwn
logo-old-bw.svg
logo-old.png
logo-old.svg
logo-old_small.png
logo.mdwn copy license info for logo onto logo page 2015-11-20 16:25:08 -04:00
logo.svg
logo_16x16.png
logo_32x32.png
logo_small.png
meta.mdwn
metadata.mdwn
news.mdwn
not.mdwn git-lfs lacks partial checkout support 2015-11-23 12:44:54 -04:00
polls.mdwn 2015 user survey 2015-10-19 15:15:04 -04:00
preferred_content.mdwn doc/*.mdwn: Minor fixes (typos, letter case) 2015-07-26 04:21:06 +02:00
privacy.mdwn
publicrepos.mdwn Fix formatting (grrr) 2016-01-02 23:29:38 +00:00
related_software.mdwn link to not 2015-08-18 13:43:01 +00:00
repomap.png
required_content.mdwn
scalability.mdwn
shortcuts.mdwn
sidebar.mdwn
sitemap.mdwn
special_remotes.mdwn add Blackblaze B2 extranal special remote to the list 2015-12-11 11:06:02 -04:00
stickers.mdwn improve 2015-11-19 18:56:24 -04:00
submodules.mdwn
summary.mdwn
sync.mdwn
testimonials.mdwn
thanks.mdwn update 2015-10-20 00:43:18 -04:00
tips.mdwn
todo.mdwn set postformtext to perhaps make it clearer that this form at the top of the page is where one reports a new bug 2015-08-02 19:20:55 -04:00
transferring_data.mdwn
trust.mdwn fix link display so the user reading this isn't led to beleive there are commands git-annex-$foo 2015-09-09 15:56:06 -04:00
tuning.mdwn doc/*.mdwn: Various typo fixes 2015-05-30 16:54:14 +02:00
upgrades.mdwn annex.thin 2015-12-27 15:59:59 -04:00
users.mdwn
videos.mdwn
Void.mdwn Creating initial Void Linux support page to match all the other Linux distros. 2015-09-07 14:03:01 +00:00
walkthrough.mdwn