CI to automate build of cargo lockfiles on different Alpine releases for git-annex aports https://gitlab.alpinelinux.org/alpine/aports/-/tree/master/community/git-annex
Find a file
Joey Hess f9c5aa84e0
add database benchmark
The benchmark shows that the database access is quite fast indeed!
And, it scales linearly to the number of keys, with one exception,
getAssociatedKey.

Based on this benchmark, I don't think I need worry about optimising
for cases where all files are locked and the database is mostly empty.
In those cases, database access will be misses, and according to this
benchmark, should add only 50 milliseconds to runtime.

(NB: There may be some overhead to getting the database opened and locking
the handle that this benchmark doesn't see.)

joey@darkstar:~/src/git-annex>./git-annex benchmark
setting up database with 1000
setting up database with 10000
benchmarking keys database/getAssociatedFiles from 1000 (hit)
time                 62.77 μs   (62.70 μs .. 62.85 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 62.81 μs   (62.76 μs .. 62.88 μs)
std dev              201.6 ns   (157.5 ns .. 259.5 ns)

benchmarking keys database/getAssociatedFiles from 1000 (miss)
time                 50.02 μs   (49.97 μs .. 50.07 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 50.09 μs   (50.04 μs .. 50.17 μs)
std dev              206.7 ns   (133.8 ns .. 295.3 ns)

benchmarking keys database/getAssociatedKey from 1000 (hit)
time                 211.2 μs   (210.5 μs .. 212.3 μs)
                     1.000 R²   (0.999 R² .. 1.000 R²)
mean                 211.0 μs   (210.7 μs .. 212.0 μs)
std dev              1.685 μs   (334.4 ns .. 3.517 μs)

benchmarking keys database/getAssociatedKey from 1000 (miss)
time                 173.5 μs   (172.7 μs .. 174.2 μs)
                     1.000 R²   (0.999 R² .. 1.000 R²)
mean                 173.7 μs   (173.0 μs .. 175.5 μs)
std dev              3.833 μs   (1.858 μs .. 6.617 μs)
variance introduced by outliers: 16% (moderately inflated)

benchmarking keys database/getAssociatedFiles from 10000 (hit)
time                 64.01 μs   (63.84 μs .. 64.18 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 64.85 μs   (64.34 μs .. 66.02 μs)
std dev              2.433 μs   (547.6 ns .. 4.652 μs)
variance introduced by outliers: 40% (moderately inflated)

benchmarking keys database/getAssociatedFiles from 10000 (miss)
time                 50.33 μs   (50.28 μs .. 50.39 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 50.32 μs   (50.26 μs .. 50.38 μs)
std dev              202.7 ns   (167.6 ns .. 252.0 ns)

benchmarking keys database/getAssociatedKey from 10000 (hit)
time                 1.142 ms   (1.139 ms .. 1.146 ms)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 1.142 ms   (1.140 ms .. 1.144 ms)
std dev              7.142 μs   (4.994 μs .. 10.98 μs)

benchmarking keys database/getAssociatedKey from 10000 (miss)
time                 1.094 ms   (1.092 ms .. 1.096 ms)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 1.095 ms   (1.095 ms .. 1.097 ms)
std dev              4.277 μs   (2.591 μs .. 7.228 μs)
2016-01-12 13:07:03 -04:00
Annex fix one more test failure with v6 unlocked file merge conflict resolution 2016-01-08 15:23:15 -04:00
Assistant avoid confusing git with a modified ctime in clean filter 2016-01-07 17:48:04 -04:00
Backend force hash to finish with file before returning 2016-01-06 22:09:36 -04:00
Build fix build warning with AMP ghc 2015-12-28 13:16:46 -04:00
CmdLine add database benchmark 2016-01-12 13:07:03 -04:00
Command add database benchmark 2016-01-12 13:07:03 -04:00
Common
Config webapp: Fix bugs that could result in a relative path such as "." being written to ~/.config/git-annex/autostart 2015-12-02 15:57:30 -04:00
Database add database benchmark 2016-01-12 13:07:03 -04:00
debian When annex.http-headers is used to set the User-Agent header, avoid sending User-Agent: git-annex 2016-01-11 12:10:38 -04:00
doc add database benchmark 2016-01-12 13:07:03 -04:00
Git optimise 2016-01-06 20:38:02 -04:00
Limit
Logs avoid unnecessary reading of git-annex branch data when matching on annex.largefiles 2015-12-04 15:06:41 -04:00
Messages avoid spawning file size polling thread when not in -J mode 2015-11-16 21:21:58 -04:00
Remote typo 2015-12-26 15:11:32 -04:00
RemoteDaemon
standalone Make the git-annex-standalone.deb prevent runshell from installing wrappers into $HOME/.ssh 2015-11-10 12:53:46 -04:00
static
templates webapp: Remove the "disable remote" feature from the UI. 2015-09-21 13:05:44 -04:00
Types test: Added --keep-failures option. 2016-01-06 13:44:12 -04:00
Upgrade use TopFilePath for associated files 2016-01-05 17:22:19 -04:00
Utility When annex.http-headers is used to set the User-Agent header, avoid sending User-Agent: git-annex 2016-01-11 12:10:38 -04:00
.ghci
.gitattributes
.gitignore add .stack-work 2015-12-28 17:19:07 -04:00
.mailmap alias for Yaroslav @ yahoo to mailmap 2015-09-09 12:38:59 -04:00
Annex.hs flush keys db queue even on exception 2015-12-23 19:38:18 -04:00
Assistant.hs unused import on windows 2015-04-20 11:27:47 -04:00
Backend.hs reorg 2015-12-15 15:34:28 -04:00
bash-completion.bash fix bash completion of filenames containing spaces 2015-07-20 11:17:35 -04:00
build.bat let build.bat also be run from cygwin terminal 2015-04-21 14:12:14 -04:00
BuildFlags.hs persistent-sqlite is now a hard build dependency, since v6 repository mode needs it. 2015-12-26 13:00:52 -04:00
CHANGELOG
Checks.hs
CmdLine.hs arrange for regional output manager to run when -J is enabled 2015-11-04 16:22:43 -04:00
Command.hs reorg 2015-12-15 15:34:28 -04:00
Common.hs add database benchmark 2016-01-12 13:07:03 -04:00
Config.hs init: Configure .git/info/attributes to use git-annex as a smudge filter. 2015-12-04 17:57:15 -04:00
configure.hs
COPYRIGHT
Creds.hs support gpg.program 2015-09-09 18:06:49 -04:00
Crypto.hs support gpg.program 2015-09-09 18:06:49 -04:00
ghci
git-annex.cabal add database benchmark 2016-01-12 13:07:03 -04:00
git-annex.hs wire tasty's option parser into the main program option parser 2015-07-13 13:20:10 -04:00
git-union-merge.hs
Git.hs init: Configure .git/info/attributes to use git-annex as a smudge filter. 2015-12-04 17:57:15 -04:00
INSTALL
Limit.hs reorg 2015-12-15 15:34:28 -04:00
Locations.hs improve name of keys database lock file 2015-12-31 13:28:18 -04:00
Logs.hs
Makefile defer find to after build 2015-12-28 17:14:29 -04:00
Messages.hs whereis --json: Make url list be included in machine-parseable form. 2016-01-06 12:33:32 -04:00
NEWS
README
Remote.hs enableremote: List uuids and descriptions of remotes that can be enabled, and accept either the uuid or the description in leu if the name. 2015-10-26 14:55:40 -04:00
Setup.hs improve warning 2015-11-03 11:06:33 -04:00
stack.yaml remove Inotify build flag 2015-12-28 14:46:01 -04:00
Test.hs defer deletion of test repos until end, fixes sqlite crash 2016-01-08 16:14:39 -04:00
Types.hs
Upgrade.hs Use git-annex init --version=6 to get v6 for now 2015-12-15 17:17:13 -04:00

git-annex allows managing files with git, without checking the file
contents into git. While that may seem paradoxical, it is useful when
dealing with files larger than git can currently easily handle, whether due
to limitations in memory, checksumming time, or disk space.

For documentation, see doc/ or <https://git-annex.branchable.com/>