mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions
git-annex/Utility
History
Joey Hess 4224fae71f
optimise read and write for Keys database (untested) 
Writes are optimised by queueing up multiple writes when possible.
The queue is flushed after the Annex monad action finishes. That makes it
happen on program termination, and also whenever a nested Annex monad action
finishes.

Reads are optimised by checking once (per AnnexState) if the database
exists. If the database doesn't exist yet, all reads return mempty.

Reads also cause queued writes to be flushed, so reads will always be
consistent with writes (as long as they're made inside the same Annex monad).
A future optimisation path would be to determine when that's not necessary,
which is probably most of the time, and avoid flushing unncessarily.

Design notes for this commit:

- separate reads from writes
- reuse a handle which is left open until program
  exit or until the MVar goes out of scope (and autoclosed then)
- writes are queued
  - queue is flushed periodically
  - immediate queue flush before any read
  - auto-flush queue when database handle is garbage collected
  - flush queue on exit from Annex monad
    (Note that this may happen repeatedly for a single database connection;
    or a connection may be reused for multiple Annex monad actions,
    possibly even concurrent ones.)
- if database does not exist (or is empty) the handle
  is not opened by reads; reads instead return empty results
- writes open the handle if it was not open previously
2015-12-23 19:18:52 -04:00
..
DirWatcher addidential debugging 2015-08-03 11:27:53 -04:00
LockFile fix build warnings under ghc 7.10 2015-12-19 17:42:45 -04:00
LockPool add timeout for pid lock waiting 2015-11-12 17:12:54 -04:00
Process merge from propellor 2015-10-28 00:18:01 -04:00
Scheduled reorg quickcheck to a separate module 2015-11-17 15:49:22 -04:00
Applicative.hs update my email address and homepage url 2015-01-21 12:50:09 -04:00
Base64.hs avoid throwing exception when String is not encoded using the filesystem encoding 2015-08-12 10:57:48 -04:00
Batch.hs update my email address and homepage url 2015-01-21 12:50:09 -04:00
Bloom.hs fix bug in back-compat ifdef 2015-09-23 13:09:08 -04:00
CoProcess.hs update my email address and homepage url 2015-01-21 12:50:09 -04:00
CopyFile.hs avoid "Defined but not used" warning on android 2015-07-02 15:24:33 -04:00
Daemon.hs avoid crashing in checkDaemon when fcntl locking is not supported 2015-11-16 14:34:30 -04:00
Data.hs disable horrible tab warning, needed in every file that Setup.hs pulls in 2015-05-10 16:31:50 -04:00
DataUnits.hs info: Added --bytes option. 2015-04-12 14:08:40 -04:00
DBus.hs Revert "When listing DBus services, also list activatable services." 2015-06-02 14:38:24 -04:00
Directory.hs windows build warning fixes 2015-08-03 15:54:29 -04:00
DirWatcher.hs removed all uses of undefined from code base 2015-04-19 00:38:29 -04:00
DiskFree.hs update my email address and homepage url 2015-01-21 12:50:09 -04:00
Dot.hs update my email address and homepage url 2015-01-21 12:50:09 -04:00
DottedVersion.hs more {-# OPTIONS_GHC -fno-warn-tabs #-} ... Forcing people who have what is merely a difference of opinion to you to do this is a bit of an asshole move. Just saying. 2015-05-10 16:38:49 -04:00
Env.hs more {-# OPTIONS_GHC -fno-warn-tabs #-} ... Forcing people who have what is merely a difference of opinion to you to do this is a bit of an asshole move. Just saying. 2015-05-10 16:38:49 -04:00
Exception.hs generalize catchHardwareFault to catchIOErrorType 2015-12-06 16:26:38 -04:00
ExternalSHA.hs fsck: When checksumming a file fails due to a hardware fault, the file is now moved to the bad directory, and the fsck proceeds. Before, the fsck immediately failed. 2015-05-27 16:40:03 -04:00
FileMode.hs bring back some deleted functions that git-repair uses 2015-12-15 20:42:35 -04:00
FileSize.hs build fix 2015-01-20 17:27:42 -04:00
FileSystemEncoding.hs optimise read and write for Keys database (untested) 2015-12-23 19:18:52 -04:00
Format.hs fix use of hifalutin terminology 2015-11-16 14:37:31 -04:00
FreeDesktop.hs use intercalate instead of MissingH's join 2015-11-17 17:27:24 -04:00
Glob.hs qualify imports to avoid conflict with regex-tdfa-rc 2015-05-30 02:03:09 -04:00
Gpg.hs support gpg.program 2015-09-09 18:06:49 -04:00
Hash.hs package qualify imports 2015-08-14 17:23:25 -04:00
HumanNumber.hs update my email address and homepage url 2015-01-21 12:50:09 -04:00
HumanTime.hs generalize parseDuration so it can be used in the ReadM monad 2015-07-08 16:08:26 -04:00
InodeCache.hs use InodeCache when dropping a key to see if a pointer file can be safely reset 2015-12-09 17:54:54 -04:00
JSONStream.hs update my email address and homepage url 2015-01-21 12:50:09 -04:00
libdiskfree.c Use statvfs on OSX. 2015-10-19 17:09:06 -04:00
libdiskfree.h Renamed diskfree.c to avoid OSX case insensativity bug. 2012-04-13 11:26:39 -04:00
libkqueue.c update my email address and homepage url 2015-01-21 12:50:09 -04:00
libkqueue.h fix prototype 2012-06-19 01:57:19 -04:00
libmounts.c update my email address and homepage url 2015-01-21 12:50:09 -04:00
libmounts.h Got removable media mount detection working on Android. 2013-05-04 16:19:25 -04:00
LinuxMkLibs.hs rejigger imports for clean build with ghc 7.10's AMP changes 2015-05-10 16:20:30 -04:00
LockFile.hs use lock pools throughout git-annex 2015-05-19 14:09:52 -04:00
LockPool.hs lock pools to work around non-concurrency/composition safety of POSIX fcntl 2015-05-18 15:57:17 -04:00
LogFile.hs update my email address and homepage url 2015-01-21 12:50:09 -04:00
Lsof.hs update my email address and homepage url 2015-01-21 12:50:09 -04:00
Matcher.hs improve comment 2015-09-15 13:12:21 -04:00
Metered.hs refactor 2015-11-16 20:27:01 -04:00
Misc.hs merge hlint changes from propellor 2015-09-13 13:39:48 -04:00
Monad.hs disable horrible tab warning, needed in every file that Setup.hs pulls in 2015-05-10 16:31:50 -04:00
Mounts.hsc rejigger imports for clean build with ghc 7.10's AMP changes 2015-05-10 16:20:30 -04:00
Network.hs more {-# OPTIONS_GHC -fno-warn-tabs #-} ... Forcing people who have what is merely a difference of opinion to you to do this is a bit of an asshole move. Just saying. 2015-05-10 16:38:49 -04:00
NotificationBroadcaster.hs update my email address and homepage url 2015-01-21 12:50:09 -04:00
OptParse.hs I've been not documenting these import Preludes used to deal with the AMP transition 2015-09-15 11:32:47 -04:00
OSX.hs disable horrible tab warning, needed in every file that Setup.hs pulls in 2015-05-10 16:31:50 -04:00
Parallel.hs update my email address and homepage url 2015-01-21 12:50:09 -04:00
PartialPrelude.hs more {-# OPTIONS_GHC -fno-warn-tabs #-} ... Forcing people who have what is merely a difference of opinion to you to do this is a bit of an asshole move. Just saying. 2015-05-10 16:38:49 -04:00
Path.hs cleanup 2015-12-06 16:36:35 -04:00
Percentage.hs update my email address and homepage url 2015-01-21 12:50:09 -04:00
PID.hs update my email address and homepage url 2015-01-21 12:50:09 -04:00
PosixFiles.hs disable horrible tab warning, needed in every file that Setup.hs pulls in 2015-05-10 16:31:50 -04:00
Process.hs merge from propellor 2015-10-28 00:18:01 -04:00
QuickCheck.hs rejigger imports for clean build with ghc 7.10's AMP changes 2015-05-10 16:20:30 -04:00
Quvi.hs quvi may output utf-8 encoded data when the conifigured locale doesn't support that; avoid crashing on such invalid encoding. 2015-11-09 12:19:23 -04:00
Rsync.hs remove Params constructor from Utility.SafeCommand 2015-06-01 13:52:23 -04:00
SafeCommand.hs use intercalate instead of MissingH's join 2015-11-17 17:27:24 -04:00
Scheduled.hs import Data.Time.Format to ensure its Read instance for LocalTime is available 2015-11-21 13:36:30 -04:00
Shell.hs update my email address and homepage url 2015-01-21 12:50:09 -04:00
SimpleProtocol.hs comment typo 2015-10-12 16:32:52 -04:00
SRV.hs rejigger imports for clean build with ghc 7.10's AMP changes 2015-05-10 16:20:30 -04:00
SshConfig.hs redundant import 2015-09-22 12:31:54 -04:00
Tense.hs update my email address and homepage url 2015-01-21 12:50:09 -04:00
ThreadLock.hs update my email address and homepage url 2015-01-21 12:50:09 -04:00
ThreadScheduler.hs update my email address and homepage url 2015-01-21 12:50:09 -04:00
TList.hs update my email address and homepage url 2015-01-21 12:50:09 -04:00
Tmp.hs improve temp dir security 2015-12-15 20:21:48 -04:00
Touch.hsc Symlink timestamp preservation code uses functions from unix-2.7.0 when available, which should be more portable. 2015-10-21 02:22:18 -04:00
URI.hs oh foo, I didn't mean to include this in the prev commit 2015-06-11 16:43:59 -04:00
Url.hs refactor 2015-10-15 10:34:19 -04:00
UserInfo.hs disable horrible tab warning, needed in every file that Setup.hs pulls in 2015-05-10 16:31:50 -04:00
Verifiable.hs Removed dependency on haskell SHA library, instead using cryptohash >= 0.11.0. 2015-04-19 11:05:32 -04:00
WebApp.hs Added support for SHA3 hashed keys (in 8 varieties), when git-annex is built using the cryptonite library. 2015-08-06 15:02:25 -04:00
WinProcess.hs Fix Windows build to work with ghc 7.10 2015-09-01 14:51:14 -07:00
Yesod.hs fix build warning when building with yesod 1.2 and newer yesod-core 2015-08-03 15:42:44 -04:00