speed up very first tree import by 25%

Reading from the cidsdb is responsible for about 25% of the runtime of
an import. Since the cidmap is used to store the same information in
ram, the cidsdb is not written to during an import any longer. And so,
if it started off empty (and updateFromLog wasn't needed), those reads
can just be skipped.

This is kind of a cheesy optimisation, since after any import from any
special remote, the database will no longer be empty, so it's a single
use optimisation. But it's probably not uncommon to start by importing a
lot of files, and it can save a lot of time then.

Sponsored-by: Brock Spratlen on Patreon
This commit is contained in:
Joey Hess 2023-06-02 13:30:30 -04:00
parent b43fb4923f
commit fe1b2dfb4b
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
4 changed files with 38 additions and 28 deletions

View file

@ -280,14 +280,14 @@ adjustExportImport' isexport isimport r rs = do
Nothing -> ifM (liftIO $ atomically $ tryPutTMVar lcklckv ())
( do
db <- ContentIdentifier.openDb
ContentIdentifier.needsUpdateFromLog db >>= \case
db' <- ContentIdentifier.needsUpdateFromLog db >>= \case
Just v -> do
cidlck <- calcRepo' gitAnnexContentIdentifierLock
withExclusiveLock cidlck $
ContentIdentifier.updateFromLog db v
Nothing -> noop
liftIO $ atomically $ putTMVar dbtv db
return db
Nothing -> pure db
liftIO $ atomically $ putTMVar dbtv db'
return db'
-- loser waits for winner to open the db and
-- can then also use its handle
, liftIO $ atomically (readTMVar dbtv)