mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

avoid double work in git-annex init, second try

Browse source 
reconcileStaged populates the db, so scanAnnexedFiles does not need to
do it again. It still makes a pass over the HEAD tree, but populating
the db was most of the expensive part.

Benchmarking with 100,000 files, git-annex init now takes 40 seconds,
vs 37 seconds with the old, buggy version of this fix. It should be
possible to win those 3 precious seconds per 100k files back, in the
case when when annex.thin is not set, with improvements to reconcileStaged
that avoid needing this second pass.

Sponsored-by: Dartmouth College's Datalad project
This commit is contained in:
Joey Hess 2021-06-08 09:27:53 -04:00
parent 22185b4a4e
commit c941ab6f5b
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
3 changed files with 10 additions and 19 deletions
Download patch file Download diff file Expand all files Collapse all files

4
Database/Keys.hs
View file

@ -339,7 +339,7 @@ reconcileStaged qh = do
(asTopFilePath file)
(SQL.WriteHandle qh)
when (dstmode /= fmtTreeItemType TreeSymlink) $
reconcilerace (asTopFilePath file) key
reconcilepointerfile (asTopFilePath file) key
return True
Nothing -> return False
procdiff mdfeeder rest
@ -367,7 +367,7 @@ reconcileStaged qh = do
_ -> return conflicted -- parse failed
procmergeconflictdiff _ _ conflicted = return conflicted
reconcilerace file key = do
reconcilepointerfile file key = do
caches <- liftIO $ SQL.getInodeCaches key (SQL.ReadHandle qh)
keyloc <- calcRepo (gitAnnexLocation key)
keypopulated <- sameInodeCache keyloc caches