mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

convert sync to use seekFilteredKeys

Browse source 
This only speeds up sync --content from 34.75 to 33.17 seconds;
location log precaching will probably be a bigger win.
This commit is contained in:
Joey Hess 2020-07-13 15:02:52 -04:00
parent a290792a4f
commit df58609804
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
2 changed files with 13 additions and 13 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

20
Command/Sync.hs
View file

@ -70,6 +70,7 @@ import qualified Database.Export as Export
import Utility.Bloom
import Utility.OptParse
import Utility.Process.Transcript
import Utility.Tuple
import Control.Concurrent.MVar
import qualified Data.Map as M
@ -652,21 +653,20 @@ seekSyncContent o rs currbranch = do
liftIO $ not <$> isEmptyMVar mvar
where
seekworktree mvar l bloomfeeder =
seekHelper id ww LsFiles.inRepo l
>>= gofiles bloomfeeder mvar
seekFilteredKeys (gofile bloomfeeder mvar) $
seekHelper fst3 ww LsFiles.inRepoDetails l
seekincludinghidden origbranch mvar l bloomfeeder =
seekHelper id ww (LsFiles.inRepoOrBranch origbranch) l
>>= gofiles bloomfeeder mvar
seekincludinghidden origbranch mvar l bloomfeeder =
seekFiltered (\f -> ifAnnexed f (gofile bloomfeeder mvar f) noop) $
seekHelper id ww (LsFiles.inRepoOrBranch origbranch) l
ww = WarnUnmatchLsFiles
gofiles bloomfeeder mvar = mapM_ $ \f ->
ifAnnexed f
(go (Right bloomfeeder) mvar (AssociatedFile (Just f)))
noop
gofile bloom mvar f k =
go (Right bloom) mvar (AssociatedFile (Just f)) k
gokey mvar bloom (k, _) = go (Left bloom) mvar (AssociatedFile Nothing) k
gokey mvar bloom (k, _) =
go (Left bloom) mvar (AssociatedFile Nothing) k
go ebloom mvar af k = do
-- Run syncFile as a command action so file transfers run

6
doc/todo/precache_logs_for_speed_with_cat-file_--buffer.mdwn
View file

@ -35,9 +35,9 @@ only gets the size of the blob, not its content.
> This was a win! Nearly 2x faster `git-annex get` seeking.
Some calls to ifAnnexed, whenAnnexed, lookupKey remain, and the above could
be used to remove them and make it faster. In particular, Command.Sync has
a ifAnnexed.
Some calls to lookupKey remain, and the above could
be used to remove them and make it faster. The ones in Annex.View and
Command.Unused seem most likely to be able to be converted.
Also, limits that look up keys still do a key lookup, before the key is
looked up efficiently. (Before these changes, the same key lookup was done