mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

convert sync to use seekFilteredKeys

Browse source 
This only speeds up sync --content from 34.75 to 33.17 seconds;
location log precaching will probably be a bigger win.
This commit is contained in:
Joey Hess 2020-07-13 15:02:52 -04:00
parent a290792a4f
commit df58609804
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
2 changed files with 13 additions and 13 deletions
Download patch file Download diff file Expand all files Collapse all files

20
Command/Sync.hs
View file

@ -70,6 +70,7 @@ import qualified Database.Export as Export
import Utility.Bloom import Utility.Bloom
import Utility.OptParse import Utility.OptParse
import Utility.Process.Transcript import Utility.Process.Transcript
import Utility.Tuple
import Control.Concurrent.MVar import Control.Concurrent.MVar
import qualified Data.Map as M import qualified Data.Map as M
@ -652,21 +653,20 @@ seekSyncContent o rs currbranch = do
liftIO $ not <$> isEmptyMVar mvar liftIO $ not <$> isEmptyMVar mvar
where where
seekworktree mvar l bloomfeeder = seekworktree mvar l bloomfeeder =
seekHelper id ww LsFiles.inRepo l seekFilteredKeys (gofile bloomfeeder mvar) $
>>= gofiles bloomfeeder mvar seekHelper fst3 ww LsFiles.inRepoDetails l
seekincludinghidden origbranch mvar l bloomfeeder = seekincludinghidden origbranch mvar l bloomfeeder =
seekHelper id ww (LsFiles.inRepoOrBranch origbranch) l seekFiltered (\f -> ifAnnexed f (gofile bloomfeeder mvar f) noop) $
>>= gofiles bloomfeeder mvar seekHelper id ww (LsFiles.inRepoOrBranch origbranch) l
ww = WarnUnmatchLsFiles ww = WarnUnmatchLsFiles
gofiles bloomfeeder mvar = mapM_ $ \f -> gofile bloom mvar f k =
ifAnnexed f go (Right bloom) mvar (AssociatedFile (Just f)) k
(go (Right bloomfeeder) mvar (AssociatedFile (Just f)))
noop
gokey mvar bloom (k, _) = go (Left bloom) mvar (AssociatedFile Nothing) k gokey mvar bloom (k, _) =
go (Left bloom) mvar (AssociatedFile Nothing) k
go ebloom mvar af k = do go ebloom mvar af k = do
-- Run syncFile as a command action so file transfers run -- Run syncFile as a command action so file transfers run

6
doc/todo/precache_logs_for_speed_with_cat-file_--buffer.mdwn
View file

@ -35,9 +35,9 @@ only gets the size of the blob, not its content.
> This was a win! Nearly 2x faster `git-annex get` seeking. > This was a win! Nearly 2x faster `git-annex get` seeking.
Some calls to ifAnnexed, whenAnnexed, lookupKey remain, and the above could Some calls to lookupKey remain, and the above could
be used to remove them and make it faster. In particular, Command.Sync has be used to remove them and make it faster. The ones in Annex.View and
a ifAnnexed. Command.Unused seem most likely to be able to be converted.
Also, limits that look up keys still do a key lookup, before the key is Also, limits that look up keys still do a key lookup, before the key is
looked up efficiently. (Before these changes, the same key lookup was done looked up efficiently. (Before these changes, the same key lookup was done