share single BranchState amoung all threads
This fixes a problem when git-annex testremote is run against a cluster
accessed via the http server. Annex.Cluster uses the location log
to find nodes that contain a key when checking if the key is present or getting
it. Just after a key was stored to a cluster node, reading the location log
was not getting the UUID of that node.
Apparently the Annex action that wrote to the location log, and the one
that read from it were run with two different Annex states. The http server
does use several different Annex threads.
BranchState was part of the AnnexState, and so two threads could have
different BranchStates.
Moved BranchState to the AnnexRead, so all threads will see the common state.
This might possibly impact performance. If one thread is writing changes to the
branch, and another thread is reading from the branch, the writing thread will
now invalidate the BranchState's cache, which will cause the reading thread to
need to do extra work. But correctness is surely more important. If did is
found to have impacted performance, it could probably be dealt with by doing
smarter BranchState cache invalidation.
Another way this might impact performance is that the BranchState has a small
cache. If several threads were reading from the branch and relying on the value
they just read still being in the case, now a cache miss will be more likely.
Increasing the BranchState cache to the number of jobs might be a good
idea to amelorate that. But the cache is currently an innefficient list,
so making it large would need changes to the data types.
(Commit 4304f1b6ae
dealt with a follow-on
effect of the bug fixed here.)
This commit is contained in:
parent
4304f1b6ae
commit
770aac97a7
4 changed files with 29 additions and 16 deletions
|
@ -181,11 +181,13 @@ data GetPrivate = GetPrivate Bool
|
|||
getJournalFileStale :: GetPrivate -> RawFilePath -> Annex JournalledContent
|
||||
getJournalFileStale (GetPrivate getprivate) file = do
|
||||
st <- Annex.getState id
|
||||
let repo = Annex.repo st
|
||||
bs <- getState
|
||||
liftIO $
|
||||
if getprivate && privateUUIDsKnown' st
|
||||
then do
|
||||
x <- getfrom (gitAnnexJournalDir (Annex.branchstate st) (Annex.repo st))
|
||||
getfrom (gitAnnexPrivateJournalDir (Annex.branchstate st) (Annex.repo st)) >>= \case
|
||||
x <- getfrom (gitAnnexJournalDir bs repo)
|
||||
getfrom (gitAnnexPrivateJournalDir bs repo) >>= \case
|
||||
Nothing -> return $ case x of
|
||||
Nothing -> NoJournalledContent
|
||||
Just b -> JournalledContent b
|
||||
|
@ -195,7 +197,7 @@ getJournalFileStale (GetPrivate getprivate) file = do
|
|||
-- happens in a merge of two
|
||||
-- git-annex branches.
|
||||
Just x' -> x' <> y
|
||||
else getfrom (gitAnnexJournalDir (Annex.branchstate st) (Annex.repo st)) >>= return . \case
|
||||
else getfrom (gitAnnexJournalDir bs repo) >>= return . \case
|
||||
Nothing -> NoJournalledContent
|
||||
Just b -> JournalledContent b
|
||||
where
|
||||
|
@ -223,8 +225,9 @@ discardIncompleteAppend v
|
|||
- journal is staged as it is run. -}
|
||||
getJournalledFilesStale :: (BranchState -> Git.Repo -> RawFilePath) -> Annex [RawFilePath]
|
||||
getJournalledFilesStale getjournaldir = do
|
||||
st <- Annex.getState id
|
||||
let d = getjournaldir (Annex.branchstate st) (Annex.repo st)
|
||||
bs <- getState
|
||||
repo <- Annex.gitRepo
|
||||
let d = getjournaldir bs repo
|
||||
fs <- liftIO $ catchDefaultIO [] $
|
||||
getDirectoryContents (fromRawFilePath d)
|
||||
return $ filter (`notElem` [".", ".."]) $
|
||||
|
@ -233,8 +236,9 @@ getJournalledFilesStale getjournaldir = do
|
|||
{- Directory handle open on a journal directory. -}
|
||||
withJournalHandle :: (BranchState -> Git.Repo -> RawFilePath) -> (DirectoryHandle -> IO a) -> Annex a
|
||||
withJournalHandle getjournaldir a = do
|
||||
st <- Annex.getState id
|
||||
let d = getjournaldir (Annex.branchstate st) (Annex.repo st)
|
||||
bs <- getState
|
||||
repo <- Annex.gitRepo
|
||||
let d = getjournaldir bs repo
|
||||
bracket (opendir d) (liftIO . closeDirectory) (liftIO . a)
|
||||
where
|
||||
-- avoid overhead of creating the journal directory when it already
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue