more space saving by not locking location log for read
Actions that need to read all the location logs, like "git annex get .", were still using a lot of memory, and profiling pointed at the location log reading as the problem. Not locking them for read, and thus avoiding the strict reading fixes the problem, although I don't quite understand why. (Oddly, -sstderr profiling did not show the memory as used, though top showed dozens of MB being used.) Anyway, it's fine to not lock location logs for read, since the log format and parser should be safe if a partial read of a file being written happens. Note that that could easily happen anyway, if doing a git pull, etc, especially if git needs to union merge in changes from elsewhere. The worst that will happen is git-annex could get a bad or out of date idea about locations and refuse to eg, --drop something.
This commit is contained in:
parent
c2651d64bc
commit
d5a0c16298
1 changed files with 1 additions and 2 deletions
|
@ -104,8 +104,7 @@ readLog file = do
|
|||
exists <- doesFileExist file
|
||||
if exists
|
||||
then do
|
||||
s <- withFileLocked file ReadMode $ \h ->
|
||||
hGetContentsStrict h
|
||||
s <- readFile file
|
||||
-- filter out any unparsable lines
|
||||
return $ filter (\l -> (status l) /= Undefined )
|
||||
$ map read $ lines s
|
||||
|
|
Loading…
Reference in a new issue