more space saving by not locking location log for read

Actions that need to read all the location logs, like "git annex get .",
were still using a lot of memory, and profiling pointed at the location log
reading as the problem. Not locking them for read, and thus avoiding the
strict reading fixes the problem, although I don't quite understand why.
(Oddly, -sstderr profiling did not show the memory as used, though top
showed dozens of MB being used.)

Anyway, it's fine to not lock location logs for read, since the log format
and parser should be safe if a partial read of a file being written happens.
Note that that could easily happen anyway, if doing a git pull, etc,
especially if git needs to union merge in changes from elsewhere. The worst
that will happen is git-annex could get a bad or out of date idea about
locations and refuse to eg, --drop something.
This commit is contained in:
Joey Hess 2010-10-30 23:34:40 -04:00
parent c2651d64bc
commit d5a0c16298

View file

@ -104,8 +104,7 @@ readLog file = do
exists <- doesFileExist file
if exists
then do
s <- withFileLocked file ReadMode $ \h ->
hGetContentsStrict h
s <- readFile file
-- filter out any unparsable lines
return $ filter (\l -> (status l) /= Undefined )
$ map read $ lines s