summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorGravatar Joey Hess <joey@kitenet.net>2010-10-30 23:34:40 -0400
committerGravatar Joey Hess <joey@kitenet.net>2010-10-30 23:34:40 -0400
commitd5a0c16298a625d80288276a92dcfebf1cb9b3de (patch)
tree9f144a5629777e728b52d5891416391757540f68
parentc2651d64bc8a384f8669879c051577ee61da27ed (diff)
more space saving by not locking location log for read
Actions that need to read all the location logs, like "git annex get .", were still using a lot of memory, and profiling pointed at the location log reading as the problem. Not locking them for read, and thus avoiding the strict reading fixes the problem, although I don't quite understand why. (Oddly, -sstderr profiling did not show the memory as used, though top showed dozens of MB being used.) Anyway, it's fine to not lock location logs for read, since the log format and parser should be safe if a partial read of a file being written happens. Note that that could easily happen anyway, if doing a git pull, etc, especially if git needs to union merge in changes from elsewhere. The worst that will happen is git-annex could get a bad or out of date idea about locations and refuse to eg, --drop something.
-rw-r--r--LocationLog.hs3
1 files changed, 1 insertions, 2 deletions
diff --git a/LocationLog.hs b/LocationLog.hs
index f92dee652..20e777c56 100644
--- a/LocationLog.hs
+++ b/LocationLog.hs
@@ -104,8 +104,7 @@ readLog file = do
exists <- doesFileExist file
if exists
then do
- s <- withFileLocked file ReadMode $ \h ->
- hGetContentsStrict h
+ s <- readFile file
-- filter out any unparsable lines
return $ filter (\l -> (status l) /= Undefined )
$ map read $ lines s