From a40ec5e03e980b7337bf01eca1661a088ee476c2 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Tue, 14 Feb 2012 11:20:30 -0400 Subject: Fixed a memory leak due to excessive strictness when committing journal files. When hashing the files, the entire list of shas was read strictly. That was entirely unnecessary, since there's a cleanup action run after they're consumed. --- Annex/Branch.hs | 2 +- Git/HashObject.hs | 2 +- debian/changelog | 2 ++ doc/bugs/git_annex_add_memory_leak.mdwn | 5 +++++ 4 files changed, 9 insertions(+), 2 deletions(-) diff --git a/Annex/Branch.hs b/Annex/Branch.hs index b2b1ed3e4..72a98ac16 100644 --- a/Annex/Branch.hs +++ b/Annex/Branch.hs @@ -190,7 +190,7 @@ commit message = whenM journalDirty $ lockJournal $ do {- Commits the staged changes in the index to the branch. - - Ensures that the branch's index file is first updated to the state - - of the brannch at branchref, before running the commit action. This + - of the branch at branchref, before running the commit action. This - is needed because the branch may have had changes pushed to it, that - are not yet reflected in the index. - diff --git a/Git/HashObject.hs b/Git/HashObject.hs index 9c99dae26..ac74f0257 100644 --- a/Git/HashObject.hs +++ b/Git/HashObject.hs @@ -19,7 +19,7 @@ hashFiles paths repo = do fileEncoding toh _ <- forkProcess (feeder toh) hClose toh - shas <- map Ref . lines <$> hGetContentsStrict fromh + shas <- map Ref . lines <$> hGetContents fromh return (shas, ender fromh pid) where git_hash_object = gitCommandLine diff --git a/debian/changelog b/debian/changelog index 8df49d925..23ade624c 100644 --- a/debian/changelog +++ b/debian/changelog @@ -24,6 +24,8 @@ git-annex (3.20120124) UNRELEASED; urgency=low its head), and records the size in the key. * Fixed to use the strict state monad, to avoid leaking all kinds of memory due to lazy state update thunks when adding/fixing many files. + * Fixed a memory leak due to excessive strictness when committing journal + files. * whereis: Prints the urls of files that the web special remote knows about. -- Joey Hess Tue, 24 Jan 2012 16:21:55 -0400 diff --git a/doc/bugs/git_annex_add_memory_leak.mdwn b/doc/bugs/git_annex_add_memory_leak.mdwn index 57ce4c0f9..b6ae60f7b 100644 --- a/doc/bugs/git_annex_add_memory_leak.mdwn +++ b/doc/bugs/git_annex_add_memory_leak.mdwn @@ -27,6 +27,11 @@ A history of the leaks: Apparently fixed in versions afer 3.20120123, but this one is not well understood. +* Committing journal files turned out to have another memory leak. + After adding a lot of files ran out of memory, this left the journal + behind and could affect other git-anne commands. Fixed in versions afer + 3.20120123. + * (Note that `git ls-files --others`, which is used to find files to add, also uses surpsisingly large amounts of memory when you have a lot of files. It buffers -- cgit v1.2.3