diff options
| author |  Joey Hess <joeyh@joeyh.name> | 2017-01-31 19:42:00 -0400 |
|---|---|---|
| committer | Joey Hess <joeyh@joeyh.name> | 2017-01-31 19:42:00 -0400 |
| commit | 593d91f74c2bd7b9fa36e501afb5ed5d292c464e (patch) | |
| tree | 38a828a14c461ecde0c0a5615be3accb98eb33ef | |
| parent | 7b1f48cbd5d78877ae3a6b6a5415a9bed6df79b2 (diff) | |
unused: Improved memory use significantly when there are a lot of differences between branches.
Argh, didn't need an accumulator here!
I think I use accumulators a lot more than I need to when recusively
processing lists..
This commit was sponsored by Jeff Goeke-Smith on Patreon.
4 files changed, 19 insertions, 4 deletions
@@ -11,6 +11,8 @@ git-annex (6.20170102) UNRELEASED; urgency=medium * Some optimisations to string splitting code. * unused: When large files are checked right into git, avoid buffering their contents in memory. + * unused: Improved memory use significantly when there are a lot + of differences between branches. -- Joey Hess <id@joeyh.name> Fri, 06 Jan 2017 15:22:06 -0400 diff --git a/Git/DiffTree.hs b/Git/DiffTree.hs index 309575aaf..dae99db41 100644 --- a/Git/DiffTree.hs +++ b/Git/DiffTree.hs @@ -101,11 +101,11 @@ getdiff command params repo = do {- Parses --raw output used by diff-tree and git-log. -} parseDiffRaw :: [String] -> [DiffTreeItem] -parseDiffRaw l = go l [] +parseDiffRaw l = go l where - go [] c = c - go (info:f:rest) c = go rest (mk info f : c) - go (s:[]) _ = error $ "diff-tree parse error near \"" ++ s ++ "\"" + go [] = [] + go (info:f:rest) = mk info f : go rest + go (s:[]) = error $ "diff-tree parse error near \"" ++ s ++ "\"" mk info f = DiffTreeItem { srcmode = readmode srcm diff --git a/doc/todo/more_efficient_memory_usage_with_git-annex_unused.mdwn b/doc/todo/more_efficient_memory_usage_with_git-annex_unused.mdwn index 6ed716ae9..908ddc208 100644 --- a/doc/todo/more_efficient_memory_usage_with_git-annex_unused.mdwn +++ b/doc/todo/more_efficient_memory_usage_with_git-annex_unused.mdwn @@ -1,3 +1,6 @@ While running *git-annex unused* on an annex with tens of thousands of items, *git-annex*'s memory usage ballooned to over 3 gigs and my PC froze. I cannot run *git-annex unused* on this annex because of this issue. If it's possible, more efficient memory management would prevent this from happening. + +> [[done]] -- assuming the memory leak I saw was the same one you saw... +> --[[Joey]] diff --git a/doc/todo/more_efficient_memory_usage_with_git-annex_unused/comment_3_3c712e871ea3cd12916497f2d8152004._comment b/doc/todo/more_efficient_memory_usage_with_git-annex_unused/comment_3_3c712e871ea3cd12916497f2d8152004._comment new file mode 100644 index 000000000..2a5130a95 --- /dev/null +++ b/doc/todo/more_efficient_memory_usage_with_git-annex_unused/comment_3_3c712e871ea3cd12916497f2d8152004._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 3""" + date="2017-01-31T23:31:03Z" + content=""" +Fixed the rest of the streaming problem. + +(Also found/fixed an unrelated memory blow up in git annex unused that +only happened when a large file got checked right into git.) +"""]] |