fix lack of laziness streaming large diffs

A commit last year that made a partial function use Maybe unfortunately
caused the whole input to need to be consumed, breaking streaming. So,
revert it.

This commit was sponsored by Nick Daly on Patreon.

1 files changed, 19 insertions, 0 deletions

diff --git a/doc/todo/more_efficient_memory_usage_with_git-annex_unused/comment_2_1eff4497bf0a3e87dc47a1226c5d4af8._comment b/doc/todo/more_efficient_memory_usage_with_git-annex_unused/comment_2_1eff4497bf0a3e87dc47a1226c5d4af8._comment
new file mode 100644
index 000000000..df285478c
--- /dev/null
+++ b/doc/todo/more_efficient_memory_usage_with_git-annex_unused/comment_2_1eff4497bf0a3e87dc47a1226c5d4af8._comment
@@ -0,0 +1,19 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 2"""
+ date="2017-01-31T20:24:04Z"
+ content="""
+The heap profile has multiple spikes (so not an accumulating memory leak).
+The diff parsing code is indeed what's using so much memory. Looks like
+data is failing to stream through that code and instead the whole
+diff output gets buffered.
+
+Aha.. Git.DiffTree.parseDiffRaw used to return a list, but changed
+in [[!commit 8d124beba8]]
+to a Maybe list in order to avoid being a partial function. But 
+that change destroyed laziness, since the whole input has to be parsed
+in order to determine if Nothing should be returned.
+
+However, fixing that only eliminated part of the spike. There's something
+else keeping data from streaming.
+"""]]