diff options
author | Joey Hess <joeyh@joeyh.name> | 2017-01-31 17:43:11 -0400 |
---|---|---|
committer | Joey Hess <joeyh@joeyh.name> | 2017-01-31 17:43:11 -0400 |
commit | 164466c987a7607a5f598b36e5b3111a68bd101f (patch) | |
tree | 592693e8cd910d8dc7438fa3137238cb00391c9a /doc/todo/more_efficient_memory_usage_with_git-annex_unused/comment_2_1eff4497bf0a3e87dc47a1226c5d4af8._comment | |
parent | 893667e2565eb707ff004e2ea6f5411be3eea936 (diff) |
fix lack of laziness streaming large diffs
A commit last year that made a partial function use Maybe unfortunately
caused the whole input to need to be consumed, breaking streaming. So,
revert it.
This commit was sponsored by Nick Daly on Patreon.
Diffstat (limited to 'doc/todo/more_efficient_memory_usage_with_git-annex_unused/comment_2_1eff4497bf0a3e87dc47a1226c5d4af8._comment')
-rw-r--r-- | doc/todo/more_efficient_memory_usage_with_git-annex_unused/comment_2_1eff4497bf0a3e87dc47a1226c5d4af8._comment | 19 |
1 files changed, 19 insertions, 0 deletions
diff --git a/doc/todo/more_efficient_memory_usage_with_git-annex_unused/comment_2_1eff4497bf0a3e87dc47a1226c5d4af8._comment b/doc/todo/more_efficient_memory_usage_with_git-annex_unused/comment_2_1eff4497bf0a3e87dc47a1226c5d4af8._comment new file mode 100644 index 000000000..df285478c --- /dev/null +++ b/doc/todo/more_efficient_memory_usage_with_git-annex_unused/comment_2_1eff4497bf0a3e87dc47a1226c5d4af8._comment @@ -0,0 +1,19 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 2""" + date="2017-01-31T20:24:04Z" + content=""" +The heap profile has multiple spikes (so not an accumulating memory leak). +The diff parsing code is indeed what's using so much memory. Looks like +data is failing to stream through that code and instead the whole +diff output gets buffered. + +Aha.. Git.DiffTree.parseDiffRaw used to return a list, but changed +in [[!commit 8d124beba8]] +to a Maybe list in order to avoid being a partial function. But +that change destroyed laziness, since the whole input has to be parsed +in order to determine if Nothing should be returned. + +However, fixing that only eliminated part of the spike. There's something +else keeping data from streaming. +"""]] |