From 18d105ac970e42976ec00aca495ad31ea33d818f Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Wed, 24 Apr 2013 16:40:35 -0400 Subject: batch commit every 5 thousand changes, not 10 thousand There's a tradeoff between making less frequent commits, and needing to use memory to store all the changes that are coming in. At 10 thousand, it needs 150 mb of memory. 5 thousand drops that down to 90 mb or so. This also turns out to have significant imact on total run time. I benchmarked 10k changes taking 27 minutes. But two 5k batches took only 21 minutes. --- Assistant/Threads/Committer.hs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'Assistant/Threads') diff --git a/Assistant/Threads/Committer.hs b/Assistant/Threads/Committer.hs index ea3f01fc5..17eefd3f5 100644 --- a/Assistant/Threads/Committer.hs +++ b/Assistant/Threads/Committer.hs @@ -140,7 +140,7 @@ humanImperceptibleDelay = threadDelay $ shouldCommit :: UTCTime -> [Change] -> Bool shouldCommit now changes | len == 0 = False - | len > 10000 = True -- avoid bloating queue too much + | len > 5000 = True -- avoid bloating change pool too much | length recentchanges < 10 = True | otherwise = False -- batch activity where -- cgit v1.2.3