From 12243d2279cae65ed163bdec1edb268817514265 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Fri, 25 Nov 2011 11:46:07 -0400 Subject: Flush json output, avoiding a buffering problem that could result in doubled output. The bug was that with --json, output lines were sometimes doubled. For example, git annex init --json would output two lines, despite only running one thing. Adding to the weirdness, this only occurred when the output was redirected to a pipe or a file. Strace showed two processes outputting the same buffered output. The second process was this writer process (only needed to work around bug #624389): _ <- forkProcess $ do hPutStr toh $ unlines paths hClose toh exitSuccess The doubled output occurs when this process exits, and ghc flushes the inherited stdout buffer. Why only when piping? I don't know, but ghc may be behaving differently when stdout is not a terminal. While this is quite possibly a ghc bug, there is a nice fix in git-annex. Explicitly flushing after each chunk of json is output works around the problem, and as a side effect, json is streamed rather than being output all at the end when performing an expensive operaition. However, note that this means all uses of putStr in git-annex must be explicitly flushed. The others were, already. --- Messages.hs | 2 +- debian/changelog | 2 ++ 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/Messages.hs b/Messages.hs index 6ea347ca4..a7f14f485 100644 --- a/Messages.hs +++ b/Messages.hs @@ -129,7 +129,7 @@ handle json normal = Annex.getState Annex.output >>= go where go Annex.NormalOutput = liftIO normal go Annex.QuietOutput = q - go Annex.JSONOutput = liftIO json + go Annex.JSONOutput = liftIO $ flushed $ json q :: Monad m => m () q = return () diff --git a/debian/changelog b/debian/changelog index e043f6c9f..cbf69b741 100644 --- a/debian/changelog +++ b/debian/changelog @@ -2,6 +2,8 @@ git-annex (3.20111123) UNRELEASED; urgency=low * Put a workaround in the directory special remote for strange behavior with VFAT filesystems on Linux (mounted with shortname=mixed) + * Flush json output, avoiding a buffering problem that could result in + doubled output. -- Joey Hess Tue, 22 Nov 2011 17:53:42 -0400 -- cgit v1.2.3