diff options
| author |  Joey Hess <joey@kitenet.net> | 2014-11-04 15:22:08 -0400 |
|---|---|---|
| committer | Joey Hess <joey@kitenet.net> | 2014-11-04 15:22:08 -0400 |
| commit | ff7715768f839e9eba0e7e7b135dc6ebea6602a4 (patch) | |
| tree | edfb4c3b4a6cbfdcdc4bdccf34f38a6a6c75aa51 /Remote/S3.hs | |
| parent | 03af2efd4f6519c612b3354e7538c89b65e07691 (diff) | |
fix memory leak
Unfortunately, I don't fully understand why it was leaking using the old
method of a lazy bytestring. I just know that it was leaking, despite
neither hGetUntilMetered nor byteStringPopper seeming to leak by
themselves.
The new method avoids the lazy bytestring, and simply reads chunks from the
handle and streams them out to the http socket.
Diffstat (limited to 'Remote/S3.hs')
| -rw-r--r-- | Remote/S3.hs | 9 |
1 files changed, 5 insertions, 4 deletions
diff --git a/Remote/S3.hs b/Remote/S3.hs index f19c42842..a1f5bf75d 100644 --- a/Remote/S3.hs +++ b/Remote/S3.hs @@ -198,10 +198,11 @@ store r h = fileStorer $ \k f p -> do let sz = if fsz - pos < partsz' then fsz - pos else partsz' - b <- liftIO $ hGetUntilMetered fh (< partsz') meter - let body = RequestBodyStream (fromIntegral sz) (mkPopper b) - S3.UploadPartResponse _ etag <- sendS3Handle h $ - S3.uploadPart (bucket info) object partnum uploadid body + let numchunks = ceiling (fromIntegral sz / defaultChunkSize) + let popper = handlePopper numchunks defaultChunkSize p fh + let req = S3.uploadPart (bucket info) object partnum uploadid $ + RequestBodyStream (fromIntegral sz) popper + S3.UploadPartResponse _ etag <- sendS3Handle h req sendparts (offsetMeterUpdate meter (toBytesProcessed sz)) (etag:etags) (partnum + 1) sendparts p [] 1 |