diff options
| author |  Joey Hess <joey@kitenet.net> | 2014-08-09 15:58:01 -0400 |
|---|---|---|
| committer | Joey Hess <joey@kitenet.net> | 2014-08-09 15:58:01 -0400 |
| commit | 1ebf529787d1766a5b6a25c6c2b58f126053816f (patch) | |
| tree | f540772502c35ede0a40b5ca978d18c406054f2e /doc/bugs/S3_memory_leaks.mdwn | |
| parent | 620d1e4273310373f38866259f9f6b3bc6995a5e (diff) | |
S3: finish converting to aws library
Implemented the Retriever.
Unfortunately, it is a fileRetriever and not a byteRetriever.
It should be possible to convert this to a byteRetiever, but I got stuck:
The conduit sink needs to process individual chunks, but a byteRetriever
needs to pass a single L.ByteString to its callback for processing. I
looked into using unsafeInerlaveIO to build up the bytestring lazily,
but the sink is already operating under conduit's inversion of control,
and does not run directly in IO anyway.
On the plus side, no more memory leak..
Diffstat (limited to 'doc/bugs/S3_memory_leaks.mdwn')
| -rw-r--r-- | doc/bugs/S3_memory_leaks.mdwn | 2 |
1 files changed, 2 insertions, 0 deletions
diff --git a/doc/bugs/S3_memory_leaks.mdwn b/doc/bugs/S3_memory_leaks.mdwn index 88dd6eaa6..7dc1e5757 100644 --- a/doc/bugs/S3_memory_leaks.mdwn +++ b/doc/bugs/S3_memory_leaks.mdwn @@ -7,6 +7,8 @@ Sending a file to S3 causes a slow memory increase toward the file size. Copying the file back from S3 causes a slow memory increase toward the file size. +> [[fixed|done]] too! --[[Joey]] + The author of hS3 is aware of the problem, and working on it. I think I have identified the root cause of the buffering; it's done by hS3 so it can resend the data if S3 sends it a 307 redirect. --[[Joey]] |