diff options
author | Joey Hess <joey@kitenet.net> | 2011-04-19 16:31:35 -0400 |
---|---|---|
committer | Joey Hess <joey@kitenet.net> | 2011-04-19 16:31:35 -0400 |
commit | 4837176897ae5ade15b23de4999c370d3ac2ef3e (patch) | |
tree | 5f4f9b62d855a69e4ba2ca110b8fafa95143b1e9 | |
parent | 030c7a056b0795adf037464d608da5170638f4ac (diff) |
update on memory leak
Finished applying to S3 the change that fixed the memory leak in bup, but
it didn't seem to help S3.. with encryption it still grows to 2x file size.
-rw-r--r-- | Remote/S3real.hs | 13 | ||||
-rw-r--r-- | doc/bugs/S3_memory_leaks.mdwn | 1 |
2 files changed, 6 insertions, 8 deletions
diff --git a/Remote/S3real.hs b/Remote/S3real.hs index f40deaf17..5095b4039 100644 --- a/Remote/S3real.hs +++ b/Remote/S3real.hs @@ -99,22 +99,19 @@ s3Setup u c = do store :: Remote Annex -> Key -> Annex Bool store r k = s3Action r False $ \(conn, bucket) -> do - content <- lazyKeyContent k + g <- Annex.gitRepo + content <- liftIO $ L.readFile $ gitAnnexLocation g k res <- liftIO $ storeHelper (conn, bucket) r k content s3Bool res storeEncrypted :: Remote Annex -> (Cipher, Key) -> Key -> Annex Bool storeEncrypted r (cipher, enck) k = s3Action r False $ \(conn, bucket) -> do - content <- lazyKeyContent k - res <- liftIO $ withEncryptedContent cipher (return content) $ \s -> do + g <- Annex.gitRepo + let f = gitAnnexLocation g k + res <- liftIO $ withEncryptedContent cipher (L.readFile f) $ \s -> do storeHelper (conn, bucket) r enck s s3Bool res -lazyKeyContent :: Key -> Annex L.ByteString -lazyKeyContent k = do - g <- Annex.gitRepo - liftIO $ L.readFile $ gitAnnexLocation g k - storeHelper :: (AWSConnection, String) -> Remote Annex -> Key -> L.ByteString -> IO (AWSResult ()) storeHelper (conn, bucket) r k content = do let object = setStorageClass storageclass $ bucketKey bucket k content diff --git a/doc/bugs/S3_memory_leaks.mdwn b/doc/bugs/S3_memory_leaks.mdwn index d7cae2933..f0522304c 100644 --- a/doc/bugs/S3_memory_leaks.mdwn +++ b/doc/bugs/S3_memory_leaks.mdwn @@ -2,6 +2,7 @@ S3 has two memory leaks. One only occurs with encryption. It was was fixed for bup, but not yet for S3, in 5985acdfad8a6791f0b2fc54a1e116cee9c12479. +(The fix I used for bup doesn't seem to work with S3.) The other occurs independant of encryption use. Copying a 100 mb file to S3 causes an immediate sharp memory spike to 119 mb. |