summaryrefslogtreecommitdiff
path: root/Annex/Link.hs
diff options
context:
space:
mode:
authorGravatar Joey Hess <joeyh@joeyh.name>2016-10-05 15:21:36 -0400
committerGravatar Joey Hess <joeyh@joeyh.name>2016-10-05 15:24:13 -0400
commit6156faa8169db42d76c063033277c9b95b2e5b61 (patch)
tree8a37fdf23f25fc4401cdfebd8c734bf39b34a737 /Annex/Link.hs
parentecfa0fc66a7c907b1177efa978b97bccd6ae7ade (diff)
Avoid using a lot of memory when large objects are present in the git repository
.. and have to be checked to see if they are a pointed to an annexed file. Cases where such memory use could occur included, but were not limited to: - git commit -a of a large unlocked file (in v5 mode) - git-annex adjust when a large file was checked into git directly Generally, any use of catKey was a potential problem. Fix by using git cat-file --batch-check to check size before catting. This adds another git batch process, which is included in the CatFileHandle for simplicity. There could be performance impact, anywhere catKey is used. Particularly likely to affect adjusted branch generation speed, and operations on unlocked files in v6 mode. Hopefully since the --batch-check and --batch read the same data, disk buffering will avoid most overhead. Leaving only the overhead of talking to the process over the pipe and whatever computation --batch-check needs to do. This commit was sponsored by Bruno BEAUFILS on Patreon.
Diffstat (limited to 'Annex/Link.hs')
-rw-r--r--Annex/Link.hs6
1 files changed, 3 insertions, 3 deletions
diff --git a/Annex/Link.hs b/Annex/Link.hs
index af20ae30d..90312a04a 100644
--- a/Annex/Link.hs
+++ b/Annex/Link.hs
@@ -26,7 +26,6 @@ import Annex.HashObject
import Utility.FileMode
import qualified Data.ByteString.Lazy as L
-import Data.Int
type LinkTarget = String
@@ -137,7 +136,8 @@ writePointerFile file k mode = do
- Only looks at the first line, as pointer files can have subsequent
- lines. -}
parseLinkOrPointer :: L.ByteString -> Maybe Key
-parseLinkOrPointer = parseLinkOrPointer' . decodeBS . L.take maxPointerSz
+parseLinkOrPointer = parseLinkOrPointer'
+ . decodeBS . L.take (fromIntegral maxPointerSz)
where
{- Want to avoid buffering really big files in git into
@@ -146,7 +146,7 @@ parseLinkOrPointer = parseLinkOrPointer' . decodeBS . L.take maxPointerSz
- 8192 bytes is plenty for a pointer to a key.
- Pad some more to allow for any pointer files that might have
- lines after the key explaining what the file is used for. -}
-maxPointerSz :: Int64
+maxPointerSz :: Integer
maxPointerSz = 81920
parseLinkOrPointer' :: String -> Maybe Key