diff options
Diffstat (limited to 'doc/bugs/checksum_loads_whole_file_into_memory.mdwn')
| -rw-r--r-- | doc/bugs/checksum_loads_whole_file_into_memory.mdwn | 24 |
1 files changed, 24 insertions, 0 deletions
diff --git a/doc/bugs/checksum_loads_whole_file_into_memory.mdwn b/doc/bugs/checksum_loads_whole_file_into_memory.mdwn new file mode 100644 index 000000000..ece328105 --- /dev/null +++ b/doc/bugs/checksum_loads_whole_file_into_memory.mdwn @@ -0,0 +1,24 @@ +Using eg, fsck with the MD5 backend loads whole files into memory. + +May only happen for very large files (40 gb) or in other specific +circumstances, including ghc version used for buildd etc. + +Observed with 6.20160217-g95bbdb8, linux standalone amd64. + +Not observed with 5.20151218-g5008846. + +Commit 7482853ddddc21f2696dcfbc82d737f03032134a may be relevant, +but I don't understand how yet. A small test program using the same +code doesn't exhibit the problem, even when built in the identical build +environment as the 6.20160217-g95bbdb8 that has the problem. + +> Update: Reverted 7482853ddddc21f2696dcfbc82d737f03032134a and indeed the +> problem got fixed. But, reverting that commit breaks the test suite on +> windows and has a FD leak, so is not desirable. This needs more +> investigation. --[[Joey]] + +>> I see it now, the checksum is a String and it was only forced to WHNF, +>> so the hashing didn't fully complete and the file got buffered. +>> Probably only occurred when fscking, and not when adding a file, +>> due to differing use patterns of the checksum. +>> [[fixed|done]] --[[Joey]] |