diff options
Diffstat (limited to 'doc/bugs/Out_of_memory_error_when_adding_large_files_to_v6_repository.mdwn')
-rw-r--r-- | doc/bugs/Out_of_memory_error_when_adding_large_files_to_v6_repository.mdwn | 55 |
1 files changed, 55 insertions, 0 deletions
diff --git a/doc/bugs/Out_of_memory_error_when_adding_large_files_to_v6_repository.mdwn b/doc/bugs/Out_of_memory_error_when_adding_large_files_to_v6_repository.mdwn new file mode 100644 index 000000000..ae5c0e6e4 --- /dev/null +++ b/doc/bugs/Out_of_memory_error_when_adding_large_files_to_v6_repository.mdwn @@ -0,0 +1,55 @@ +### Please describe the problem. + +When trying to add a large file (compared to the amount of RAM on the machine) to a v6 repository using `git add large_file.iso`, I get `fatal: Out of memory, realloc failed`. Adding the file with `git annex add large_file.iso` works fine. + +### What steps will reproduce the problem? + +On a machine with 4 GiB of RAM, I'm trying to add a 2.8 GiB file called `large_file.iso` to a freshly created v6 repository: + +``` +$ mkdir annex +$ cd annex +$ git init +$ git annex init --version=6 +$ cp /tmp/large_file.iso . +$ git add large_file.iso +fatal: Out of memory, realloc failed +``` + + +### What version of git-annex are you using? On what operating system? + +- git-annex version: 6.20170101 +- git version 2.11.0 +- Debian GNU/Linux 9.0 (stretch) + +I also tried git-annex 6.20170321, with no success. + +### Please provide any additional information below. + +``` +$ GIT_TRACE=1 git add large_file.iso +06:48:23.048295 git.c:371 trace: built-in: git 'add' 'large_file.iso' +fatal: Out of memory, realloc failed +06:48:23.048725 run-command.c:350 trace: run_command: 'git-annex smudge --clean '\''large_file.iso'\''' +``` + +So the problem seems to be with Git trying to allocate memory the size of the file, rather that git-annex itself: + +``` +$ ltrace git add large_file.iso 2>&1 | grep realloc | tail -n3 +realloc(0, 2902458369) = 0 +realloc(0, 2902458369) = 0 +fputc('\n', 0x7ff2c98ce520Out of memory, realloc failed +``` + +``` +$ git-annex smudge --clean large_file.iso < large_file.iso +/annex/objects/SHA256E-s2902458368--f8ef73594445624eabc1ea4a567ebedb8841fd248347c1aa01538a8de705c5c1.iso +``` + +I have no idea why it needs to do that, though. + +### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) + +I've been using git-annex v5 repositories without any issues, and with smaller files, repository v6 works great! |