diff options
Diffstat (limited to 'doc/bugs/fails_to_handle_lot_of_files.mdwn')
| -rw-r--r-- | doc/bugs/fails_to_handle_lot_of_files.mdwn | 445 |
1 files changed, 0 insertions, 445 deletions
diff --git a/doc/bugs/fails_to_handle_lot_of_files.mdwn b/doc/bugs/fails_to_handle_lot_of_files.mdwn deleted file mode 100644 index 470a5180f..000000000 --- a/doc/bugs/fails_to_handle_lot_of_files.mdwn +++ /dev/null @@ -1,445 +0,0 @@ - git-annex version: 3.20111011 - local repository version: 3 - default repository version: 3 - supported repository versions: 3 - upgrade supported from repository versions: 0 1 2 - -I just created a new remote on a USB drive and wanted to copy my files over. git-annex wasn't too happy about that ;) -I included a few OK transfers as there was an error before git-annex ran into a wall. As I could easily access that temp file after it aborted, I suspect something went wrong internally before git-annex started to throw those errors. - -Please note the "_n TIMES_" comments. It's how often I got the same error message... - - - - git annex copy . --to USB --fast - - copy redacted.JPG (to USB...) - redacted - 4035668 100% 77.91MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 4036374 bytes received 31 bytes 8072810.00 bytes/sec - total size is 4035668 speedup is 1.00 - ok - copy redacted.NEF (to USB...) - redacted - 18002094 100% 74.19MB/s 0:00:00 (xfer#1, to-check=0/1) - WARNING: redacted failed verification -- update retained (will try again). - redacted - 18002094 100% 19.60MB/s 0:00:00 (xfer#2, to-check=0/1) - rsync: open "copy_target/.git/annex/tmp/redacted_E13" failed: Permission denied (13) - - sent 36008841 bytes received 52 bytes 24005928.67 bytes/sec - total size is 18002094 speedup is 0.50 - rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1070) [sender=3.0.8] - - rsync failed -- run git annex again to resume file transfer - failed - copy redacted.JPG (to USB...) - redacted - 3687111 100% 39.16MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 3687773 bytes received 31 bytes 2458536.00 bytes/sec - total size is 3687111 speedup is 1.00 - ok - copy redacted.NEF (to USB...) - redacted - 17877177 100% 79.15MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 17879573 bytes received 31 bytes 11919736.00 bytes/sec - total size is 17877177 speedup is 1.00 - ok - copy redacted.JPG (to USB...) - redacted - 3694921 100% 40.14MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 3695583 bytes received 31 bytes 2463742.67 bytes/sec - total size is 3694921 speedup is 1.00 - ok - copy redacted.NEF (to USB...) - redacted - 17875448 100% 71.20MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 17877844 bytes received 31 bytes 11918583.33 bytes/sec - total size is 17875448 speedup is 1.00 - ok - copy redacted.JPG (to USB...) - redacted - 3833377 100% 62.49MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 3834055 bytes received 31 bytes 2556057.33 bytes/sec - total size is 3833377 speedup is 1.00 - ok - copy redacted.NEF (to USB...) - redacted - 17938200 100% 65.43MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 17940604 bytes received 31 bytes 11960423.33 bytes/sec - total size is 17938200 speedup is 1.00 - ok - copy redacted.JPG (to USB...) - redacted - 4512557 100% 83.77MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 4513319 bytes received 31 bytes 3008900.00 bytes/sec - total size is 4512557 speedup is 1.00 - ok - copy redacted.NEF (to USB...) - redacted - 18001641 100% 76.16MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 18004053 bytes received 31 bytes 12002722.67 bytes/sec - total size is 18001641 speedup is 1.00 - ok - copy redacted.JPG (to USB...) - redacted - 4394272 100% 50.11MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 4395022 bytes received 31 bytes 8790106.00 bytes/sec - total size is 4394272 speedup is 1.00 - ok - copy redacted.NEF (to USB...) - redacted - 18095781 100% 73.30MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 18098205 bytes received 31 bytes 12065490.67 bytes/sec - total size is 18095781 speedup is 1.00 - ok - copy redacted.JPG (to USB...) - redacted - 4683795 100% 65.23MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 4684577 bytes received 31 bytes 9369216.00 bytes/sec - total size is 4683795 speedup is 1.00 - ok - copy redacted.NEF (to USB...) - redacted - 18172801 100% 74.25MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 18175233 bytes received 31 bytes 36350528.00 bytes/sec - total size is 18172801 speedup is 1.00 - ok - copy redacted.JPG (to USB...) - redacted - 4486231 100% 77.22MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 4486989 bytes received 31 bytes 8974040.00 bytes/sec - total size is 4486231 speedup is 1.00 - ok - copy redacted.NEF (to USB...) - redacted - 17860427 100% 68.56MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 17862823 bytes received 31 bytes 35725708.00 bytes/sec - total size is 17860427 speedup is 1.00 - ok - copy redacted.JPG (to USB...) - redacted - 4499768 100% 36.41MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 4500530 bytes received 31 bytes 9001122.00 bytes/sec - total size is 4499768 speedup is 1.00 - ok - copy redacted.NEF (to USB...) - redacted - 17840132 100% 74.48MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 17842524 bytes received 31 bytes 11895036.67 bytes/sec - total size is 17840132 speedup is 1.00 - ok - copy redacted.JPG (to USB...) - redacted - 4358032 100% 75.00MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 4358774 bytes received 31 bytes 8717610.00 bytes/sec - total size is 4358032 speedup is 1.00 - ok - copy redacted.NEF (to USB...) - redacted - 18084753 100% 61.48MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 18087173 bytes received 31 bytes 12058136.00 bytes/sec - total size is 18084753 speedup is 1.00 - ok - copy redacted.JPG (to USB...) - redacted - 4270213 100% 68.49MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 4270947 bytes received 31 bytes 2847318.67 bytes/sec - total size is 4270213 speedup is 1.00 - ok - copy redacted.NEF (to USB...) - redacted - 17661246 100% 68.34MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 17663614 bytes received 31 bytes 11775763.33 bytes/sec - total size is 17661246 speedup is 1.00 - ok - copy redacted.JPG (to USB...) - redacted - 4538305 100% 63.19MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 4539071 bytes received 31 bytes 9078204.00 bytes/sec - total size is 4538305 speedup is 1.00 - ok - copy redacted.NEF (to USB...) - redacted - 18672466 100% 68.90MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 18674958 bytes received 31 bytes 12449992.67 bytes/sec - total size is 18672466 speedup is 1.00 - ok - copy redacted.JPG (to USB...) - redacted - 4453445 100% 73.96MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 4454199 bytes received 31 bytes 8908460.00 bytes/sec - total size is 4453445 speedup is 1.00 - ok - copy redacted.NEF (to USB...) - redacted - 18495494 100% 59.28MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 18497966 bytes received 31 bytes 12331998.00 bytes/sec - total size is 18495494 speedup is 1.00 - ok - copy redacted.JPG (to USB...) - redacted - 4255858 100% 70.66MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 4256588 bytes received 31 bytes 1702647.60 bytes/sec - total size is 4255858 speedup is 1.00 - ok - copy redacted.NEF (to USB...) - redacted - 18376531 100% 69.15MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 18378987 bytes received 31 bytes 36758036.00 bytes/sec - total size is 18376531 speedup is 1.00 - ok - copy redacted.JPG (to USB...) - redacted - 4013365 100% 48.67MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 4014067 bytes received 31 bytes 8028196.00 bytes/sec - total size is 4013365 speedup is 1.00 - ok - copy redacted.NEF (to USB...) - redacted - 17606341 100% 51.73MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 17608705 bytes received 31 bytes 11739157.33 bytes/sec - total size is 17606341 speedup is 1.00 - ok - copy redacted.JPG (to USB...) - redacted - 4179869 100% 74.62MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 4180591 bytes received 31 bytes 8361244.00 bytes/sec - total size is 4179869 speedup is 1.00 - ok - copy redacted.NEF (to USB...) - redacted - 18382569 100% 67.05MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 18385025 bytes received 31 bytes 12256704.00 bytes/sec - total size is 18382569 speedup is 1.00 - ok - copy redacted.JPG (to USB...) - redacted - 4318363 100% 44.91MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 4319101 bytes received 31 bytes 8638264.00 bytes/sec - total size is 4318363 speedup is 1.00 - ok - copy redacted.NEF (to USB...) - redacted - 17715958 100% 72.69MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 17718334 bytes received 31 bytes 11812243.33 bytes/sec - total size is 17715958 speedup is 1.00 - ok - copy redacted.JPG (to USB...) - redacted - 4241893 100% 65.81MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 4242623 bytes received 31 bytes 8485308.00 bytes/sec - total size is 4241893 speedup is 1.00 - ok - copy redacted.NEF (to USB...) - redacted - 17717287 100% 71.77MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 17719663 bytes received 31 bytes 11813129.33 bytes/sec - total size is 17717287 speedup is 1.00 - ok - copy redacted.JPG (to USB...) - redacted - 4488380 100% 49.99MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 4489138 bytes received 31 bytes 2992779.33 bytes/sec - total size is 4488380 speedup is 1.00 - ok - copy redacted.NEF (to USB...) - redacted - 17770208 100% 38.80MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 17772592 bytes received 31 bytes 11848415.33 bytes/sec - total size is 17770208 speedup is 1.00 - ok - copy redacted.JPG (to USB...) - redacted - 4603958 100% 76.48MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 4604732 bytes received 31 bytes 9209526.00 bytes/sec - total size is 4603958 speedup is 1.00 - ok - copy redacted.NEF (to USB...) - redacted - 18744380 100% 74.66MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 18746884 bytes received 31 bytes 12497943.33 bytes/sec - total size is 18744380 speedup is 1.00 - ok - copy redacted.JPG (to USB...) - redacted - 4592098 100% 79.06MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 4592872 bytes received 31 bytes 3061935.33 bytes/sec - total size is 4592098 speedup is 1.00 - ok - copy redacted.NEF (to USB...) - redacted - 18746205 100% 43.00MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 18748709 bytes received 31 bytes 12499160.00 bytes/sec - total size is 18746205 speedup is 1.00 - ok - copy redacted.JPG (to USB...) - redacted - 7493353 100% 80.85MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 7494479 bytes received 31 bytes 14989020.00 bytes/sec - total size is 7493353 speedup is 1.00 - ok - copy redacted.NEF (to USB...) - redacted - 19496768 100% 81.77MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 19499360 bytes received 31 bytes 12999594.00 bytes/sec - total size is 19496768 speedup is 1.00 - ok - copy redacted.JPG (to USB...) - redacted - 5462482 100% 82.19MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 5463360 bytes received 31 bytes 10926782.00 bytes/sec - total size is 5462482 speedup is 1.00 - ok - copy redacted.NEF (to USB...) - redacted - 19669815 100% 80.37MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 19672431 bytes received 31 bytes 13114974.67 bytes/sec - total size is 19669815 speedup is 1.00 - ok - copy redacted.JPG (to USB...) - redacted - 5449487 100% 57.40MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 5450365 bytes received 31 bytes 3633597.33 bytes/sec - total size is 5449487 speedup is 1.00 - ok - copy redacted.NEF (to USB...) - redacted - 19633259 100% 74.18MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 19635871 bytes received 31 bytes 13090601.33 bytes/sec - total size is 19633259 speedup is 1.00 - ok - copy redacted.JPG (to USB...) - redacted - 5392184 100% 62.33MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 5393054 bytes received 31 bytes 3595390.00 bytes/sec - total size is 5392184 speedup is 1.00 - ok - copy redacted.NEF (to USB...) - redacted - 18912104 100% 65.00MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 18914628 bytes received 31 bytes 12609772.67 bytes/sec - total size is 18912104 speedup is 1.00 - ok - copy redacted.JPG (to USB...) - redacted - 4869300 100% 80.92MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 4870106 bytes received 31 bytes 9740274.00 bytes/sec - total size is 4869300 speedup is 1.00 - ok - copy redacted.NEF (to USB...) - redacted - 20178932 100% 68.13MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 20181608 bytes received 31 bytes 13454426.00 bytes/sec - total size is 20178932 speedup is 1.00 - ok - copy redacted.JPG (to USB...) - redacted - 4995425 100% 86.05MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 4996247 bytes received 31 bytes 9992556.00 bytes/sec - total size is 4995425 speedup is 1.00 - ok - copy redacted.NEF (to USB...) - redacted - 19970679 100% 76.36MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 19973331 bytes received 31 bytes 13315574.67 bytes/sec - total size is 19970679 speedup is 1.00 - ok - copy redacted.JPG (to USB...) - redacted - 7905795 100% 66.45MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 7906973 bytes received 31 bytes 15814008.00 bytes/sec - total size is 7905795 speedup is 1.00 - ok - copy redacted.NEF (to USB...) - redacted - 21234069 100% 78.07MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 21236877 bytes received 31 bytes 8494763.20 bytes/sec - total size is 21234069 speedup is 1.00 - ok - copy redacted.JPG (to USB...) - redacted - 7963979 100% 62.51MB/s 0:00:00 (xfer#1, to-check=0/1) - - sent 7965165 bytes received 31 bytes 5310130.67 bytes/sec - total size is 7963979 speedup is 1.00 - git ["--git-dir=copy_target/.git","--work-tree=copy_target","update-index","-z","--index-info"]: Error in fork: forkProcess: resource exhausted (Resource temporarily unavailable) - - git-annex: user error (git ["--git-dir=copy_target/.git","--work-tree=copy_target","update-index","-z","--index-info"]: Error in fork: forkProcess: resource exhausted (Resource temporarily unavailable)) - failed - _506 TIMES_ (user error (Error in fork: forkProcess: resource exhausted (Resource temporarily unavailable))) failed - _11 TIMES_ copy foo (createPipe: resource exhausted (Too many open files)) failed - _2 TIMES_ (user error (Error in fork: forkProcess: resource exhausted (Resource temporarily unavailable))) failed - _8574 TIMES_: copy foo (createPipe: resource exhausted (Too many open files)) failed - git-annex: createPipe: resource exhausted (Too many open files) - failed - git-annex: 9101 failed - - % ls copy_target/.git/annex/tmp/redacted_E13 copy_target/.git/annex/tmp/SHA512E-redacted_E13 # works - % find source -type l | wc -l - 13554 - % find copy_target -type l | wc -l - 13554 - % find copy_target/.git/annex/objects -type f | wc -l - 4455 - % find source -type f | wc -l - 13554 - -> Fixed unreaped process leak. -> (This has nothing to do with NTFS). Ran test with 10k files -> [[done]] --[[Joey]] |