diff options
| -rw-r--r-- | doc/forum/Speed_up_remote_transfers.mdwn | 5 | ||||
| -rw-r--r-- | doc/forum/Speed_up_remote_transfers/comment_1_bd97e1c47338322c5232cea7cdbc26fc._comment | 18 |
2 files changed, 23 insertions, 0 deletions
diff --git a/doc/forum/Speed_up_remote_transfers.mdwn b/doc/forum/Speed_up_remote_transfers.mdwn new file mode 100644 index 000000000..04ebdc8f9 --- /dev/null +++ b/doc/forum/Speed_up_remote_transfers.mdwn @@ -0,0 +1,5 @@ +I am currently trying to transfer git-annex managed files to my server. + +I am getting them with `git annex /path`. However, these are very many small files, and git annex seems to take a tiny break in between them, so the process takes considerably longer than uploading a single file of the same size would. + +Do you have any suggestions for improving my transfer speed in this case? diff --git a/doc/forum/Speed_up_remote_transfers/comment_1_bd97e1c47338322c5232cea7cdbc26fc._comment b/doc/forum/Speed_up_remote_transfers/comment_1_bd97e1c47338322c5232cea7cdbc26fc._comment new file mode 100644 index 000000000..f4e9194dd --- /dev/null +++ b/doc/forum/Speed_up_remote_transfers/comment_1_bd97e1c47338322c5232cea7cdbc26fc._comment @@ -0,0 +1,18 @@ +[[!comment format=mdwn + username="CandyAngel" + subject="comment 1" + date="2015-06-06T14:57:35Z" + content=""" +I did this using a couple of tools (off the top of my head): + +git annex find path/ --format '${key}\n' > /tmp/keyslist (outputs a list of keys) +find .git/annex/objects -type f | grep -wFf /tmp/keyslist > /tmp/filelist (outputs a list of files) + +Then I use 'resync -avhP --files-from=/tmp/filelist . othermachine:some/tmp/dir' to transfer the files to the other machine. + +Then I 'git annex import some/tmp/dir' to inject the content, then just delete the additional symlinks (and reset the index). + +This speeds things up a bit. Even more speed can be had using tar/netcat. + +Hope this points you in the right direction. +"""]] |