diff options
author | 2015-06-06 12:10:20 -0400 | |
---|---|---|
committer | 2015-06-06 12:10:20 -0400 | |
commit | 53a1c73c7aa36c7bfb6a82a742ed1ef8ce3ed7cd (patch) | |
tree | 845aae9f87878d4450d2ea35f81f152df23a1409 | |
parent | 4828ceed7367c22cb42ac46c1dfd6d1560217fa8 (diff) | |
parent | 39b095cc25bbe072102bd6ad04cb2a9fe146b696 (diff) |
Merge branch 'master' of ssh://git-annex.branchable.com
-rw-r--r-- | doc/forum/Speed_up_remote_transfers.mdwn | 5 | ||||
-rw-r--r-- | doc/forum/Speed_up_remote_transfers/comment_1_bd97e1c47338322c5232cea7cdbc26fc._comment | 18 |
2 files changed, 23 insertions, 0 deletions
diff --git a/doc/forum/Speed_up_remote_transfers.mdwn b/doc/forum/Speed_up_remote_transfers.mdwn new file mode 100644 index 000000000..04ebdc8f9 --- /dev/null +++ b/doc/forum/Speed_up_remote_transfers.mdwn @@ -0,0 +1,5 @@ +I am currently trying to transfer git-annex managed files to my server. + +I am getting them with `git annex /path`. However, these are very many small files, and git annex seems to take a tiny break in between them, so the process takes considerably longer than uploading a single file of the same size would. + +Do you have any suggestions for improving my transfer speed in this case? diff --git a/doc/forum/Speed_up_remote_transfers/comment_1_bd97e1c47338322c5232cea7cdbc26fc._comment b/doc/forum/Speed_up_remote_transfers/comment_1_bd97e1c47338322c5232cea7cdbc26fc._comment new file mode 100644 index 000000000..f4e9194dd --- /dev/null +++ b/doc/forum/Speed_up_remote_transfers/comment_1_bd97e1c47338322c5232cea7cdbc26fc._comment @@ -0,0 +1,18 @@ +[[!comment format=mdwn + username="CandyAngel" + subject="comment 1" + date="2015-06-06T14:57:35Z" + content=""" +I did this using a couple of tools (off the top of my head): + +git annex find path/ --format '${key}\n' > /tmp/keyslist (outputs a list of keys) +find .git/annex/objects -type f | grep -wFf /tmp/keyslist > /tmp/filelist (outputs a list of files) + +Then I use 'resync -avhP --files-from=/tmp/filelist . othermachine:some/tmp/dir' to transfer the files to the other machine. + +Then I 'git annex import some/tmp/dir' to inject the content, then just delete the additional symlinks (and reset the index). + +This speeds things up a bit. Even more speed can be had using tar/netcat. + +Hope this points you in the right direction. +"""]] |