Merge branch 'master' of ssh://git-annex.branchable.com

6 files changed, 188 insertions, 1 deletions

diff --git a/doc/bugs/Allow_automatic_retry_git_annex_get.mdwn b/doc/bugs/Allow_automatic_retry_git_annex_get.mdwn
new file mode 100644
index 000000000..da28eae7c
--- /dev/null
+++ b/doc/bugs/Allow_automatic_retry_git_annex_get.mdwn
@@ -0,0 +1,121 @@
+### Please describe the problem.
+Often, when I `git annex get` a large file over ssh, it will hang in the middle, and I'll have to C-c to cancel it, and try again. This strategy works, but I often have to keep canceling and retrying 5-10 times for a large file. This is unfortunate. I think maybe rsync is getting overwhelmed. 
+
+Request: Could you please add a flag --autoretry so that if `get` hangs, cancel and try again. This generally works, because of the rsync --partial flag, so it eventually gets to the end of the file. Thanks!
+
+### What steps will reproduce the problem?
+
+
+### What version of git-annex are you using? On what operating system?
+git-annex version: 6.20160511-1~ubuntu14.04.1~ppa1
+Ubuntu 14.04
+
+### Please provide any additional information below.
+
+[[!format sh """
+# If you can, paste a complete transcript of the problem occurring here.
+# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
+
+$ git annex get --debug
+[2016-08-26 15:59:19.476393] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","ls-files","--cached","-z","--"]
+get clerk/H101-2.json [2016-08-26 15:59:19.483388] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","git-annex"]
+[2016-08-26 15:59:19.485433] process done ExitSuccess
+[2016-08-26 15:59:19.485676] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","--hash","refs/heads/git-annex"]
+[2016-08-26 15:59:19.48755] process done ExitSuccess
+[2016-08-26 15:59:19.487727] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","log","refs/heads/git-annex..4455cba13d289aad3c25ddc097b5ca02e7fc0ffb","--pretty=%H","-n1"]
+[2016-08-26 15:59:19.489793] process done ExitSuccess
+[2016-08-26 15:59:19.4899] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","log","refs/heads/git-annex..7af97d408ed894922cd08bd2091dab5bb1cf1df9","--pretty=%H","-n1"]
+[2016-08-26 15:59:19.49133] process done ExitSuccess
+[2016-08-26 15:59:19.491939] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch"]
+(from origin...) 
+[2016-08-26 15:59:19.495688] read: rsync ["--progress","--inplace","--perms","-e","'ssh' '-S' '.git/annex/ssh/voteviewdev' '-o' 'ControlMaster=auto' '-o' 'ControlPersist=yes' '-T' 'voteviewdev' 'git-annex-shell ''sendkey'' ''/srv/annex'' ''--debug'' ''SHA256E-s49892641--7390d24449d67e1d60c31306bfc8b189f7c4ed223cc44bbd0096222b8ede3b92.json'' --uuid ff59187e-f48c-47d0-b75d-320a6a4ce115 ''--'' ''remoteuuid=da1361a1-3f1a-47b8-8165-daf8c30d543d'' ''unlocked='' ''direct='' ''associatedfile=clerk/H101-2.json'' ''--'''","--","dummy:",".git/annex/tmp/SHA256E-s49892641--7390d24449d67e1d60c31306bfc8b189f7c4ed223cc44bbd0096222b8ede3b92.json"]
+[2016-08-26 15:58:37.018182] transfer start
+[2016-08-26 15:58:37.023135] call: rsync ["--server","-t","--inplace","-e.Lsf",".","--sender","../../srv/annex/.git/annex/objects/2G/fP/SHA256E-s49892641--7390d24449d67e1d60c31306bfc8b189f7c4ed223cc44bbd0096222b8ede3b92.json/SHA256E-s49892641--7390d24449d67e1d60c31306bfc8b189f7c4ed223cc44bbd0096222b8ede3b92.json"]
+SHA256E-s49892641--7390d24449d67e1d60c31306bfc8b189f7c4ed223cc44bbd0096222b8ede3b92.json
+     49,101,968  98%   46.83MB/s    0:00:00  [2016-08-26 15:59:22.809273] feed: ssh ["-S",".git/annex/ssh/voteviewdev","-o","ControlMaster=auto","-o","ControlPersist=yes","-T","voteviewdev","git-annex-shell 'transferinfo' '/srv/annex' '--debug' 'SHA256E-s49892641--7390d24449d67e1d60c31306bfc8b189f7c4ed223cc44bbd0096222b8ede3b92.json' --uuid ff59187e-f48c-47d0-b75d-320a6a4ce115 '--' 'remoteuuid=da1361a1-3f1a-47b8-8165-daf8c30d543d' 'associatedfile=clerk/H101-2.json' '--'"]
+     49,892,641 100%   39.52MB/s    0:00:01 (xfr#1, to-chk=0/1)
+[2016-08-26 15:58:39.034666] process done ExitSuccess
+[2016-08-26 15:58:39.038811] transfer done
+[2016-08-26 15:59:23.09308] process done ExitSuccess
+(checksum...) [2016-08-26 15:59:23.093582] read: sha256sum [".git/annex/tmp/SHA256E-s49892641--7390d24449d67e1d60c31306bfc8b189f7c4ed223cc44bbd0096222b8ede3b92.json"]
+[2016-08-26 15:59:23.331872] process done ExitSuccess
+ok
+get clerk/H102-1.json (from origin...) 
+[2016-08-26 15:59:23.337838] read: rsync ["--progress","--inplace","--perms","-e","'ssh' '-S' '.git/annex/ssh/voteviewdev' '-o' 'ControlMaster=auto' '-o' 'ControlPersist=yes' '-T' 'voteviewdev' 'git-annex-shell ''sendkey'' ''/srv/annex'' ''--debug'' ''SHA256E-s41311329--69c3b054a3fe9676133605730d85b7fcef8696f6782d402a524e41b836253891.json'' --uuid ff59187e-f48c-47d0-b75d-320a6a4ce115 ''--'' ''remoteuuid=da1361a1-3f1a-47b8-8165-daf8c30d543d'' ''unlocked='' ''direct='' ''associatedfile=clerk/H102-1.json'' ''--'''","--","dummy:",".git/annex/tmp/SHA256E-s41311329--69c3b054a3fe9676133605730d85b7fcef8696f6782d402a524e41b836253891.json"]
+[2016-08-26 15:59:23.351903] process done ExitSuccess
+[2016-08-26 15:58:39.394898] transfer start
+[2016-08-26 15:58:39.400323] call: rsync ["--server","-t","--inplace","-e.Lsf",".","--sender","../../srv/annex/.git/annex/objects/6Z/QP/SHA256E-s41311329--69c3b054a3fe9676133605730d85b7fcef8696f6782d402a524e41b836253891.json/SHA256E-s41311329--69c3b054a3fe9676133605730d85b7fcef8696f6782d402a524e41b836253891.json"]
+SHA256E-s41311329--69c3b054a3fe9676133605730d85b7fcef8696f6782d402a524e41b836253891.json
+      1,998,848   4%    1.91MB/s    0:00:20  [2016-08-26 15:59:24.608637] feed: ssh ["-S",".git/annex/ssh/voteviewdev","-o","ControlMaster=auto","-o","ControlPersist=yes","-T","voteviewdev","git-annex-shell 'transferinfo' '/srv/annex' '--debug' 'SHA256E-s41311329--69c3b054a3fe9676133605730d85b7fcef8696f6782d402a524e41b836253891.json' --uuid ff59187e-f48c-47d0-b75d-320a6a4ce115 '--' 'remoteuuid=da1361a1-3f1a-47b8-8165-daf8c30d543d' 'associatedfile=clerk/H102-1.json' '--'"]
+     10,616,832  25%    1.54MB/s    0:00:19  
+
+
+
+# End of transcript or log.
+"""]]
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+
+
+
+
+
+### Please describe the problem.
+Often, when I `git annex get` a large file over ssh, it will hang in the middle, and I'll have to C-c to cancel it, and try again. This strategy works, but I often have to keep canceling and retrying 5-10 times for a large file. This is unfortunate. I think maybe rsync is getting overwhelmed. 
+
+Request: Could you please add a flag --autoretry so that if `get` hangs, cancel and try again. This generally works, because of the rsync --partial flag, so it eventually gets to the end of the file. Thanks!
+
+### What steps will reproduce the problem?
+
+
+### What version of git-annex are you using? On what operating system?
+git-annex version: 6.20160511-1~ubuntu14.04.1~ppa1
+Ubuntu 14.04
+
+### Please provide any additional information below.
+
+[[!format sh """
+# If you can, paste a complete transcript of the problem occurring here.
+# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
+
+$ git annex get --debug
+[2016-08-26 15:59:19.476393] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","ls-files","--cached","-z","--"]
+get clerk/H101-2.json [2016-08-26 15:59:19.483388] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","git-annex"]
+[2016-08-26 15:59:19.485433] process done ExitSuccess
+[2016-08-26 15:59:19.485676] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","--hash","refs/heads/git-annex"]
+[2016-08-26 15:59:19.48755] process done ExitSuccess
+[2016-08-26 15:59:19.487727] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","log","refs/heads/git-annex..4455cba13d289aad3c25ddc097b5ca02e7fc0ffb","--pretty=%H","-n1"]
+[2016-08-26 15:59:19.489793] process done ExitSuccess
+[2016-08-26 15:59:19.4899] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","log","refs/heads/git-annex..7af97d408ed894922cd08bd2091dab5bb1cf1df9","--pretty=%H","-n1"]
+[2016-08-26 15:59:19.49133] process done ExitSuccess
+[2016-08-26 15:59:19.491939] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch"]
+(from origin...) 
+[2016-08-26 15:59:19.495688] read: rsync ["--progress","--inplace","--perms","-e","'ssh' '-S' '.git/annex/ssh/voteviewdev' '-o' 'ControlMaster=auto' '-o' 'ControlPersist=yes' '-T' 'voteviewdev' 'git-annex-shell ''sendkey'' ''/srv/annex'' ''--debug'' ''SHA256E-s49892641--7390d24449d67e1d60c31306bfc8b189f7c4ed223cc44bbd0096222b8ede3b92.json'' --uuid ff59187e-f48c-47d0-b75d-320a6a4ce115 ''--'' ''remoteuuid=da1361a1-3f1a-47b8-8165-daf8c30d543d'' ''unlocked='' ''direct='' ''associatedfile=clerk/H101-2.json'' ''--'''","--","dummy:",".git/annex/tmp/SHA256E-s49892641--7390d24449d67e1d60c31306bfc8b189f7c4ed223cc44bbd0096222b8ede3b92.json"]
+[2016-08-26 15:58:37.018182] transfer start
+[2016-08-26 15:58:37.023135] call: rsync ["--server","-t","--inplace","-e.Lsf",".","--sender","../../srv/annex/.git/annex/objects/2G/fP/SHA256E-s49892641--7390d24449d67e1d60c31306bfc8b189f7c4ed223cc44bbd0096222b8ede3b92.json/SHA256E-s49892641--7390d24449d67e1d60c31306bfc8b189f7c4ed223cc44bbd0096222b8ede3b92.json"]
+SHA256E-s49892641--7390d24449d67e1d60c31306bfc8b189f7c4ed223cc44bbd0096222b8ede3b92.json
+     49,101,968  98%   46.83MB/s    0:00:00  [2016-08-26 15:59:22.809273] feed: ssh ["-S",".git/annex/ssh/voteviewdev","-o","ControlMaster=auto","-o","ControlPersist=yes","-T","voteviewdev","git-annex-shell 'transferinfo' '/srv/annex' '--debug' 'SHA256E-s49892641--7390d24449d67e1d60c31306bfc8b189f7c4ed223cc44bbd0096222b8ede3b92.json' --uuid ff59187e-f48c-47d0-b75d-320a6a4ce115 '--' 'remoteuuid=da1361a1-3f1a-47b8-8165-daf8c30d543d' 'associatedfile=clerk/H101-2.json' '--'"]
+     49,892,641 100%   39.52MB/s    0:00:01 (xfr#1, to-chk=0/1)
+[2016-08-26 15:58:39.034666] process done ExitSuccess
+[2016-08-26 15:58:39.038811] transfer done
+[2016-08-26 15:59:23.09308] process done ExitSuccess
+(checksum...) [2016-08-26 15:59:23.093582] read: sha256sum [".git/annex/tmp/SHA256E-s49892641--7390d24449d67e1d60c31306bfc8b189f7c4ed223cc44bbd0096222b8ede3b92.json"]
+[2016-08-26 15:59:23.331872] process done ExitSuccess
+ok
+get clerk/H102-1.json (from origin...) 
+[2016-08-26 15:59:23.337838] read: rsync ["--progress","--inplace","--perms","-e","'ssh' '-S' '.git/annex/ssh/voteviewdev' '-o' 'ControlMaster=auto' '-o' 'ControlPersist=yes' '-T' 'voteviewdev' 'git-annex-shell ''sendkey'' ''/srv/annex'' ''--debug'' ''SHA256E-s41311329--69c3b054a3fe9676133605730d85b7fcef8696f6782d402a524e41b836253891.json'' --uuid ff59187e-f48c-47d0-b75d-320a6a4ce115 ''--'' ''remoteuuid=da1361a1-3f1a-47b8-8165-daf8c30d543d'' ''unlocked='' ''direct='' ''associatedfile=clerk/H102-1.json'' ''--'''","--","dummy:",".git/annex/tmp/SHA256E-s41311329--69c3b054a3fe9676133605730d85b7fcef8696f6782d402a524e41b836253891.json"]
+[2016-08-26 15:59:23.351903] process done ExitSuccess
+[2016-08-26 15:58:39.394898] transfer start
+[2016-08-26 15:58:39.400323] call: rsync ["--server","-t","--inplace","-e.Lsf",".","--sender","../../srv/annex/.git/annex/objects/6Z/QP/SHA256E-s41311329--69c3b054a3fe9676133605730d85b7fcef8696f6782d402a524e41b836253891.json/SHA256E-s41311329--69c3b054a3fe9676133605730d85b7fcef8696f6782d402a524e41b836253891.json"]
+SHA256E-s41311329--69c3b054a3fe9676133605730d85b7fcef8696f6782d402a524e41b836253891.json
+      1,998,848   4%    1.91MB/s    0:00:20  [2016-08-26 15:59:24.608637] feed: ssh ["-S",".git/annex/ssh/voteviewdev","-o","ControlMaster=auto","-o","ControlPersist=yes","-T","voteviewdev","git-annex-shell 'transferinfo' '/srv/annex' '--debug' 'SHA256E-s41311329--69c3b054a3fe9676133605730d85b7fcef8696f6782d402a524e41b836253891.json' --uuid ff59187e-f48c-47d0-b75d-320a6a4ce115 '--' 'remoteuuid=da1361a1-3f1a-47b8-8165-daf8c30d543d' 'associatedfile=clerk/H102-1.json' '--'"]
+     10,616,832  25%    1.54MB/s    0:00:19  
+
+
+
+# End of transcript or log.
+"""]]
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+
+
diff --git a/doc/bugs/Allow_automatic_retry_git_annex_get/comment_1_a419d2868b4f99098644e84eef07d47c._comment b/doc/bugs/Allow_automatic_retry_git_annex_get/comment_1_a419d2868b4f99098644e84eef07d47c._comment
new file mode 100644
index 000000000..3b2b139e4
--- /dev/null
+++ b/doc/bugs/Allow_automatic_retry_git_annex_get/comment_1_a419d2868b4f99098644e84eef07d47c._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="adamboche@d77483462c747486c7a7341a9080d1341eeddfe6"
+ nickname="adamboche"
+ subject="Retry rsync on failure"
+ date="2016-08-26T23:32:46Z"
+ content="""
+This may be useful for providing an in-product solution to this problem.
+http://unix.stackexchange.com/a/165417/126433
+"""]]
diff --git a/doc/bugs/git_annex_info_--json_--fast_outputs_text_in_addition_to_the_json_string.mdwn b/doc/bugs/git_annex_info_--json_--fast_outputs_text_in_addition_to_the_json_string.mdwn
new file mode 100644
index 000000000..eb8a1ab90
--- /dev/null
+++ b/doc/bugs/git_annex_info_--json_--fast_outputs_text_in_addition_to_the_json_string.mdwn
@@ -0,0 +1,40 @@
+### Please describe the problem.
+
+`git annex info --json --fast 2>/dev/null` returns lines like `(merging origin/synced/git-annex into git-annex...)` befor the actual JSON string if updates where received with eg. `git pull` and `git annex sync` was not run yet.
+
+I would expect that the additional verbose output is suppressed if the --json switch is used (or maybe send to stderr?).
+
+However, it's easy to work around by piping the output through `grep -v -E '^\(.*\)$'`.
+
+I came across this when using `git annex info` to determine the description of the current repo:
+
+```
+git annex info --json --fast 2>/dev/null | grep -v -E '^\(.*\)$' | jq '."trusted repositories"+."semitrusted repositories"+."untrusted repositories" | [.[] | select(.here == true)] | .[length-1] | .description' | sed 's/^"\(.*\)"$/\1/'
+```
+
+which is used in some mr scripts that automatically set the repo description after cloning a git annex repo, based on the machine it us run on.
+
+### What steps will reproduce the problem?
+
+* clone a repo with git annex branches
+* run `git annex info --json --fast 2>/dev/null`
+
+### What version of git-annex are you using? On what operating system?
+
+git-annex version: 6.20160511-1~eugenesan~xenial1
+
+### Please provide any additional information below.
+
+[[!format sh """
+$ git annex info --json --fast 2>/dev/null
+(merging origin/synced/git-annex into git-annex...)
+(recording state in git...)
+{"command":"info","repository mode":"indirect","trusted repositories":[],"semitrusted repositories":[...}
+"""]]
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+
+I really like git annex and its concepts and I'm currently in the process of adding all my bigger files that don't fit into git repos to git annex repos which are managed on my machines using mr (another great tool!) and some mr scripts that automate working with git annex repos.
+So far it works very well and it seems like I finally found a scalable and convenient solution to protect all my files and make them accessible from everywhere.
+
+
diff --git a/doc/forum/Same_files_are_synced_over_and_over_again/comment_1_1260b5ede5a88a95e98855363455ca41._comment b/doc/forum/Same_files_are_synced_over_and_over_again/comment_1_1260b5ede5a88a95e98855363455ca41._comment
new file mode 100644
index 000000000..87b51550b
--- /dev/null
+++ b/doc/forum/Same_files_are_synced_over_and_over_again/comment_1_1260b5ede5a88a95e98855363455ca41._comment
@@ -0,0 +1,7 @@
+[[!comment format=mdwn
+ username="Horus"
+ subject="comment 1"
+ date="2016-08-26T08:32:05Z"
+ content="""
+Nobody having any clue on that?
+"""]]
diff --git a/doc/sync/comment_20_537b27219871a565ae7bb7f357cd3793._comment b/doc/sync/comment_20_537b27219871a565ae7bb7f357cd3793._comment
new file mode 100644
index 000000000..4a3721eb6
--- /dev/null
+++ b/doc/sync/comment_20_537b27219871a565ae7bb7f357cd3793._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="kartynnik@c8b57e8688e66fe58144cae5833b23948ae9b599"
+ nickname="kartynnik"
+ subject="Branch names containing slashes"
+ date="2016-08-29T17:30:44Z"
+ content="""
+1) When I have a branch \"some/branch/name\" containing slashes in its name, git-annex sync strips everything up to the last slash and creates \"synced/name\", which may clash with \"some/other/name\". Is there a workaround?
+
+2) Could the \"don't use synced branches\" behavior referred in the comments above somehow be configured on the repository side so that everyone cloning it doesn't need to configure it for himself?
+"""]]
diff --git a/doc/tips/Git_annex_and_Calibre.mdwn b/doc/tips/Git_annex_and_Calibre.mdwn
index 71f955656..95d010611 100644
--- a/doc/tips/Git_annex_and_Calibre.mdwn
+++ b/doc/tips/Git_annex_and_Calibre.mdwn
@@ -113,7 +113,7 @@ You could also use direct mode in place of the auto unlock feature
 
     git annex direct
 
-The remove the `post-commit` git hook (or do not add it). Its a
+Then remove the `post-commit` git hook (or do not add it). Its a
 simpler solution, but remember that interaction between git annex direct
 repositories and plain git are complex and sometimes downright dangerous. See [[direct mode]] for details.