Merge branch 'master' into tor

27 files changed, 583 insertions, 1 deletions

diff --git a/doc/bugs/Adding_zip_or_7z_or_tar_archive_builds_for_windows.mdwn b/doc/bugs/Adding_zip_or_7z_or_tar_archive_builds_for_windows.mdwn
index 6cca0082c..ae1f7c522 100644
--- a/doc/bugs/Adding_zip_or_7z_or_tar_archive_builds_for_windows.mdwn
+++ b/doc/bugs/Adding_zip_or_7z_or_tar_archive_builds_for_windows.mdwn
@@ -1,3 +1,5 @@
 Having a windows build of Git-Annex in an archived format would be very usefull for automation, and deploy.
 Could it be possible to add this to the buildserver of gitannex?
 
+[[!tag moreinfo]]
+
diff --git a/doc/bugs/Adding_zip_or_7z_or_tar_archive_builds_for_windows/comment_1_70480ffd417788f18cd75a9b625ecf3b._comment b/doc/bugs/Adding_zip_or_7z_or_tar_archive_builds_for_windows/comment_1_70480ffd417788f18cd75a9b625ecf3b._comment
new file mode 100644
index 000000000..3bd7381e1
--- /dev/null
+++ b/doc/bugs/Adding_zip_or_7z_or_tar_archive_builds_for_windows/comment_1_70480ffd417788f18cd75a9b625ecf3b._comment
@@ -0,0 +1,19 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2016-11-16T18:41:25Z"
+ content="""
+It would be helpful to have more details, such as an example of software
+distributed for windows that way, or documentation of how such an archive
+is used on windows.
+
+The git-annex Windows installer is a exe file that uses the NullSoft
+installation system. As far as I know that's pretty common in the Windows
+world.
+
+I don't see any point in zipping up the single exe. It would be possible to
+make a zip containing all the files that instlling the exe installs. But,
+the installation process has to integrate git-annex with git, it installs
+menu items, etc. A zip file would not be able to handle that integration.
+So its use seems limited to me.
+"""]]
diff --git a/doc/bugs/DBG__58___running___96____47__Users__47__joey__47__homebrew__47__opt__47__gpg-agent__47__bin__47__gpg-agent__39___for_testing_failed.mdwn b/doc/bugs/DBG__58___running___96____47__Users__47__joey__47__homebrew__47__opt__47__gpg-agent__47__bin__47__gpg-agent__39___for_testing_failed.mdwn
new file mode 100644
index 000000000..a3b85bdf2
--- /dev/null
+++ b/doc/bugs/DBG__58___running___96____47__Users__47__joey__47__homebrew__47__opt__47__gpg-agent__47__bin__47__gpg-agent__39___for_testing_failed.mdwn
@@ -0,0 +1,30 @@
+### Please describe the problem.
+Issue uploading to S3 remote (Dreamhost)
+
+### What steps will reproduce the problem?
+git-annex copy massart/f16_Web1/screencaptures/IMG_5159.MOV --to=cloud
+on my repo
+
+### What version of git-annex are you using? On what operating system?
+6.20161031-g0a58e94
+OS-X 10.11.6
+
+### Please provide any additional information below.
+I am using a different WIFI I haven't used before. Maybe it is blocking something…
+
+[[!format sh """
+git-annex copy massart/f16_Web1/screencaptures/IMG_5159.MOV --to=cloud
+copy massart/f16_Web1/screencaptures/IMG_5159.MOV (checking cloud...) (to cloud...) 
+17%           0.0 B/s 0sgpg: error running `/Users/joey/homebrew/opt/gpg-agent/bin/gpg-agent': probably not installed
+gpg: DBG: running `/Users/joey/homebrew/opt/gpg-agent/bin/gpg-agent' for testing failed: Configuration error
+gpg: can't connect to the agent: IPC connect call failed
+gpg: problem with the agent: No agent running
+35%       1021.8KB/s 30s
+  user error (gpg ["--quiet","--trust-model","always","--batch","--passphrase-fd","26","--symmetric","--force-mdc","--no-textmode"] exited 2)
+failed
+git-annex: copy: 1 failed
+"""]]
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+Yes.
+
diff --git a/doc/bugs/Single_space_in_file_name_causes_git_annex_add_to_fail.mdwn b/doc/bugs/Single_space_in_file_name_causes_git_annex_add_to_fail.mdwn
new file mode 100644
index 000000000..096562199
--- /dev/null
+++ b/doc/bugs/Single_space_in_file_name_causes_git_annex_add_to_fail.mdwn
@@ -0,0 +1,57 @@
+### Please describe the problem.
+
+If a filename has a single space (and only one space), `git annex add` will fail out with the following message:
+
+    add one two git-annex: unknown response from git cat-file ("HEAD:./one two missing",Ref "HEAD:./one two")
+    CallStack (from HasCallStack):
+      error, called at ./Git/CatFile.hs:102:28 in main:Git.CatFile
+
+### What steps will reproduce the problem?
+
+Run the following:
+
+    git init .
+    git annex init
+    touch "one two"
+    # this will cause error
+    git annex add "one two"
+    touch "one two three"
+    # this is fine
+    git annex add "one two three"
+
+### What version of git-annex are you using? On what operating system?
+
+Output of `git annex version`
+
+    git-annex version: 6.20161027
+    build flags: Assistant Webapp Pairing Testsuite S3(multipartupload)(storageclasses) WebDAV Inotify DBus DesktopNotify XMPP ConcurrentOutput TorrentParser MagicMime Feeds Quvi
+    key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
+    remote types: git gcrypt S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external
+    local repository version: 5
+    supported repository versions: 3 5 6
+    upgrade supported from repository versions: 0 1 2 3 4 5
+    operating system: linux x86_64
+
+Operating System: Linux (NixOS 16.09.909.238c7e0 (Flounder))
+
+### Please provide any additional information below.
+
+Maybe related to [https://git-annex.branchable.com/forum/unknown_response_from_git_cat-file/](https://git-annex.branchable.com/forum/unknown_response_from_git_cat-file/) or [https://git-annex.branchable.com/bugs/git_annex_import_fails_on_filenames_with_newlines_in_them/](https://git-annex.branchable.com/bugs/git_annex_import_fails_on_filenames_with_newlines_in_them/)?
+
+EDIT: Somewhat surprisingly, if I build from source using `cabal`, everything works fine.
+
+    .cabal-sandbox/bin/git-annex version
+    git-annex version: 6.20161113-g1e88c12
+    build flags: Assistant Webapp Pairing Testsuite WebDAV Inotify ConcurrentOutput TorrentParser MagicMime Feeds Quvi
+    key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
+    remote types: git gcrypt bup directory rsync web bittorrent webdav tahoe glacier ddar hook external
+
+Not sure whether this means that this bug is actually fixed or whether it's an artifact of how things are built in Nix.
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+
+Just starting out with it as a way of archiving some of my media seems to work great apart from this. Thanks a bunch!
+
+> This bug was already fixed in git-annex 6.20161031. I told the Debian
+> maintainer about the bug fix at the time; package has not been updated
+> yet. [[done]] on git-annex side. --[[Joey]] 
diff --git a/doc/bugs/Single_space_in_file_name_causes_git_annex_add_to_fail/comment_1_0cf0856c6408c9c588133023a3a6ba8f._comment b/doc/bugs/Single_space_in_file_name_causes_git_annex_add_to_fail/comment_1_0cf0856c6408c9c588133023a3a6ba8f._comment
new file mode 100644
index 000000000..5f8b120e2
--- /dev/null
+++ b/doc/bugs/Single_space_in_file_name_causes_git_annex_add_to_fail/comment_1_0cf0856c6408c9c588133023a3a6ba8f._comment
@@ -0,0 +1,12 @@
+[[!comment format=mdwn
+ username="gfa@1e86118cd41fbfea50004af221471ad97b55af18"
+ nickname="gfa"
+ avatar="http://cdn.libravatar.org/avatar/4678da4da55c67fa668e31ea0a76b201"
+ subject="same on Debian"
+ date="2016-11-14T09:13:19Z"
+ content="""
+I face the same issue on Debian testing
+
+git-annex 6.20161012-1
+git 1:2.10.2-1
+"""]]
diff --git a/doc/bugs/Webapp_missing_CSS_and_JS_resources___40__401_Unauthorized__41__.mdwn b/doc/bugs/Webapp_missing_CSS_and_JS_resources___40__401_Unauthorized__41__.mdwn
index 0c3fc16a1..17fe3ac25 100644
--- a/doc/bugs/Webapp_missing_CSS_and_JS_resources___40__401_Unauthorized__41__.mdwn
+++ b/doc/bugs/Webapp_missing_CSS_and_JS_resources___40__401_Unauthorized__41__.mdwn
@@ -30,4 +30,5 @@ operating system: darwin x86_64
 
 ### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
 
-
+> [[done]], my fix worked! Don't know entirely why it was needed..
+> --[[Joey]]
diff --git a/doc/bugs/Webapp_missing_CSS_and_JS_resources___40__401_Unauthorized__41__/comment_3_54bd11140dbe794182263c1a062ad031._comment b/doc/bugs/Webapp_missing_CSS_and_JS_resources___40__401_Unauthorized__41__/comment_3_54bd11140dbe794182263c1a062ad031._comment
new file mode 100644
index 000000000..cb6e9b4d2
--- /dev/null
+++ b/doc/bugs/Webapp_missing_CSS_and_JS_resources___40__401_Unauthorized__41__/comment_3_54bd11140dbe794182263c1a062ad031._comment
@@ -0,0 +1,21 @@
+[[!comment format=mdwn
+ username="christopher@5845ecd3cef9edadd4dc084df00e1fa60ce311eb"
+ nickname="christopher"
+ avatar="http://cdn.libravatar.org/avatar/4b722efb21f38d9944730c93727bc602"
+ subject="comment 3"
+ date="2016-11-15T12:15:37Z"
+ content="""
+Hi Joey,
+
+I installed git-annex using the homebrew recipe from https://github.com/Homebrew/homebrew-core/blob/master/Formula/git-annex.rb, OS X 10.11.6 (15G31)
+
+These are the dependencies reported by homebrew:
+
+Build: ghc ✔, cabal-install ✔, pkg-config ✔
+Required: gsasl ✔, libidn ✔, libmagic ✔, gnutls ✔, quvi ✔
+
+I've re-installed using \"brew install git-annex --HEAD\" to pull in your latest commit and I can confirm that everything works as expected and the /static/ resources load correctly.
+
+Thanks,
+Chris
+"""]]
diff --git a/doc/bugs/When_stopping___96__git_annex_get__96___files_left_broken.mdwn b/doc/bugs/When_stopping___96__git_annex_get__96___files_left_broken.mdwn
new file mode 100644
index 000000000..a8df72354
--- /dev/null
+++ b/doc/bugs/When_stopping___96__git_annex_get__96___files_left_broken.mdwn
@@ -0,0 +1,40 @@
+### Please describe the problem.
+
+```
+> git annex get Narnia/
+get Narnia/Course of a Generation/01 Sail Around the World.mp3 (from Seagate...) 
+SHA256E-s8395599--2fea961006a279f0765c45755b35a06f0a4fc6bfbab6118182ebc693d7b47a91.mp3
+      8,395,599 100%   29.65MB/s    0:00:00 (xfr#1, to-chk=0/1)
+(checksum...) ^C⏎                                   
+```
+
+```
+> mpv ~/Music/sorted/Narnia/Course\ of\ a\ Generation/
+Playing: /home/philip/Music/sorted/Narnia/Course of a Generation/
+[file] This is a directory - adding to playlist.
+
+Playing: /home/philip/Music/sorted/Narnia/Course of a Generation/01 Sail Around the World.mp3
+Failed to recognize file format.
+
+Playing: /home/philip/Music/sorted/Narnia/Course of a Generation/02 When the Stars Are Falling.mp3
+```
+
+```
+> git annex version
+git-annex version: 6.20161012
+build flags: Assistant Webapp Pairing Testsuite S3(multipartupload)(storageclasses) WebDAV Inotify DBus DesktopNotify XMPP ConcurrentOutput TorrentParser MagicMime Feeds Quvi
+key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
+remote types: git gcrypt S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external
+local repository version: 6
+supported repository versions: 3 5 6
+upgrade supported from repository versions: 0 1 2 3 4 5
+operating system: linux x86_64
+```
+
+Any consecutive `git annex get` commands don’t notice that the file is not completely transferred and leave it in a broken state.
+`git annex get --failed` does not correct the problem.
+
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+
+Yes, it (kind of) works for keeping my music library in sync.
diff --git a/doc/bugs/When_stopping___96__git_annex_get__96___files_left_broken/comment_1_9392346203c561b88f30fa2ce7540b76._comment b/doc/bugs/When_stopping___96__git_annex_get__96___files_left_broken/comment_1_9392346203c561b88f30fa2ce7540b76._comment
new file mode 100644
index 000000000..4f90ddfa6
--- /dev/null
+++ b/doc/bugs/When_stopping___96__git_annex_get__96___files_left_broken/comment_1_9392346203c561b88f30fa2ce7540b76._comment
@@ -0,0 +1,22 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2016-11-16T18:36:34Z"
+ content="""
+Thing is, git-annex get does not update the file in place. Only once the
+entire file is downloaded, and its content is verified correct is it moved
+into a place where you can access it.
+
+So, it seems much more likely to me that the content of the file, as
+originally added to git-annex, was bad, and the it had just finished
+verifying the content and moving it into place when you interruped the
+command.
+
+Please check with `git annex fsck` on the file and see if it determines
+it has the content git-annex expects it to have.
+
+However, I notice you're using a v6 repository. Is the file an unlocked
+file? It's possible that in that specific case there could be a bug.
+I've interrupted `git annex get` on a nearly daily basis for years, but
+v6 is still experimental and not as well tested.
+"""]]
diff --git a/doc/bugs/__34__commitBuffer__58___invalid_argument___40__invalid_character__41____34___during___34__git_annex_sync__34__.mdwn b/doc/bugs/__34__commitBuffer__58___invalid_argument___40__invalid_character__41____34___during___34__git_annex_sync__34__.mdwn
new file mode 100644
index 000000000..d4e3ad471
--- /dev/null
+++ b/doc/bugs/__34__commitBuffer__58___invalid_argument___40__invalid_character__41____34___during___34__git_annex_sync__34__.mdwn
@@ -0,0 +1,52 @@
+### Please describe the problem.
+
+In my unlocked adjusted branch, I get a lot of errors during "git annex sync". It appears to work fine otherwise (the files actually get synced). Below is what I see on the terminal. The repository is otherwise clean (no local or remote changes).
+This has started to happen around a month ago, though I cannot pinpoint the exact version. This is in the same repo you used to debug the disappearing files in direct mode recently (thanks a lot btw!).
+
+### What version of git-annex are you using? On what operating system?
+
+[[!format sh """
+$ git annex version
+git-annex version: 6.20161110-gd48f4ca
+build flags: Assistant Webapp Pairing Testsuite S3(multipartupload)(storageclasses) WebDAV Inotify DBus DesktopNotify XMPP ConcurrentOutput TorrentParser MagicMime Feeds Quvi
+key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
+remote types: git gcrypt S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external
+$ lsb_release -a
+No LSB modules are available.
+Distributor ID: Debian
+Description:    Debian GNU/Linux 8.6 (jessie)
+Release:        8.6
+Codename:       jessie
+"""]]
+
+### Please provide any additional information below.
+
+[[!format sh """
+$ git annex sync --content
+commit
+On branch adjusted/master(unlocked)
+nothing to commit, working tree clean
+ok
+pull origin
+remote: Counting objects: 113, done.
+remote: Compressing objects: 100% (113/113), done.
+remote: Total 113 (delta 112), reused 0 (delta 0)
+Receiving objects: 100% (113/113), 7.16 KiB | 0 bytes/s, done.
+Resolving deltas: 100% (112/112), completed with 112 local objects.
+From /srv/annex/bilder
+   97a4806..78cb4ef  git-annex  -> origin/git-annex
+ok
+(merging origin/git-annex into git-annex...)
+
+git-annex: fd:25: commitBuffer: invalid argument (invalid character)
+failed
+
+git-annex: fd:25: commitBuffer: invalid argument (invalid character)
+failed
+
+[...]
+
+git-annex: fd:25: commitBuffer: invalid argument (invalid character)
+failed
+git-annex: sync: 2653 failed
+"""]]
diff --git a/doc/bugs/add_fails_with_v6_repo_when_four_levels_deep/comment_1_e308245bf81a536db6f9a2b743d912bf._comment b/doc/bugs/add_fails_with_v6_repo_when_four_levels_deep/comment_1_e308245bf81a536db6f9a2b743d912bf._comment
new file mode 100644
index 000000000..a5d988fae
--- /dev/null
+++ b/doc/bugs/add_fails_with_v6_repo_when_four_levels_deep/comment_1_e308245bf81a536db6f9a2b743d912bf._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2016-11-16T18:49:09Z"
+ content="""
+I'm not able to reproduce the problem with your test case and git-annex
+version 6.20161012.
+
+Can you still reproduce it after upgrading?
+"""]]
diff --git a/doc/bugs/addurl_pathdepth_description_misleading/comment_3_2744e42db662486b46e203a72c3e56c7._comment b/doc/bugs/addurl_pathdepth_description_misleading/comment_3_2744e42db662486b46e203a72c3e56c7._comment
new file mode 100644
index 000000000..45be25186
--- /dev/null
+++ b/doc/bugs/addurl_pathdepth_description_misleading/comment_3_2744e42db662486b46e203a72c3e56c7._comment
@@ -0,0 +1,21 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 3"""
+ date="2016-11-16T19:04:37Z"
+ content="""
+Seems to work as described here:
+
+	joey@darkstar:~/tmp/r>rm localhost__joey_blog_index.html 
+	joey@darkstar:~/tmp/r>git annex addurl --pathdepth 2 http://localhost/~joey/blog/index.html
+	addurl blog/index.html (downloading http://localhost/~joey/blog/index.html ...) 
+	/home/joey/tmp/r/ 100%[===========>]  40.70K  --.-KB/s    in 0s      
+	ok
+	(recording state in git...)
+	joey@darkstar:~/tmp/r>ls
+	blog/
+	joey@darkstar:~/tmp/r>ls blog
+	index.html
+
+It would probably help if you can provide a test case where it does not work
+as described.
+"""]]
diff --git a/doc/bugs/git-annex-fsck___34__-all__34___flag_doesn__39__t_work_for_special_remote.mdwn b/doc/bugs/git-annex-fsck___34__-all__34___flag_doesn__39__t_work_for_special_remote.mdwn
new file mode 100644
index 000000000..80193cd54
--- /dev/null
+++ b/doc/bugs/git-annex-fsck___34__-all__34___flag_doesn__39__t_work_for_special_remote.mdwn
@@ -0,0 +1,28 @@
+### Please describe the problem.
+I tried to use `git-annex-fsck --all --from remote` to check files on a special remote, but git-annex did a scan of the local repo instead. If I don't use the `--all` flag, it correctly checks the files on the remote (but just the files in the current checked out branch).
+
+### What steps will reproduce the problem?
+    mkdir repo
+    mkdir special
+    cd repo
+    git init
+    git annex init
+    git annex initremote special type=directory directory=../special encryption=none
+    touch testfile
+    git annex add testfile
+    git annex copy testfile --to special
+    chmod -R +w ../special/*
+    rm -r ../special/*
+    git annex fsck --all --from special   # should check special remote but checks local repo instead
+    git diff git-annex^ git-annex         # activity log shows that it checked special remote
+    git annex fsck --from special         # correctly checks special remote, identifies missing file
+
+
+### What version of git-annex are you using? On what operating system?
+6.20161012 on Ubuntu 16.10
+
+### Have you had any luck using git-annex before?
+Yes, it's been very helpful for managing large files between laptops, desktops, external storage, and remote storage.
+
+> Thanks for an excellent test case and a clear bug report. I've fixed this
+> bug. [[done]] --[[Joey]]
diff --git a/doc/bugs/git-annex_won__39__t_execute_on_WD_My_Cloud_NAS/comment_8_48026cf7c187e97d53d15d35ed2c3670._comment b/doc/bugs/git-annex_won__39__t_execute_on_WD_My_Cloud_NAS/comment_8_48026cf7c187e97d53d15d35ed2c3670._comment
new file mode 100644
index 000000000..493031115
--- /dev/null
+++ b/doc/bugs/git-annex_won__39__t_execute_on_WD_My_Cloud_NAS/comment_8_48026cf7c187e97d53d15d35ed2c3670._comment
@@ -0,0 +1,14 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 8"""
+ date="2016-11-16T21:48:49Z"
+ content="""
+The arm daily build now uses a 32kb page size. So try
+<https://downloads.kitenet.net/git-annex/autobuild/armel/git-annex-standalone-armel.tar.gz>
+
+That has been verified to fix the problem on a Drobo 5N.
+
+This may still not be enough for some of the affected NAS devices, which
+use a 64kb page size. Unfortunately, gold fails to link with a 64kb page
+size: <http://bugs.debian.org/844467>
+"""]]
diff --git a/doc/design/assistant/telehash.mdwn b/doc/design/assistant/telehash.mdwn
index 373f1a575..5c410999f 100644
--- a/doc/design/assistant/telehash.mdwn
+++ b/doc/design/assistant/telehash.mdwn
@@ -123,6 +123,14 @@ so won't want to type that in. Need discovery.
   for Bob to confirm he's ready to finish pairing, this will fail,
   because Bob won't get to that point if the authtoken is intercepted.
 
+## local lan detection
+
+At connection time, after authentication, the remote can send
+(ip address, ssh host key). Try sshing to the ip address to check if
+the host key matches. If so, can enable a ssh remote, which will
+be cheaper than using the transport. Send the ssh public key back to the
+remote to get it authorized.
+
 ## remotedaemon
 
 See [[git-remote-daemon]] for its design.
diff --git a/doc/devblog/day_425__tor.mdwn b/doc/devblog/day_425__tor.mdwn
new file mode 100644
index 000000000..08fe21cdd
--- /dev/null
+++ b/doc/devblog/day_425__tor.mdwn
@@ -0,0 +1,23 @@
+Have waited too long for some next-generation encrypted P2P network, like
+telehash to emerge. Time to stop waiting; tor hidden services are not as
+cutting edge, but should work. Updated the [[design|design/assistant/telehash]]
+and started implementation in the `tor` branch.
+
+Unfortunately, Tor's default configuration does not enable the ControlPort.
+And, changing that in the configuration could be problimatic. This
+makes it harder than it ought to be to register a tor hidden service.
+So, I implemented a `git annex enable-tor` command, which can be run as root
+to set it up. The webapp will probably use `su-to-root` or `gksu` to run it. 
+There's some Linux-specific parts in there, and it uses a socket for
+communication between tor and the hidden service, which may cause problems
+for Windows porting later.
+
+Next step will be to get `git annex remotedaemon` to run as a tor hidden
+service.
+
+Also made a `no-xmpp` branch which removes xmpp support from the assistant.
+That will remove 3000 lines of code when it's merged. Will probably wait
+until after tor hidden services are working.
+
+Today's work was sponsored by Jake Vosloo on
+[Patreon](https://www.patreon.com/joeyh/).
diff --git a/doc/devblog/day_425__tor/comment_1_1dd41fa32eb3867d764f3238005b5b81._comment b/doc/devblog/day_425__tor/comment_1_1dd41fa32eb3867d764f3238005b5b81._comment
new file mode 100644
index 000000000..fe609cab0
--- /dev/null
+++ b/doc/devblog/day_425__tor/comment_1_1dd41fa32eb3867d764f3238005b5b81._comment
@@ -0,0 +1,11 @@
+[[!comment format=mdwn
+ username="grawity@2ea26be48562f66fcb9b66307da72b1e2e37453f"
+ nickname="grawity"
+ avatar="http://cdn.libravatar.org/avatar/7003e967f47003bae82966aa373de8ef"
+ subject="comment 1"
+ date="2016-11-15T18:01:18Z"
+ content="""
+…or `pkexec`, which is present on many systems and generally integrates better with whatever DE/non-DE the user may be running.
+
+(OTOH, pkexec does not set up X11 access – then again, root helpers shouldn't need it.)
+"""]]
diff --git a/doc/devblog/day_426__grab_bag.mdwn b/doc/devblog/day_426__grab_bag.mdwn
new file mode 100644
index 000000000..4a3bf908b
--- /dev/null
+++ b/doc/devblog/day_426__grab_bag.mdwn
@@ -0,0 +1,63 @@
+Fixed one howler of a bug today. Turns out that
+`git annex fsck --all --from remote` didn't actually check the content of
+the remote, but checked the local repository. Only `--all` was buggy;
+`git annex fsck --from remote` was ok. Don't think this is crash priority
+enough to make a release for, since only `--all` is affected.
+
+Somewhat uncomfortably made `git annex sync` pass
+`--allow-unrelated-histories` to git merge. While I do think that git's
+recent refusal to merge unrelated histories is good in general, the
+problem is that initializing a direct mode repository involves making an
+empty commit. So merging from a remote into such a direct mode repository
+means merging unrelated histories, while an indirect mode repository doesn't.
+Seems best to avoid such inconsistencies, and the only way I could see to
+do it is to always use `--allow-unrelated-histories`. May revisit this once
+direct mode is finally removed.
+
+Using the git-annex arm standalone bundle on some WD NAS boxes used to
+work, and then it seems they changed their kernel to use a nonstandard page
+size, and broke it. This actually seems to be a 
+[bug in the gold linker](http://bugs.debian.org/844467), which defaults to an
+unncessarily small page size on arm. The git-annex arm bundle is being
+adjusted to try to deal with this.
+
+ghc 8 made `error` include some backtrace information. While it's really
+nice to have backtraces for unexpected exceptions in Haskell, it turns
+out that git-annex used `error` a lot with the intent of showing an error
+message to the user, and a backtrace clutters up such messages. So,
+bit the bullet and checked through every `error` in git-annex and made such
+ones not include a backtrace.
+
+Also, I've been considering what protocol to use between git-annex nodes
+when communicating over tor. One way would be to make it very similar to
+`git-annex-shell`, using rsync etc, and possibly reusing code from
+git-annex-shell. However, it can take a while to make a connection across
+the tor network, and that method seems to need a new connection for each
+file transfered etc. Also thought about using a http based protocol. The
+servant library is great for that, you get both http client and server
+implementations almost for free. Resuming interrupted transfers might
+complicate it, and the hidden service side would need to listen on a unix
+socket, instead of the regular http port. It might be worth it to use http
+for tor, if it could be reused for git-annex http servers not on the tor
+network. But, then I'd have to make the http server support git pull and
+push over http in a way that's compatable with how git uses http, including
+authentication. Which is a whole nother ball of complexity. So, I'm leaning
+instead to using a simple custom protocol something like:
+
+		> AUTH $localuuid $token
+		< AUTH-OK $remoteuuid
+		> SENDPACK $length
+		> $gitdata
+		< RECVPACK $length
+		< $gitdata
+		> GET $pos $key
+		< SENDING $length
+		< $bytes
+		> GET-OK
+		> PUT $key
+		< SEND $pos
+		> SENDING $length
+		> $bytes
+		< PUT-OK
+
+Today's work was sponsored by Riku Voipio.
diff --git a/doc/devblog/day_426__grab_bag/comment_1_4d01c756850032d351fa99188a3301a7._comment b/doc/devblog/day_426__grab_bag/comment_1_4d01c756850032d351fa99188a3301a7._comment
new file mode 100644
index 000000000..7b5a2949a
--- /dev/null
+++ b/doc/devblog/day_426__grab_bag/comment_1_4d01c756850032d351fa99188a3301a7._comment
@@ -0,0 +1,11 @@
+[[!comment format=mdwn
+ username="https://anarc.at/openid/"
+ nickname="anarcat"
+ avatar="http://cdn.libravatar.org/avatar/b36dcf65657dd36128161355d8920a99503def9461c1bb212410980fe6f07125"
+ subject="how about reusing the special remote protocol?"
+ date="2016-11-16T21:58:08Z"
+ content="""
+git-annex already has a custom protocol detailed in [[design/external_special_remote_protocol]]. it could be quite useful to have that protocol extended to support direct object transfer instead of having to mess around with temporary files like may remotes do, for example...
+
+maybe that makes no sense at all, i don't know. :) --[[anarcat]]
+"""]]
diff --git a/doc/forum/Sending_requests_across_the_network/comment_3_9859c46db3527ad329c8e0df06edd153._comment b/doc/forum/Sending_requests_across_the_network/comment_3_9859c46db3527ad329c8e0df06edd153._comment
new file mode 100644
index 000000000..c63404e0e
--- /dev/null
+++ b/doc/forum/Sending_requests_across_the_network/comment_3_9859c46db3527ad329c8e0df06edd153._comment
@@ -0,0 +1,11 @@
+[[!comment format=mdwn
+ username="neocryptek@659edac901ffbc8e541a974f8f18987eeafc63bd"
+ nickname="neocryptek"
+ avatar="http://cdn.libravatar.org/avatar/d9bfdefa9b503f1ac4844a686618374e"
+ subject="comment 3"
+ date="2016-11-13T22:39:44Z"
+ content="""
+Thanks, that makes sense.
+
+All git annex repositories using the same branch will have the same (symlink) working directory right (assuming entire network has been synced eventually)?
+"""]]
diff --git a/doc/forum/sync_--content__44___fatal_is_outside_repository_errors/comment_2_09c62e4abf4ccc0d2e030ef5e1bcdf71._comment b/doc/forum/sync_--content__44___fatal_is_outside_repository_errors/comment_2_09c62e4abf4ccc0d2e030ef5e1bcdf71._comment
new file mode 100644
index 000000000..fcca1b28e
--- /dev/null
+++ b/doc/forum/sync_--content__44___fatal_is_outside_repository_errors/comment_2_09c62e4abf4ccc0d2e030ef5e1bcdf71._comment
@@ -0,0 +1,12 @@
+[[!comment format=mdwn
+ username="andrew"
+ avatar="http://cdn.libravatar.org/avatar/acc0ece1eedf07dd9631e7d7d343c435"
+ subject="how to investigate"
+ date="2016-11-16T15:37:01Z"
+ content="""
+Any thoughts? I am unsure how to investigate where this problem is. I assume these files are in my git repo or git-annex objects but I can't seem to find them using any search commands.
+
+Thanks,
+
+Andrew
+"""]]
diff --git a/doc/forum/sync_--content__44___fatal_is_outside_repository_errors/comment_2_8f694afa77f5a835c826d29d46d44615._comment b/doc/forum/sync_--content__44___fatal_is_outside_repository_errors/comment_2_8f694afa77f5a835c826d29d46d44615._comment
new file mode 100644
index 000000000..ccaeeb409
--- /dev/null
+++ b/doc/forum/sync_--content__44___fatal_is_outside_repository_errors/comment_2_8f694afa77f5a835c826d29d46d44615._comment
@@ -0,0 +1,30 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 2"""
+ date="2016-11-16T18:52:07Z"
+ content="""
+It would be helpful if you said what version of git-annex you are using.
+
+And, is your git-annex repository using the new experimental v6 format? One
+user reported a similar error message with a v6 git-annex repository. See
+[[bugs/assistant_crashes_in_TransferScanner]]
+
+Or might your repository be using direct mode?
+
+So, please paste in `git annex version` and `git annex info` output.
+
+It kind of looks like it's having difficulty determining where the top of
+the git repository is, or constructing a relative path to the git
+repository.
+
+Are there any symlinks in the path to /Users/andrew/notes ? Eg, is /Users
+a symlink, or /Users/andrew a symlink, or //Users/andrew/notes itself
+symlinked to elsewhere?
+
+Does only `git annex sync --content` fail? What if you run, eg
+`git annex copy --auto --to cloud` and `git annex get --auto --from cloud`,
+does that fail similarly, or does it succeed?
+
+You say it's only failing for some files. Do the filenames that it's
+failing on contain any non-ascii characters?
+"""]]
diff --git a/doc/forum/vanilla_git_repo_as_special_remote__63__.mdwn b/doc/forum/vanilla_git_repo_as_special_remote__63__.mdwn
new file mode 100644
index 000000000..94fa54865
--- /dev/null
+++ b/doc/forum/vanilla_git_repo_as_special_remote__63__.mdwn
@@ -0,0 +1,27 @@
+Right now I have separate "normal" Git repositories and separate Git annex repositories and I would love to have Git annex track and sync everything for me. The problem I have is I'd like to use "real" Git content tracking for some data (ex: text files) where I'd like to get normal Git features with (ex: diff). I'd like to combine normal Git content tracking with Git annex location tracking and syncing if possible. Ideally the cost (ex: increased git repo size and git slowdown) of content tracking would not need to be propagated across the entire git annex network, just on repos that want it (just like git annex only copies content to clients who want it and symlink the rest).
+
+The largefiles config provides a mechanism to add content to git directly in git annex, but that cost would be applied across the entire network, not opt-in per client.
+
+Ideally I'd like this situation:
+
+1. Git annex tracking everything as symlinks. No content is checked into these git repos.
+2. A subset of git annex content (ex: subfolder) synced to a normal remote non-annex git repository (ex: GitHub). This Git repo has content tracked in git itself.
+
+And I could use the git annex repos to sync everything. Somehow the git annex repo would know that the #2 remote was a "special content git remote" and push any content updates as normal git content commits.
+
+Or an adjusted branch that had the content tracked and I could sync that content branch around to only the remotes where I wanted the content history stored in git (since adjusted branches don't seem to annex sync by default). But master would just track the symlinks of those files and be synced around to all annexes.
+
+Can adjusted branches do this somehow?
+
+Some references:
+
+* [[special_remotes/external]]
+* [[design/adjusted_branches]]
+* [[todo/hide_missing_files]]
+* [[tips/largefiles]]
+* [[submodules]]
+* [[forum/git-subtree_support__63__]]
+
+Thanks!
+
+-neocryptek
diff --git a/doc/forum/vanilla_git_repo_as_special_remote__63__/comment_1_67e186265ae21f2cd8451750152f2a6d._comment b/doc/forum/vanilla_git_repo_as_special_remote__63__/comment_1_67e186265ae21f2cd8451750152f2a6d._comment
new file mode 100644
index 000000000..65397495a
--- /dev/null
+++ b/doc/forum/vanilla_git_repo_as_special_remote__63__/comment_1_67e186265ae21f2cd8451750152f2a6d._comment
@@ -0,0 +1,13 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2016-11-16T18:30:12Z"
+ content="""
+You can use bup as a special remote, which will store the content in a git
+repository. But, not in a form that git diff can be used with.
+
+[[git-annex-diffdriver]] can be used to make `git diff` work on annexed
+files. For example:
+
+	export GIT_EXTERNAL_DIFF="git-annex diffdriver -- diff -u --"
+"""]]
diff --git a/doc/special_remotes/S3/comment_28_c4dafad82a898eafd6d9e3703fad2c48._comment b/doc/special_remotes/S3/comment_28_c4dafad82a898eafd6d9e3703fad2c48._comment
new file mode 100644
index 000000000..864974205
--- /dev/null
+++ b/doc/special_remotes/S3/comment_28_c4dafad82a898eafd6d9e3703fad2c48._comment
@@ -0,0 +1,12 @@
+[[!comment format=mdwn
+ username="David_K"
+ avatar="http://cdn.libravatar.org/avatar/09dd8544695feb9b8d8ee54e4ff0168d"
+ subject="comment 28"
+ date="2016-11-16T01:28:14Z"
+ content="""
+I'd like to reiterate a question that was unanswered above:
+
+Is there a way to tell the S3 backend to store the files as they are named locally, instead of by hashed content name? i.e., I've annexed foo/bar.txt and annex puts it in s3 as mybucket.name/foo/bar.txt instead of mybucket.name/GPGHMACSHA1-random.txt
+
+
+"""]]
diff --git a/doc/tips/How_to_retroactively_annex_a_file_already_in_a_git_repo/comment_7_603db6818d33663b70b917c04fd8485b._comment b/doc/tips/How_to_retroactively_annex_a_file_already_in_a_git_repo/comment_7_603db6818d33663b70b917c04fd8485b._comment
new file mode 100644
index 000000000..5527c2b43
--- /dev/null
+++ b/doc/tips/How_to_retroactively_annex_a_file_already_in_a_git_repo/comment_7_603db6818d33663b70b917c04fd8485b._comment
@@ -0,0 +1,30 @@
+[[!comment format=mdwn
+ username="https://launchpad.net/~stephane-gourichon-lpad"
+ nickname="stephane-gourichon-lpad"
+ avatar="http://cdn.libravatar.org/avatar/02d4a0af59175f9123720b4481d55a769ba954e20f6dd9b2792217d9fa0c6089"
+ subject="&quot;Hmm, guyz? Are you serious with these scripts?&quot; Well, what's the matter?"
+ date="2016-11-15T10:58:32Z"
+ content="""
+## Wow, scary
+
+Dilyin's comment is scary.  It suggests bad things can happen, but is not very clear.
+
+Bloated history is one thing.  
+Obviously broken repo is bad but can be (slowly) recovered from remotes.  
+Subtly crippled history that you don't notice can be a major problem (especially once you have propagated it to all your remotes to \"recover from bloat\").
+
+## More common than it seems
+
+There's a case probably more common than people actually report: mistakenly doing `git add` instead of `git annex add` and realizing it only after a number of commits. Doing `git annex add` at that time will have the file duplicated (regular git and annex). 
+
+Extra wish: when doing `git annex add` of a file that is already present in git history, `git-annex` could notice and tell.
+
+## Simple solution?
+
+Can anyone elaborate on the scripts provided here, are they safe?  What can happen if improperly used or in corner cases?
+
+* \"files are replaced with symlinks and are in the index\" -> so what ?
+* \"Make sure that you don't have annex.largefiles settings that would prevent annexing the files.\" -> What would happen? Also `.gitattributes`.
+
+Thank you.
+"""]]
diff --git a/doc/todo/xmpp_removal.mdwn b/doc/todo/xmpp_removal.mdwn
index 9eb040780..c517c33f9 100644
--- a/doc/todo/xmpp_removal.mdwn
+++ b/doc/todo/xmpp_removal.mdwn
@@ -21,5 +21,7 @@ telehash. But, can't wait on that forever..
 XMPP support is already disabled by default in some builds of git-annex,
 notably the stack build. It's never worked on Windows.
 
+The [[no-xmpp]] branch is ready for merging.
+
 Next step is probably to default the flag to false by default,
 except for in a few builds like the Debian package and standalone builds.