Merge branch 'master' of ssh://git-annex.branchable.com

14 files changed, 284 insertions, 10 deletions

diff --git a/doc/bugs/Android:_does_not_upload_added_files___40__neither_to___34__Transfer__34___or___34__Full_backup__34___server__41__/comment_4_b54c169a96e263e12495690fe14d8b4a._comment b/doc/bugs/Android:_does_not_upload_added_files___40__neither_to___34__Transfer__34___or___34__Full_backup__34___server__41__/comment_4_b54c169a96e263e12495690fe14d8b4a._comment
new file mode 100644
index 000000000..7453e7cea
--- /dev/null
+++ b/doc/bugs/Android:_does_not_upload_added_files___40__neither_to___34__Transfer__34___or___34__Full_backup__34___server__41__/comment_4_b54c169a96e263e12495690fe14d8b4a._comment
@@ -0,0 +1,14 @@
+[[!comment format=mdwn
+ username="etset"
+ ip="188.83.111.161"
+ subject="comment 4"
+ date="2013-12-29T15:12:43Z"
+ content="""
+I also can't transfer from and to an Android tablet with the git-annex assistant. The `daemon.log` shows the same error, `Invalid pid specified`, repeated several times.
+
+Enabling the debug log mode always shows lines similar to this one before each \"Invalid pid\" line: `[2013-12-29 14:50:49 WET] chat: nice [\"ionice\", \"-c3\", \"git-annex\", \"transferkeys\"]`.
+
+However, using the `git annex get` and `git annex copy` commands to fetch and send the files from the same tablet work as expected.
+
+I can post the full log later, if needed.
+"""]]
diff --git a/doc/bugs/on_encfs__44___.fuse__95__hiddenXXXXX_files_left_behind.mdwn b/doc/bugs/on_encfs__44___.fuse__95__hiddenXXXXX_files_left_behind.mdwn
new file mode 100644
index 000000000..6c8f73f5e
--- /dev/null
+++ b/doc/bugs/on_encfs__44___.fuse__95__hiddenXXXXX_files_left_behind.mdwn
@@ -0,0 +1,52 @@
+### Please describe the problem.
+
+When git annex is used on a encfs mount, ``.fuse_hiddenXXXX`` files are left behind in ``.git/annex/objects/*/*/`` when I ``git annex move`` files off the encfs
+
+### What steps will reproduce the problem?
+
+    mkdir normal plain enc
+    encfs -o kernel_cache "$(realpath enc)" "$(realpath plain)"
+
+    cd normal
+    git init
+    git annex init normal
+
+    cd ../plain
+    git init
+    git annex init enc
+    dd if=/dev/urandom bs=1M count=11 of=11m
+    git annex add 11m
+    git remote add normal "$(realpath ../normal)"
+    du -hscL *
+        << 11MB-ish >>
+    du -hsc .git/annex/objects
+        << 11MB-ish >>
+    git annex move --to=normal 11m
+    du -hscL *
+        << 0 >>
+    du -hsc .git/annex/objects
+        << 11MB-ish but should be 0-ish >>
+    tree -a .git/annex/objects
+        << ??/??/SHA1-*/.fuse_hidden??????? >>
+    lsof .git/annex/objects/??/??/SHA*/.fuse_hidden*
+        << shows no programs have this file open >>
+
+
+### What version of git-annex are you using? On what operating system?
+
+5.20131221 on debian unstable (installed by aptitude from debian/main)
+
+### Please provide any additional information below.
+
+I did a couple quick internet searches on these .fuse_hidden files, and (if a random person on the internet is correct) they are created by enfs (or maybe fuse itself) when a file is unlinked while open. It is supposed to be deleted when the file is closed.
+
+Possibilities come to mind:
+
+*    There's a bug in encfs/fuse
+*    git annex fails to close the file, and whatever cleanup happens when the linux kernel closes git annex fails to trigger encfs/fuse's real unlinking/cleanup thing
+
+
+
+I'm happy to help with further testing, experimenting, etc. I'm good at git, the commandline, programming, etc but I don't know Haskell.
+
+- Jason <jason@jasonwoof.com>
diff --git a/doc/bugs/one_test_fail_on_osx.mdwn b/doc/bugs/one_test_fail_on_osx.mdwn
new file mode 100644
index 000000000..3f59e2a73
--- /dev/null
+++ b/doc/bugs/one_test_fail_on_osx.mdwn
@@ -0,0 +1,27 @@
+### Please describe the problem.
+
+git-annex test reports: 1 out of 120 tests failed
+
+    ...
+      Unit Tests (indirect)
+    ...
+        lock:                                           FAIL
+          lock --force failed
+    ...
+
+### What version of git-annex are you using? On what operating system?
+
+    $ git-annex version
+    git-annex version: 5.20131221-g00d1673
+    build flags: Assistant Webapp Pairing Testsuite S3 WebDAV FsEvents XMPP DNS Feeds Quvi TDFA CryptoHash
+    key/value backends: SHA256E SHA1E SHA512E SHA224E SHA384E SKEIN256E SKEIN512E SHA256 SHA1 SHA512 SHA224 SHA384 SKEIN256 SKEIN512 WORM URL
+    remote types: git gcrypt S3 bup directory rsync web webdav glacier hook
+    local repository version: 3
+    default repository version: 3
+    supported repository versions: 3 5
+    upgrade supported from repository versions: 0 1 2 4
+
+    $ git --version
+    git version 1.8.4
+
+    System Version: OS X 10.9 (13A603)
diff --git a/doc/bugs/webdav_special_remote_fails_on_android___40__box.com__41__.mdwn b/doc/bugs/webdav_special_remote_fails_on_android___40__box.com__41__.mdwn
new file mode 100644
index 000000000..95c4a77d3
--- /dev/null
+++ b/doc/bugs/webdav_special_remote_fails_on_android___40__box.com__41__.mdwn
@@ -0,0 +1,30 @@
+### Please describe the problem.
+
+The Box.com/WebDAV special remote fails with this error on my Android tablet:
+
+    WebDAV failed to write file: InternalIOException user error (Network.Socket.setSocketOption: socket option NoDelay unsupported on this system): user error
+
+### What steps will reproduce the problem?
+
+ 1. Install the Android beta from the download page.
+ 2. Create a repository (using the assistant)
+ 3. (At this point I configured XMPP, if that's relevant.)
+ 4. Configure a Box.com repository using the assistant
+
+The repository will fail to add with the above error message.
+
+### What version of git-annex are you using? On what operating system?
+
+git-annex 5.20131221-gf8c928d
+
+Android 4.0.4<br>
+Kernel 2.6.35-palm-tenderloin<br>
+CyanogenMod 9-20121202-NIGHTLY-tenderloin (HP Touchpad)
+
+### Please provide any additional information below.
+
+I have successfully set up the same Box.com account via the assistant on another machine.
+
+I don't see any mention of WebDAV in daemon.log and friends.  
+
+It seems like NODELAY should be optional, as it's just a performance optimization (and disabling packet coalescing may be battery-hostile on a mobile device, too).
diff --git a/doc/design/external_special_remote_protocol/comment_3_c763e44d06d9f50f0d357889b180b819._comment b/doc/design/external_special_remote_protocol/comment_3_c763e44d06d9f50f0d357889b180b819._comment
new file mode 100644
index 000000000..08d405c8e
--- /dev/null
+++ b/doc/design/external_special_remote_protocol/comment_3_c763e44d06d9f50f0d357889b180b819._comment
@@ -0,0 +1,15 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawmkBwMWvNKZZCge_YqobCSILPMeK6xbFw8"
+ nickname="TobiasTheViking"
+ subject="Feature requests"
+ date="2013-12-28T13:57:35Z"
+ content="""
+PREPARE-Failure ErrorMsg (matching INITREMOTE-FAILURE ErrorMsg)
+
+Also, i'd like for the following to overwrite existing credentials/configs 
+MYFOLDER=\"testfolder\" MYLOGIN=\"login\" MYPASSWORD=\"pword\" MYURL=\"http://webdav/\"    git annex enableremote owncloud type=external externaltype=owncloud --debug
+
+This would also be needed for refreshing oauth.
+
+
+"""]]
diff --git a/doc/forum/DO_NOT_use_ntfs-3g_on_mac_osx_for_annex.mdwn b/doc/forum/DO_NOT_use_ntfs-3g_on_mac_osx_for_annex.mdwn
new file mode 100644
index 000000000..16750a1fb
--- /dev/null
+++ b/doc/forum/DO_NOT_use_ntfs-3g_on_mac_osx_for_annex.mdwn
@@ -0,0 +1,41 @@
+After testing around git-annex on my mac using ntfs-3g to access the removable harddrive, I conclude that this is very dangerous. 
+
+Here are some example error snippets:
+
+    add man/man1/git-tar-tree.1 
+    git-annex: /Volumes/SeagateExpansion/annex/testdirect/.git/annex/objects/0K/JJ/SHA256E-s3805--948fd508a364e036ec28f5287d11825062aea2c0ed86a3a89ed871edc6ddc7a8.1/: openTempFile: interrupted (Interrupted system call)
+    failed
+
+    add man/man1/git-bundle.1 
+    git-annex: /Volumes/SeagateExpansion/annex/testdirect/.git/annex/tmp/e3f_804_SHA256E-s8739--569a4b4fe1072e17ff0aa9a2f67d0130b9ec3ed23f96c5a4ea38ada82f50b4aa.1.log: rename: does not exist (No such file or directory)
+    failed
+
+    add git-core/git-completion.bash failed
+    add man/man7/gitglossary.7 failed
+    (Recording state in git...)
+
+    git-annex: /Volumes/SeagateExpansion/annex/testdirect/.git/annex/journal/.fuse_hidden00005dd50000010a: removeLink: does not exist (No such file or directory)
+    failed
+    git-annex: add: 3 failed
+
+It's interesting that after each git-annex add ., it would got fewer failures. But in the end it stuck at 2 files. After 60 times doing: git-annex add .
+
+It went from this:
+
+    git-annex: add: 248 failed
+
+to this:
+
+    git-annex: add: 2 failed
+
+Now the repository has 246 objects, let's try to convert it into an indirect one:
+
+    git-annex indirect
+    ...
+    git-annex info
+    ...
+    annexed files in working tree: 9
+
+Only 9 files made into the indirect repository! Remaining files stay as original state. Now it would be a nightmare to try to recover form this mess.
+
+I discourage anyone from using ntfs-3g on mac for annexing. Especially direct & indirect mode. Bare repo seems to be fine however.
diff --git a/doc/forum/Webapp_not_watching_repos/comment_2_8ab2d4c18ddc99774e44c4f4401bfa57._comment b/doc/forum/Webapp_not_watching_repos/comment_2_8ab2d4c18ddc99774e44c4f4401bfa57._comment
new file mode 100644
index 000000000..63573b023
--- /dev/null
+++ b/doc/forum/Webapp_not_watching_repos/comment_2_8ab2d4c18ddc99774e44c4f4401bfa57._comment
@@ -0,0 +1,12 @@
+[[!comment format=mdwn
+ username="https://me.yahoo.com/a/FHnTlSBo1eCGJRwueeKeB6.RCaPbGMPr5jxx8A--#ce0d8"
+ nickname="Hamza"
+ subject="comment 2"
+ date="2013-12-27T22:40:06Z"
+ content="""
+I don't have a desktop env. just plain xmonad, so the correct sequence to start the webapp is,
+
+    git annex assistant --autostart && git annex webapp
+
+right?
+"""]]
diff --git a/doc/forum/confused_about_external_drives.mdwn b/doc/forum/confused_about_external_drives.mdwn
new file mode 100644
index 000000000..9fdb5c979
--- /dev/null
+++ b/doc/forum/confused_about_external_drives.mdwn
@@ -0,0 +1,30 @@
+First off, to get that off my chest: I am not using Gnome, KDE, XFCE or any "desktop manager" (well, not quite true, I use some bits of XFCE, but it's easier to assume I'm not).
+
+So automounting is always a little tricky for me, and I often use `pmount` to mount external drives and flash cards.
+
+Now, I have tried to add an external drive as a backup to one of my git-annex repositories. I zero'd the drive, added a GPT partition (I wanted labels), formatted it as `ext4` and then gave it to my user:
+
+    cfdisk /dev/sdd
+    mkfs -t ext4 /dev/sdd1
+    mount /dev/sdd1 /mnt
+    chown anarcat /mnt
+    umount /mnt
+
+Now, at this step I would have assumed the drive would be seen by the webapp, but that wasn't the case. Maybe I wasn't patient enough? In any case, at some point I got tired and went to the commandline to add the drive as a remote. I did the following:
+
+    mkdir -p /media/ata-WDC_WD15EADS-00P8B0_WD-WMAVU0748851-part1
+    mount /dev/disk/by-id/ata-WDC_WD15EADS-00P8B0_WD-WMAVU0748851-part1 /media/ata-WDC_WD15EADS-00P8B0_WD-WMAVU0748851-part1
+    cd /srv/mp3
+    git annex remote add backup /media/ata-WDC_WD15EADS-00P8B0_WD-WMAVU0748851-part1/mp3
+    git init /media/ata-WDC_WD15EADS-00P8B0_WD-WMAVU0748851-part1/mp3
+    git annex sync
+    git annex copy --to backup
+
+So that worked, and the webapp was now seeing `backup` as yet another repository, but not quite removeable. So failed. Scratching that, I stopped the assistant, and trashed the repository:
+
+    rm -rf /media/ata-WDC_WD15EADS-00P8B0_WD-WMAVU0748851-part1/mp3
+    git remote rm backup
+
+But now, oddly enough, the webapp would see the external drive after being restarted, and I was able to add that drive.
+
+What am I doing wrong? how is the assistant finding those drives? --[[anarcat]]
diff --git a/doc/forum/luks_encrypted_disk_support.mdwn b/doc/forum/luks_encrypted_disk_support.mdwn
new file mode 100644
index 000000000..e2f9efc06
--- /dev/null
+++ b/doc/forum/luks_encrypted_disk_support.mdwn
@@ -0,0 +1,3 @@
+Is there a way to easily access, say, a FDE (Full Disk Encrypted) external drive using git-annex?
+
+My use case is that I do not wish to encrypt each file individually - and find it more reliable to use a real filesystem encryption on this external hard drive. --[[anarcat]]
diff --git a/doc/forum/No_output_from_git-annex_status_v5.20131224-g692aa01/comment_3_decd893fab265d0d0dc8c20f6a70d8e4._comment b/doc/forum/trusted_repositories:_fatal:_not_a_git_repo.mdwn
index 74e2a89da..f94bd3804 100644
--- a/doc/forum/No_output_from_git-annex_status_v5.20131224-g692aa01/comment_3_decd893fab265d0d0dc8c20f6a70d8e4._comment
+++ b/doc/forum/trusted_repositories:_fatal:_not_a_git_repo.mdwn
@@ -1,15 +1,9 @@
-[[!comment format=mdwn
- username="https://openid.stackexchange.com/user/e65e6d0e-58ba-41de-84cc-1f2ba54cf574"
- nickname="Mica Semrick"
- subject="Now I'm getting..."
- date="2013-12-27T08:01:44Z"
- content="""
+When issuing a `git annex info`, I get:
+
 ~~~
 $ git-annex info
 repository mode: indirect
-trusted repositories: fatal: Not a git repository: '/home/micas/Music/.git'
-fatal: Not a git repository: '/home/micas/Music/.git'
-0
+trusted repositories: fatal: Not a git repository: '/home/micas/Music/.git' fatal: Not a git repository: '/home/micas/Music/.git' 0
 semitrusted repositories: 3
 	00000000-0000-0000-0000-000000000001 -- web
  	85f8a5ea-6278-11e2-9978-ebb59e8f37a2 -- here (Music annex backup)
@@ -26,4 +20,13 @@ backend usage:
 	SHA256: 10692
 	SHA256E: 30
 ~~~
-"""]]
+
+The troubling part (I think) is `trusted repositories: fatal: Not a git repository: '/home/micas/Music/.git' fatal: Not a git repository: '/home/micas/Music/.git' 0`
+
+Is there a command I can use to show all the uuid of known remotes? I thought I had marked all remotes pointing to /home/micas/Music/.git as `dead`
+
+Is there another reason for the output? How can I get trusted repos back to 0?
+
+**EDIT:**
+
+I removed the folder (was not a git repo at the time of the error) `/home/micas/Music` and the error went away.
diff --git a/doc/install/ArchLinux/comment_5_cbc960cd78bf5b90e3bb6cb605d6d970._comment b/doc/install/ArchLinux/comment_5_cbc960cd78bf5b90e3bb6cb605d6d970._comment
new file mode 100644
index 000000000..b3ed0846b
--- /dev/null
+++ b/doc/install/ArchLinux/comment_5_cbc960cd78bf5b90e3bb6cb605d6d970._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="http://alerque.com/"
+ nickname="Caleb"
+ subject="Out of date"
+ date="2013-12-28T22:04:24Z"
+ content="""
+The AUR package you reference is woefully out of date. I have updated the standalone variant so it might be worth using that until the maintainer catches up.
+
+    yaourt -Sy git-annex-standalone
+"""]]
diff --git a/doc/special_remotes/xmpp/comment_11_05d9a67b9331656b4035399641cedb1b._comment b/doc/special_remotes/xmpp/comment_11_05d9a67b9331656b4035399641cedb1b._comment
new file mode 100644
index 000000000..4b343bda2
--- /dev/null
+++ b/doc/special_remotes/xmpp/comment_11_05d9a67b9331656b4035399641cedb1b._comment
@@ -0,0 +1,22 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawm0yDI6AsAuXPDHIDXm5H9EGDXkuwb1efA"
+ nickname="Yann"
+ subject="XMPP not working in Raspberry PI build"
+ date="2013-12-27T23:16:23Z"
+ content="""
+Hi,
+
+I have the same problem that users Carlo and RaspberryPie have: XMPP not working and no XMPP log in the build.
+
+I did a quick strace during the daemon start and I noticed than the raspberry pi build doesn't even try to read the creds/xmpp files contrary to the standard git-annex package on my ubuntu workstation.
+
+To me, it seems the XMPP thread is not started at all.
+Does someone know any reason why the XMPP thread could not be started ?
+
+If there is anything I can do to help debug this issue, let me know.
+
+Yann
+
+
+
+"""]]
diff --git a/doc/todo/Bittorrent-like_features/comment_2_83148bd5c5c5e6c2eff3ad6e1d4fb82c._comment b/doc/todo/Bittorrent-like_features/comment_2_83148bd5c5c5e6c2eff3ad6e1d4fb82c._comment
new file mode 100644
index 000000000..ed6e92fd8
--- /dev/null
+++ b/doc/todo/Bittorrent-like_features/comment_2_83148bd5c5c5e6c2eff3ad6e1d4fb82c._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawmTIL7ubr5opWM69Q5VtCxuxC2H0SSnzic"
+ nickname="Petr"
+ subject="Regarding the approach to connect all nodes in one network and stream from them"
+ date="2013-12-29T07:32:05Z"
+ content="""
+Have a look at http://socialvpn.wordpress.com/. Behind that project is couple of interesting tools worth to investigate. 
+"""]]
diff --git a/doc/users/anarcat.mdwn b/doc/users/anarcat.mdwn
index c7054ff19..df98c9330 100644
--- a/doc/users/anarcat.mdwn
+++ b/doc/users/anarcat.mdwn
@@ -39,3 +39,10 @@ Fixed
 
 [[!inline pages="bugs/* and !bugs/done and link(bugs/done) and
 link(users/anarcat) and !bugs/*/*" feeds=no actions=yes archive=yes show=0]]
+
+Forum posts
+===========
+
+Forums where I posted.
+
+[[!inline pages="forum/* and link(users/anarcat)" sort=mtime feeds=no actions=yes archive=yes show=0]]