Merge branch 'master' of ssh://git-annex.branchable.com

4 files changed, 200 insertions, 0 deletions

diff --git a/doc/bugs/drop_from_special_hybrid_encryption_remote_fails.mdwn b/doc/bugs/drop_from_special_hybrid_encryption_remote_fails.mdwn
new file mode 100644
index 000000000..13e06bd28
--- /dev/null
+++ b/doc/bugs/drop_from_special_hybrid_encryption_remote_fails.mdwn
@@ -0,0 +1,57 @@
+### Please describe the problem.
+
+I am trying to drop a file from a special, hybrid encryption remote. It says "failed", but I cannot figure out why, the commands in --debug all seem to exit with ExitSuccess.
+
+### What steps will reproduce the problem?
+
+Not sure how to reproduce from scratch, see log below for what happens when I do 
+git annex drop file -f specialremote ...
+
+### What version of git-annex are you using? On what operating system?
+
+[[!format sh """
+$ git annex version
+git-annex version: 5.20150916+gitg79661ef-1~ndall+1
+build flags: Assistant Webapp Webapp-secure Pairing Testsuite S3 WebDAV Inotify DBus DesktopNotify XMPP DNS Feeds Quvi TDFA Database
+key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
+remote types: git gcrypt S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external
+local repository version: 5
+supported repository version: 5
+upgrade supported from repository versions: 0 1 2 4
+"""]]
+
+### Please provide any additional information below.
+
+[[!format sh """
+[2015-12-27 00:15:58.093036] read: git ["--git-dir=../../.git","--work-tree=../..","--literal-pathspecs","ls-files","--cached","-z","--","20150816_cccamp/DSC04537.ARW"]
+[2015-12-27 00:15:58.147092] read: git ["--git-dir=../../.git","--work-tree=../..","--literal-pathspecs","show-ref","git-annex"]
+[2015-12-27 00:15:58.150596] process done ExitSuccess
+[2015-12-27 00:15:58.150901] read: git ["--git-dir=../../.git","--work-tree=../..","--literal-pathspecs","show-ref","--hash","refs/heads/git-annex"]
+[2015-12-27 00:15:58.155607] process done ExitSuccess
+[2015-12-27 00:15:58.157843] read: git ["--git-dir=../../.git","--work-tree=../..","--literal-pathspecs","log","refs/heads/git-annex..85eb4dbbebcc100c2414f5a71f4fb9a22f0239fb","-n1","--pretty=%H"]
+[2015-12-27 00:15:58.162227] process done ExitSuccess
+[2015-12-27 00:15:58.162985] read: git ["--git-dir=../../.git","--work-tree=../..","--literal-pathspecs","log","refs/heads/git-annex..c0747d4c69cf80aa0290f1385454eb76a4c21032","-n1","--pretty=%H"]
+[2015-12-27 00:15:58.168515] process done ExitSuccess
+[2015-12-27 00:15:58.168888] read: git ["--git-dir=../../.git","--work-tree=../..","--literal-pathspecs","log","refs/heads/git-annex..0357da603dc61b28814125650def17fb0e3f82ca","-n1","--pretty=%H"]
+[2015-12-27 00:15:58.174589] process done ExitSuccess
+[2015-12-27 00:15:58.1749] read: git ["--git-dir=../../.git","--work-tree=../..","--literal-pathspecs","log","refs/heads/git-annex..4673c36870f0a536b4bedc3f5826f2d6b39c9d6a","-n1","--pretty=%H"]
+[2015-12-27 00:15:58.214221] process done ExitSuccess
+[2015-12-27 00:15:58.215418] read: git ["--git-dir=../../.git","--work-tree=../..","--literal-pathspecs","log","refs/heads/git-annex..75de80c0acf77e04e5b2b4c6f7947aec75603989","-n1","--pretty=%H"]
+[2015-12-27 00:15:58.243546] process done ExitSuccess
+[2015-12-27 00:15:58.244412] chat: git ["--git-dir=../../.git","--work-tree=../..","--literal-pathspecs","cat-file","--batch"]
+[2015-12-27 00:15:58.250349] read: git ["config","--null","--list"]
+[2015-12-27 00:15:58.250859] read: git ["config","--null","--list"]
+[2015-12-27 00:15:58.253927] process done ExitSuccess
+[2015-12-27 00:15:58.254538] chat: git ["--git-dir=../../.git","--work-tree=../..","--literal-pathspecs","check-attr","-z","--stdin","annex.backend","annex.numcopies","--"]
+[2015-12-27 00:15:58.255324] read: git ["--version"]
+[2015-12-27 00:15:58.257803] process done ExitSuccess
+drop stratohidrive 20150816_cccamp/DSC04537.ARW [2015-12-27 00:15:58.328068] chat: gpg ["--batch","--no-tty","--use-agent","--quiet","--trust-model","always","--decrypt"]
+[2015-12-27 00:15:58.354229] process done ExitSuccess
+failed
+git-annex: drop: 1 failed
+# End of transcript or log.
+"""]]
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+
+As mentioned before, I am very, very happy with git-annex :-) Discovery of 2015 for me.
diff --git a/doc/forum/Create_new_remote_from_copied_repository__63__.mdwn b/doc/forum/Create_new_remote_from_copied_repository__63__.mdwn
new file mode 100644
index 000000000..af6f65324
--- /dev/null
+++ b/doc/forum/Create_new_remote_from_copied_repository__63__.mdwn
@@ -0,0 +1,3 @@
+Newbie question. I decided to use git-annex for backing up my raw photo archive to a remote server. I was a bit pressed for time when I did this and did not really have time to learn about git-annex enough to do this in the correct way. I ended up creating a local repository with all my photos and then I simply copied this (using the ordinary system copy command) to an external drive that is now connected to a raspberry pi at my parents house (in another city) which I connect to through ssh. So the local repository and the "remote" are identical with the same name, description and uuid and everything. Is it possible to convert the "remote" repository (clone) to a "proper" git-annex remote? Can I just change the name or reinitialize the remote or something like that? This would definitely save a lot of time and hazzle since all the data is already there. Or am I better off just biting the bullet and create a new remote from scratch and sync the content from my local repository (600 Gb) over the Internet?   
+Thanks!  
+/David
diff --git a/doc/forum/Future_proofing___47___disaster_recovery_with_an_encrypted_special_remote/comment_6_ab0b660e5a3d7fc7d3d3a4b460022da5._comment b/doc/forum/Future_proofing___47___disaster_recovery_with_an_encrypted_special_remote/comment_6_ab0b660e5a3d7fc7d3d3a4b460022da5._comment
new file mode 100644
index 000000000..eda99e459
--- /dev/null
+++ b/doc/forum/Future_proofing___47___disaster_recovery_with_an_encrypted_special_remote/comment_6_ab0b660e5a3d7fc7d3d3a4b460022da5._comment
@@ -0,0 +1,139 @@
+[[!comment format=mdwn
+ username="wsha.code+ga@b38779424f41c5701bbe5937340be43ff1474b2d"
+ subject="comment 6"
+ date="2015-12-26T12:31:13Z"
+ content="""
+joey, thanks for answering my questions. I have put together a bash script to check my understanding of how the different encryption methods work. The script is below. With it, I am able to generate the encrypted keys for items for `hybrid` and `shared` encrypted `directory` remotes and to decrypt items for `hybrid`, `shared`, and `pubkey` encrypted `directory` remotes (and presumably other special remotes but I tested with `directory` remotes since those were simplest). The only thing I have not been able to get to work is the generation of the encrypted item keys for `pubkey` remotes. Do you see what I am doing wrong for that case? Once I figure that out, I can make a tip entry on the wiki with the script.
+
+	#!/usr/bin/env bash
+
+	# args: file_path remote decrypt decrypt_path
+
+	# For shared, pubkey, and hybrid encryption:
+	# 1. Take in symlink and output encrypted key
+	# 2. Take in path to encrypted file and print decrypted file to stdout
+	#
+	# args: -r remote [-k symlink] [-d encrypted_file_path]
+
+	usage() {
+		echo \"Usage: ga_decrypt.sh -r REMOTE [-k SYMLINK] [-d FILE]\"
+		echo \"\"
+		echo \"    Either lookups up key on REMOTE for annex file linked with SYMLINK\"
+		echo \"    or decrypts FILE encrypted for REMOTE.\"
+		echo \"\"
+		echo \"    -r: REMOTE is special remote to use\"
+		echo \"    -k: SYMLINK is symlink in annex to print encrypted special remote key for\"
+		echo \"    -d: FILE is path to special remote file to decrypt to STDOUT\"
+		echo \"\"
+		echo \"NOTES: \"
+		echo \"    * Run in an indirect git annex repo.\"
+		echo \"    * Must specify -k or -d.\"
+		echo \"    * -k prints the key including the leading directory names used for a \"
+		echo \"       directory remote (even if REMOTE is not a directory remote)\"
+		echo \"    * -d works on a locally accessible file. It does not fetch a remote file\"
+		echo \"    * Must have gpg and openssl\"
+	}
+
+	decrypt_cipher() {
+		cipher=\"$1\"
+		echo \"$(echo -n \"$cipher\" | base64 -d | gpg --decrypt --quiet)\"
+	}
+
+	lookup_key() {
+		encryption=\"$1\"
+		cipher=\"$2\"
+		symlink=\"$3\"
+
+		if [ \"$encryption\" == \"hybrid\" ] || [ \"$encryption\" == \"pubkey\" ]; then
+			cipher=\"$(decrypt_cipher \"$cipher\")\"
+		fi
+
+		# Pull out MAC cipher from beginning of cipher
+		if [ \"$encryption\" = \"hybrid\" ] ; then
+			cipher=\"$(echo -n \"$cipher\" | head  -c 256 )\"
+		elif [ \"$encryption\" = \"shared\" ] ; then
+			cipher=\"$(echo -n \"$cipher\" | base64 -d | tr -d '\n' | head  -c 256 )\"
+		elif [ \"$encryption\" = \"pubkey\" ] ; then
+			# ???
+			: # Use entire base64 decrypted cipher
+			# cipher=\"$(echo -n \"$cipher\" | head  -c 256 )\"
+			# cipher=\"$(echo -n \"$cipher\" | base64 -d | tr -d '\n' | head  -c 256 )\"
+			# cipher=\"$(echo -n \"$cipher\" | base64 -d | tr -d '\n' )\"
+		fi
+
+		annex_key=\"$(basename \"$(readlink \"$symlink\")\")\"
+		hash=\"$(echo -n \"$annex_key\" | openssl dgst -sha1 -hmac \"$cipher\" | sed 's/(stdin)= //')\"
+		key=\"GPGHMACSHA1--$hash\"
+		checksum=\"$(echo -n $key | md5sum)\"
+		echo \"${checksum:0:3}/${checksum:3:3}/$key\"
+	}
+
+	decrypt_file() {
+		encryption=\"$1\"
+		cipher=\"$2\"
+		file_path=\"$3\"
+
+		if [ \"$encryption\" = \"pubkey\" ] ; then
+			gpg --quiet --decrypt \"${file_path}\"
+		else
+			if [ \"$encryption\" = \"hybrid\" ] ; then
+				cipher=\"$(decrypt_cipher \"$cipher\" | tail -c +257)\"
+			elif [ \"$encryption\" = \"shared\" ] ; then
+				cipher=\"$(echo -n \"$cipher\" | base64 -d | tr -d '\n' | tail  -c +257 )\"
+			fi
+			gpg --quiet --batch --passphrase \"$cipher\" --output - \"${file_path}\"
+		fi
+	}
+
+	main() {
+		OPTIND=1
+
+		mode=\"\"
+		remote=\"\"
+
+		while getopts \"r:k:d:\" opt; do
+			case \"$opt\" in
+				r)  remote=\"$OPTARG\"
+					;;
+				k)  if [ -z \"$mode\" ] ; then
+						mode=\"lookup key\"
+					else
+						usage
+						exit 2
+					fi
+					symlink=\"$OPTARG\"
+					;;
+				d)  if [ -z \"$mode\" ] ; then
+						mode=\"decrypt file\"
+					else
+						usage
+						exit 2
+					fi
+					file_path=\"$OPTARG\"
+					;;
+			esac
+		done
+
+		if [ -z \"$mode\" ] || [ -z \"$remote\" ] ; then
+			usage
+			exit 2
+		fi
+
+		shift $((OPTIND-1))
+
+		# Pull out config for desired remote name
+		remote_config=\"$(git show git-annex:remote.log | grep 'name='\"$remote \")\"
+
+		# Get encryption type and cipher from config
+		encryption=\"$(echo \"$remote_config\" | grep -oP 'encryption\=.*? ' | tr -d ' \n' | sed 's/encryption=//')\"
+		cipher=\"$(echo \"$remote_config\" | grep -oP 'cipher\=.*? ' | tr -d ' \n' | sed 's/cipher=//')\"
+
+		if [ \"$mode\" = \"lookup key\" ] ; then
+			lookup_key \"$encryption\" \"$cipher\" \"$symlink\"
+		elif [ \"$mode\" = \"decrypt file\" ] ; then
+			decrypt_file \"$encryption\" \"$cipher\" \"${file_path}\"
+		fi
+	}
+
+	main \"$@\"
+"""]]
diff --git a/doc/forum/pushing_to_android.mdwn b/doc/forum/pushing_to_android.mdwn
new file mode 100644
index 000000000..030ca5c79
--- /dev/null
+++ b/doc/forum/pushing_to_android.mdwn
@@ -0,0 +1 @@
+Dear git-annex community, I've just installed git-annex on my CyanogenMod phone. What I'd like to do is to synchronize my music folder from my desktop to my phone. I.e. I'm using [sshelper](http://arachnoid.com/android/SSHelper/index.html) to start OpenSSH server on the phone. The question is, how can I make git and git-annex available to SSHelper's shell so that I can git push to the phone?