external special remote documentation and example script

3 files changed, 160 insertions, 3 deletions

diff --git a/doc/special_remotes/external.mdwn b/doc/special_remotes/external.mdwn
new file mode 100644
index 000000000..5d150e583
--- /dev/null
+++ b/doc/special_remotes/external.mdwn
@@ -0,0 +1,20 @@
+There are three ways to implement a new special remote:
+
+1. Using the [[hook]] special remote to tell git-annex what commands
+   to run to store and retrieve data. This is the easiest way, and
+   is great for prototyping.
+2. Writing it in Haskell and adding it to git-annex.
+3. Writing a program in any language you like that speaks the
+   [[external_special_remote_protocol]].
+
+This page is all about writing new external special remotes. It's not hard!
+
+* All you need is to make a program with a name like `git-annex-remote-$bar`.
+* Install it in PATH.
+* When the user runs `git annex initremote foo type=external externaltype=$bar`,
+  it will use your program.
+
+Here's a simple shell script example, which can easily be adapted
+to run whatever commands you need. ([[download|example.sh]])
+
+[[!inline raw=yes pages="special_remotes/external/example.sh"]]
diff --git a/doc/special_remotes/external/example.sh b/doc/special_remotes/external/example.sh
new file mode 100755
index 000000000..07dcb2705
--- /dev/null
+++ b/doc/special_remotes/external/example.sh
@@ -0,0 +1,131 @@
+#!/bin/sh
+# git-annex external special remote program
+# 
+# This is basically the same as git-annex's built-in directory special remote.
+# 
+# Install in PATH as git-annex-remote-directorya
+#
+# Copyright 2013 Joey Hess; licenced under the GNU GPL version 3 or higher.
+
+set -e
+
+# This program speaks a line-based protocol on stdin and stdout.
+# When running any commands, their stdout should be redirected to stderr
+# (or /dev/null) to avoid messing up the protocol.
+runcmd () {
+	"$@" >&2
+}
+
+# Gets a value from the remote's configuration, and stores it in RET
+getconfig () {
+	echo GETCONFIG "$1"
+	read resp
+	set -- $resp
+	case "$1" in
+		VALUE)
+			RET="$2"
+		;;
+		*)
+			RET=""
+		;;
+	esac
+}
+
+# Sets LOC to the location to use to store a key.
+mylocation () {
+	echo HASHDIR "$1"
+	read resp
+	set -- $resp
+	case "$1" in
+		VALUE)
+			LOC="$hashdir/$1"
+		;;
+		*)
+			LOC=
+		;;
+	esac
+}
+
+echo VERSION 1
+
+while read line; do
+	set -- $line
+	case "$1" in
+		INITREMOTE)
+			# XXX do anything necessary to create resources
+			# used by the remote. Try to be idempotent.
+			# Use GETCONFIG to get any needed configuration
+			# settings, and SETCONFIG to set any persistent
+			# configuration settings.
+			getconfig directory
+			mydirectory="$RET"
+			if [ -z "$mydirectory" ]; then
+				echo INITREMOTE-FAILURE "You need to set directory="
+			else
+				mkdir -p "$mydirectory"
+				echo INITREMOTE-SUCCESS
+			fi
+		;;
+		GETCOST)
+			echo COST-UNKNOWN
+		;;
+		PREPARE)
+			# XXX Use GETCONFIG to get configuration settings,
+			# and do anything needed to get ready for using the
+			# special remote here.
+			getconfig directory
+			mydirectory="$RET"
+		;;
+		TRANSFER)
+			key="$3"
+			file="$4"
+			case "$2" in
+				STORE)
+					# XXX upload file here
+					# XXX when possible, send PROGRESS
+					calclocation "$key"
+					mkdir -p "$(dirname "$LOC")"
+					runcmd cp -v "$file" "$LOC"
+					echo TRANSFER-SUCCESS STORE "$key"
+				;;
+				RETRIEVE)
+					# XXX download file here
+					calclocation "$key"
+					runcmd cp -v "$LOC" "$file"
+					echo TRANSFER-SUCCESS RETRIEVE "$key"
+				;;
+			esac
+		;;
+		CHECKPRESENT)
+			key="$2"
+			calclocation "$key"
+			if [ -e "$LOC" ]; then
+				echo CHECKPRESENT-SUCCESS "$key"
+			else
+				if [ -d "$mydirectory" ]; then
+					echo CHECKPRESENT-FAILURE "$key"
+				else
+					# If the directory does not exist,
+					# the remote is not available.
+					# (A network remote would similarly
+					# fail with CHECKPRESENT-UNKNOWN
+					# if it couldn't be contacted).
+					echo CHECKPRESENT-UNKNOWN "$key" "this remote is not currently available"
+				fi
+			fi
+		;;
+		REMOVE)
+			key="$2"
+			calclocation "$key"
+			# Note that it's not a failure to remove a
+			# key that is not present, so -f is used.
+			runcmd rm -f "$LOC"
+			echo REMOVE-SUCCESS "$key"
+		;;
+		*)
+			echo UNKNOWN-REQUEST
+		;;
+	esac	
+done
+
+# XXX anything that needs to be done at shutdown can be done here
diff --git a/doc/special_remotes/hook.mdwn b/doc/special_remotes/hook.mdwn
index eaea940a7..8cf31ed02 100644
--- a/doc/special_remotes/hook.mdwn
+++ b/doc/special_remotes/hook.mdwn
@@ -1,9 +1,12 @@
 This special remote type lets you store content in a remote of your own
-devising.
+devising, configured via some simple hooks.
 
 It's not recommended to use this remote type when another like [[rsync]]
 or [[directory]] will do. If your hooks are not carefully written, data
-could be lost.
+could be lost. 
+
+If you're building a special remote for others to use,
+instead consider building an [[external_special_remote|external]].
 
 ## example
 
@@ -68,6 +71,9 @@ The settings to use in git config for the hook commands are as follows:
 
 ## combined hook program
 
+This interface is deprecated -- it's better, and not much harder,
+to write an [[external_special_remote|external]]!
+
 Rather than setting all of the above hooks, you can write a single
 program that handles everything, and set a single hook to make it be used.
 
@@ -75,7 +81,7 @@ program that handles everything, and set a single hook to make it be used.
 	# git annex initremote mydemorepo type=hook hooktype=demo encryption=none
 
 The program just needs to look at the `ANNEX_ACTION` environment variable
-to see what it's being asked to do  For example:
+to see what it's being asked to do. For example:
 
 [[!format sh """
 #!/bin/sh