ddar special remote

4 files changed, 79 insertions, 0 deletions

diff --git a/doc/git-annex.mdwn b/doc/git-annex.mdwn
index d5408a2ae..bb741810c 100644
--- a/doc/git-annex.mdwn
+++ b/doc/git-annex.mdwn
@@ -1647,6 +1647,12 @@ Here are all the supported configuration settings.
   the location of the bup repository to use. Normally this is automatically
   set up by `git annex initremote`, but you can change it if needed.
 
+* `remote.<name>.ddarrepo`
+
+  Used by ddar special remotes, this configures
+  the location of the ddar repository to use. Normally this is automatically
+  set up by `git annex initremote`, but you can change it if needed.
+
 * `remote.<name>.directory`
 
   Used by directory special remotes, this configures
diff --git a/doc/special_remotes.mdwn b/doc/special_remotes.mdwn
index e3004d2cf..cc52dfc55 100644
--- a/doc/special_remotes.mdwn
+++ b/doc/special_remotes.mdwn
@@ -10,6 +10,7 @@ They cannot be used by other git commands though.
 * [[S3]] (Amazon S3, and other compatible services)
 * [[Amazon_Glacier|glacier]]
 * [[bup]]
+* [[ddar]]
 * [[gcrypt]] (encrypted git repositories!)
 * [[directory]]
 * [[rsync]]
diff --git a/doc/special_remotes/ddar.mdwn b/doc/special_remotes/ddar.mdwn
new file mode 100644
index 000000000..8de86e2c7
--- /dev/null
+++ b/doc/special_remotes/ddar.mdwn
@@ -0,0 +1,40 @@
+This special remote type stores file contents in a
+[ddar](https://github.com/basak/ddar) repository. This provides easy
+de-duplication when you use git-annex to manage many files that are similar.
+
+Unlike bup, ddar uses its own storage format, which allows for both creation
+and deletion of de-deduplicated files. In addition to using local storage, ddar
+archives can be remote, providing that ddar is installed on the remote machine
+and ssh is available to it.
+
+See [[walkthrough/using_ddar]] for usage examples.
+
+## encryption
+
+Encryption is nominally supported, but is not useful. Since effective
+encryption necessarily obfuscates file contents, similar areas across different
+files are no longer visible to ddar and cannot be de-duplicated.
+
+## compression
+
+The same caveat with encryption also generally applies to compression, since
+file compression changes file contents such that similar regions across files
+no longer appear similar. An exception is `gzip --rsyncable`, which is
+specifically designed to work around this issue. This is the only compression
+mechanism with which de-duplication remains effective.
+
+## configuration
+
+These parameters can be passed to `git annex initremote` to configure ddar:
+
+* `encryption` - One of "none", "hybrid", "shared", or "pubkey".
+  See [[encryption]]. However, note that encryption renders all de-duplication
+ineffective.
+
+* `keyid` - Specifies the gpg key to use for [[encryption]].
+
+* `ddarrepo` - Required. This is passed to `ddar` as the path to the ddar
+  archive to use. If it doesn't exist, the ddar repository will be created
+  automatically when a file is first copied to it. To use a remote ddar
+  repository, use a colon (`:`) to separate the hostname from the remote path.
+  Example: "ddarrepo=example.com:/big/myddar" or "ddarrepo=/big/myddar"
diff --git a/doc/walkthrough/using_ddar.mdwn b/doc/walkthrough/using_ddar.mdwn
new file mode 100644
index 000000000..1d7cdd749
--- /dev/null
+++ b/doc/walkthrough/using_ddar.mdwn
@@ -0,0 +1,32 @@
+Another [[special_remote|special_remotes]] that git-annex can use is
+a [[special_remotes/ddar]] repository. ddar stores large file contents
+in a directory structure of its own, with deduplication. For remote
+repositories, ddar requires that ssh is available on the remote, with ddar also
+installed remotely. When copying files to the remote, ddar only needs to send
+over the network the parts of the files that are not already present remotely.
+
+Unlike bup, ddar uses its own storage format, which allows for both creation
+and deletion of de-deduplicated files.
+
+Here's how to create a ddar remote, and describe it.
+
+[[!template id=note text="""
+Instead of specifying a remote system, you could choose to make a bup
+remote that is only accessible on the current system, by passing
+"ddarrepo=/big/myddar".
+"""]]
+
+	# git annex initremote myddar type=ddar encryption=none ddarrepo=example.com:/big/myddar
+	initremote ddar (bup init)
+	Initialized empty Git repository in /big/myddar/
+	ok
+	# git annex describe myddar "my bup repository at example.com"
+	describe myddar ok
+
+Now the remote can be used like any other remote.
+
+	# git annex move my_cool_big_file --to myddar
+	move my_cool_big_file (to myddar...)
+	ok
+
+See [[special_remotes/bup]] for details.