diff options
author | 2014-05-15 19:44:00 +0100 | |
---|---|---|
committer | 2014-05-15 16:32:44 -0400 | |
commit | 9914df2d424beb62a59bd3c82ca57b2d2f146e7c (patch) | |
tree | 920d16f734b11d52ce8b483195a64dc110c77097 /doc | |
parent | 1991541d1e4fdaa205bd8bdda31bb10ab4b49aef (diff) |
ddar special remote
Diffstat (limited to 'doc')
-rw-r--r-- | doc/git-annex.mdwn | 6 | ||||
-rw-r--r-- | doc/special_remotes.mdwn | 1 | ||||
-rw-r--r-- | doc/special_remotes/ddar.mdwn | 40 | ||||
-rw-r--r-- | doc/walkthrough/using_ddar.mdwn | 32 |
4 files changed, 79 insertions, 0 deletions
diff --git a/doc/git-annex.mdwn b/doc/git-annex.mdwn index d5408a2ae..bb741810c 100644 --- a/doc/git-annex.mdwn +++ b/doc/git-annex.mdwn @@ -1647,6 +1647,12 @@ Here are all the supported configuration settings. the location of the bup repository to use. Normally this is automatically set up by `git annex initremote`, but you can change it if needed. +* `remote.<name>.ddarrepo` + + Used by ddar special remotes, this configures + the location of the ddar repository to use. Normally this is automatically + set up by `git annex initremote`, but you can change it if needed. + * `remote.<name>.directory` Used by directory special remotes, this configures diff --git a/doc/special_remotes.mdwn b/doc/special_remotes.mdwn index e3004d2cf..cc52dfc55 100644 --- a/doc/special_remotes.mdwn +++ b/doc/special_remotes.mdwn @@ -10,6 +10,7 @@ They cannot be used by other git commands though. * [[S3]] (Amazon S3, and other compatible services) * [[Amazon_Glacier|glacier]] * [[bup]] +* [[ddar]] * [[gcrypt]] (encrypted git repositories!) * [[directory]] * [[rsync]] diff --git a/doc/special_remotes/ddar.mdwn b/doc/special_remotes/ddar.mdwn new file mode 100644 index 000000000..8de86e2c7 --- /dev/null +++ b/doc/special_remotes/ddar.mdwn @@ -0,0 +1,40 @@ +This special remote type stores file contents in a +[ddar](https://github.com/basak/ddar) repository. This provides easy +de-duplication when you use git-annex to manage many files that are similar. + +Unlike bup, ddar uses its own storage format, which allows for both creation +and deletion of de-deduplicated files. In addition to using local storage, ddar +archives can be remote, providing that ddar is installed on the remote machine +and ssh is available to it. + +See [[walkthrough/using_ddar]] for usage examples. + +## encryption + +Encryption is nominally supported, but is not useful. Since effective +encryption necessarily obfuscates file contents, similar areas across different +files are no longer visible to ddar and cannot be de-duplicated. + +## compression + +The same caveat with encryption also generally applies to compression, since +file compression changes file contents such that similar regions across files +no longer appear similar. An exception is `gzip --rsyncable`, which is +specifically designed to work around this issue. This is the only compression +mechanism with which de-duplication remains effective. + +## configuration + +These parameters can be passed to `git annex initremote` to configure ddar: + +* `encryption` - One of "none", "hybrid", "shared", or "pubkey". + See [[encryption]]. However, note that encryption renders all de-duplication +ineffective. + +* `keyid` - Specifies the gpg key to use for [[encryption]]. + +* `ddarrepo` - Required. This is passed to `ddar` as the path to the ddar + archive to use. If it doesn't exist, the ddar repository will be created + automatically when a file is first copied to it. To use a remote ddar + repository, use a colon (`:`) to separate the hostname from the remote path. + Example: "ddarrepo=example.com:/big/myddar" or "ddarrepo=/big/myddar" diff --git a/doc/walkthrough/using_ddar.mdwn b/doc/walkthrough/using_ddar.mdwn new file mode 100644 index 000000000..1d7cdd749 --- /dev/null +++ b/doc/walkthrough/using_ddar.mdwn @@ -0,0 +1,32 @@ +Another [[special_remote|special_remotes]] that git-annex can use is +a [[special_remotes/ddar]] repository. ddar stores large file contents +in a directory structure of its own, with deduplication. For remote +repositories, ddar requires that ssh is available on the remote, with ddar also +installed remotely. When copying files to the remote, ddar only needs to send +over the network the parts of the files that are not already present remotely. + +Unlike bup, ddar uses its own storage format, which allows for both creation +and deletion of de-deduplicated files. + +Here's how to create a ddar remote, and describe it. + +[[!template id=note text=""" +Instead of specifying a remote system, you could choose to make a bup +remote that is only accessible on the current system, by passing +"ddarrepo=/big/myddar". +"""]] + + # git annex initremote myddar type=ddar encryption=none ddarrepo=example.com:/big/myddar + initremote ddar (bup init) + Initialized empty Git repository in /big/myddar/ + ok + # git annex describe myddar "my bup repository at example.com" + describe myddar ok + +Now the remote can be used like any other remote. + + # git annex move my_cool_big_file --to myddar + move my_cool_big_file (to myddar...) + ok + +See [[special_remotes/bup]] for details. |