diff options
| -rw-r--r-- | Remote/S3.hs | 4 | ||||
| -rw-r--r-- | doc/backends.mdwn | 7 | ||||
| -rw-r--r-- | doc/special_remotes.mdwn | 38 | ||||
| -rw-r--r-- | doc/walkthrough.mdwn | 1 | ||||
| -rw-r--r-- | doc/walkthrough/moving_file_content_between_repositories.mdwn | 4 | ||||
| -rw-r--r-- | doc/walkthrough/using_Amazon_S3.mdwn | 15 |
6 files changed, 65 insertions, 4 deletions
diff --git a/Remote/S3.hs b/Remote/S3.hs index 818cde203..bc010bf0b 100644 --- a/Remote/S3.hs +++ b/Remote/S3.hs @@ -45,7 +45,7 @@ genRemote r = do hasKeyCheap = False } -{- S3 remotes have a remote.<name>.annex-s3bucket config setting. +{- S3 remotes have a remote.<name>.annex-s3-bucket config setting. - Git.Repo does not normally generate remotes for things that - have no configured url, so the Git.Repo objects have to be - constructed as coming from an unknown location. -} @@ -55,7 +55,7 @@ findS3Remotes r = map construct remotepairs remotepairs = Map.toList $ filterremotes $ Git.configMap r filterremotes = Map.filterWithKey (\k _ -> s3remote k) construct (k,_) = Git.repoRemoteNameSet Git.repoFromUnknown k - s3remote k = startswith "remote." k && endswith ".annex-s3bucket" k + s3remote k = startswith "remote." k && endswith ".annex-s3-bucket" k tryS3ConfigRead :: Git.Repo -> Annex Git.Repo tryS3ConfigRead r = error "TODO" diff --git a/doc/backends.mdwn b/doc/backends.mdwn index 22164850a..b0a2c882a 100644 --- a/doc/backends.mdwn +++ b/doc/backends.mdwn @@ -9,6 +9,10 @@ to retrieve the file's content (its value). Multiple pluggable backends are supported, and a single repository can use different backends for different files. +These backends can transfer file contents in configured git remotes. +It's also possible to use [[special_remotes]], such as Amazon S3 with +these backends. + * `WORM` ("Write Once, Read Many") This backend assumes that any file with the same basename, size, and modification time has the same content. So with this backend, files can be moved around, but should never be added to @@ -19,6 +23,9 @@ can use different backends for different files. * `SHA512`, `SHA384`, `SHA256`, `SHA224` -- Like SHA1, but larger checksums. Mostly useful for the very paranoid, or anyone who is researching checksum collisions and wants to annex their colliding data. ;) + +These backends store file contents in other key/value stores. + * `URL` -- This backend downloads the file's content from an external URL. The `annex.backends` git-config setting can be used to list the backends diff --git a/doc/special_remotes.mdwn b/doc/special_remotes.mdwn new file mode 100644 index 000000000..a62d55f5b --- /dev/null +++ b/doc/special_remotes.mdwn @@ -0,0 +1,38 @@ +Most [[backends]] can transfer data to and from configured git remotes. +Normally those remotes are normal git repositories (bare and non-bare), +that store the file contents in their own git annex directory. + +But, git-annex also extends git's concept of remotes, with these special +types of remotes. These can be used just like any normal remote by git-annex. +They cannot be used by other git commands though. + +## Amazon S3 + +Stores file contents in a bucket in Amazon S3 or a similar service. + +Example of configuring such a remote: + + git config remote.mys3.annex-s3-bucket myannex + export ANNEX_S3_ACCESS_KEY_ID="08TJMT99S3511WOZEP91" + export ANNEX_S3_SECRET_ACCESS_KEY="s3kr1t" + +That creates a remote named "mys3" using the bucket named "myannex", +which will be created if it doesn't already exist. + +Here is the full set of configurable settings for Amazon S3. +Each setting can be configured on a per-remote basis in git-config, +or globally in an environment variable. + +* `remote.$name.annex-s3-secret-access-key` `ANNEX_S3_SECRET_ACCESS_KEY` + Your S3 password. Usually stored in the environment variable + to avoid it being exposed. +* `remote.$name.annex-s3-access-key-id` `ANNEX_S3_ACCESS_KEY_ID` + Your S3 access key. For example, "". Does not need to be kept + private. +* `remote.$name.annex-s3-host` `ANNEX_S3_HOST` + Host to connect to. Default is s3.amazonaws.com. +* `remote.$name.annex-s3-port` `ANNEX_S3_PORT` + Port to connect to. Default is 80. +* `remote.$name.annex-s3-datacenter` `ANNEX_S3_DATACENTER` + Name of the datacenter to use. Default is "US"; + other valid values include "EU", "us-west-1", and "ap-southeast-1". diff --git a/doc/walkthrough.mdwn b/doc/walkthrough.mdwn index 3b4f7d56a..53f0be6bb 100644 --- a/doc/walkthrough.mdwn +++ b/doc/walkthrough.mdwn @@ -14,6 +14,7 @@ A walkthrough of the basic features of git-annex. modifying_annexed_files using_ssh_remotes moving_file_content_between_repositories + using_Amazon_S3 using_the_URL_backend using_the_SHA1_backend migrating_data_to_a_new_backend diff --git a/doc/walkthrough/moving_file_content_between_repositories.mdwn b/doc/walkthrough/moving_file_content_between_repositories.mdwn index 6b3e3f4e8..27dffe913 100644 --- a/doc/walkthrough/moving_file_content_between_repositories.mdwn +++ b/doc/walkthrough/moving_file_content_between_repositories.mdwn @@ -6,8 +6,8 @@ server to your laptop. Doing that by hand (by using `git annex get` and makes it very easy. # git annex move my_cool_big_file --to usbdrive - move my_cool_big_file (moving to usbdrive...) ok + move my_cool_big_file (to usbdrive...) ok # git annex move video/hackity_hack_and_kaxxt.mov --from fileserver - move video/hackity_hack_and_kaxxt.mov (moving from fileserver...) + move video/hackity_hack_and_kaxxt.mov (from fileserver...) WORM-s86050597-m1274316523--hackity_hack_and_kax 100% 82MB 199.1KB/s 07:02 ok diff --git a/doc/walkthrough/using_Amazon_S3.mdwn b/doc/walkthrough/using_Amazon_S3.mdwn new file mode 100644 index 000000000..d7222731b --- /dev/null +++ b/doc/walkthrough/using_Amazon_S3.mdwn @@ -0,0 +1,15 @@ +git-annex extends git's usual remotes with some [[special_remotes]], that +are not git repositories. This way you can set up a remote using say, +Amazon S3, and use git-annex to transfer files into the cloud. + + # git config remote.mys3.annex-s3-bucket myannex + # export ANNEX_S3_ACCESS_KEY_ID="08TJMT99S3511WOZEP91" + # export ANNEX_S3_SECRET_ACCESS_KEY="s3kr1t" + # git annex copy my_cool_big_file --to mys3 + copy my_cool_big_file (to mys3...) ok + # git annex move video/hackity_hack_and_kaxxt.mov --to mys3 + move video/hackity_hack_and_kaxxt.mov (to mys3...) ok + +An Amazon S3 remote works just like a ssh remote, except it does not have +a git repository at the other end, and it costs you money. :) For full +details about setting them up, see [[special_remotes]]. |