diff options
author | 2015-09-17 17:20:01 -0400 | |
---|---|---|
committer | 2015-09-17 17:20:01 -0400 | |
commit | 61421a4a686eef076b0592384a3ce6e8573bd04e (patch) | |
tree | e4bcd9cd4c82c7b559eee7ddf0159ebfb799f99e | |
parent | 253f1b2b6c988b1b2148b9d41e4de5bedaa73fba (diff) |
S3 storage classes expansion
Added support for storageclass=STANDARD_IA to use Amazon's
new Infrequently Accessed storage.
Also allows using storageclass=NEARLINE to use Google's NearLine storage.
The necessary changes to aws to support this are in
https://github.com/aristidb/aws/pull/176
-rw-r--r-- | Remote/S3.hs | 4 | ||||
-rw-r--r-- | debian/changelog | 5 | ||||
-rw-r--r-- | doc/bugs/Support_non-default_storage_classes_with_Google_Cloud_Storage/comment_1_a10aaa758daee3ca0b064c60c0382ce8._comment | 18 | ||||
-rw-r--r-- | doc/special_remotes/S3.mdwn | 16 | ||||
-rw-r--r-- | git-annex.cabal | 2 |
5 files changed, 39 insertions, 6 deletions
diff --git a/Remote/S3.hs b/Remote/S3.hs index fe407f204..5c5a43661 100644 --- a/Remote/S3.hs +++ b/Remote/S3.hs @@ -499,8 +499,8 @@ getBucketName = map toLower <$$> M.lookup "bucket" getStorageClass :: RemoteConfig -> S3.StorageClass getStorageClass c = case M.lookup "storageclass" c of - Just "REDUCED_REDUNDANCY" -> S3.ReducedRedundancy - _ -> S3.Standard + Just s -> S3.OtherStorageClass (T.pack s) + Nothing -> S3.Standard getPartSize :: RemoteConfig -> Maybe Integer getPartSize c = readSize dataUnits =<< M.lookup "partsize" c diff --git a/debian/changelog b/debian/changelog index 0ded62f12..e55a52a66 100644 --- a/debian/changelog +++ b/debian/changelog @@ -2,6 +2,11 @@ git-annex (5.20150917) UNRELEASED; urgency=medium * info: Don't allow use in a non-git-annex repository, since it uses the git-annex branch and would create it if it were missing. + * S3: Added support for storageclass=STANDARD_IA to use Amazon's + new Infrequently Accessed storage. + (Needs aws-0.13.0.) + * S3: Use aws-0.13.0, which allows using storageclass=NEARLINE + to use Google's NearLine storage. -- Joey Hess <id@joeyh.name> Wed, 16 Sep 2015 12:23:33 -0400 diff --git a/doc/bugs/Support_non-default_storage_classes_with_Google_Cloud_Storage/comment_1_a10aaa758daee3ca0b064c60c0382ce8._comment b/doc/bugs/Support_non-default_storage_classes_with_Google_Cloud_Storage/comment_1_a10aaa758daee3ca0b064c60c0382ce8._comment new file mode 100644 index 000000000..bc4d5e95c --- /dev/null +++ b/doc/bugs/Support_non-default_storage_classes_with_Google_Cloud_Storage/comment_1_a10aaa758daee3ca0b064c60c0382ce8._comment @@ -0,0 +1,18 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 1""" + date="2015-09-17T21:09:24Z" + content=""" +It's now possible to use storageclass=NEARLINE, when git-annex is built +with aws-0.13.0. So, the approach of manually creating the bucket with the +desired storage class should work now. + +I'm unsure if the first method, of letting git-annex create the bucket, +will work now. Can you test? It may work now too with +storageclass=NEARLINE. While no storage class is currently specified when +creating the bucket (that's not in the S3 api at all); but once the bucket +exists, with whatever storage class is default, git-annex will specify +NEARLINE when storing objects in it. Seems a good chance this will work, +and it'd be easier than extending the aws library with google-specific +features. +"""]] diff --git a/doc/special_remotes/S3.mdwn b/doc/special_remotes/S3.mdwn index 33f0410bb..f4d1c552e 100644 --- a/doc/special_remotes/S3.mdwn +++ b/doc/special_remotes/S3.mdwn @@ -37,9 +37,19 @@ the S3 remote. "us-west-1", "us-west-2", "ap-southeast-1", "ap-southeast-2", and "sa-east-1". -* `storageclass` - Default is "STANDARD". If you have configured git-annex - to preserve multiple [[copies]], consider setting this to "REDUCED_REDUNDANCY" - to save money. +* `storageclass` - Default is "STANDARD". + Consult S3 provider documentation for pricing details and available + storage classes. + + When using Amazon S3, if you have configured git-annex to preserve + multiple [[copies]], consider setting this to "REDUCED_REDUNDANCY" + to save money. Or, if the remote will be used for backup or archival, + and so its files are Infrequently Accessed, "STANDARD_IA" is also a + good choice to save money. + + Note that changing the storage class of an existing S3 remote will + affect new objects sent to the remote, but not objects already + stored there. * `host` and `port` - Specify in order to use a different, S3 compatable service. diff --git a/git-annex.cabal b/git-annex.cabal index 27c2675f8..fcab1b7b1 100644 --- a/git-annex.cabal +++ b/git-annex.cabal @@ -178,7 +178,7 @@ Executable git-annex Build-Depends: regex-compat if flag(S3) - Build-Depends: conduit, conduit-extra, aws (>= 0.9.2), http-client + Build-Depends: conduit, conduit-extra, aws (>= 0.13.0), http-client CPP-Options: -DWITH_S3 if flag(WebDAV) |