| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
|
| |
instead of the default DNS-style access.
untested
|
| |
|
|
|
|
| |
* Fix failure to build with aws-0.13.0.
* When built with aws-0.13.0, the S3 special remote can be used to create
google nearline buckets, by setting storageclass=NEARLINE.
|
| |
|
|
|
|
|
|
|
|
| |
Added support for storageclass=STANDARD_IA to use Amazon's
new Infrequently Accessed storage.
Also allows using storageclass=NEARLINE to use Google's NearLine storage.
The necessary changes to aws to support this are in
https://github.com/aristidb/aws/pull/176
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
| |
In my tests, this has to be set when uploading a file to the bucket
and then the file can be accessed using the bucketname.s3.amazonaws.com
url.
Setting it when creating the bucket didn't seem to make the whole bucket
public, or allow accessing files stored in it. But I have gone ahead and
also sent it when creating the bucket just in case that is needed in some
case.
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
The minimum allowsed size actually refers to the part size!
|
| |
|
|
|
|
| |
This reverts commit 2ba5af49c94b97c586220c3553367988ef095934.
I misunderstood the cause of the problem.
|
| |
|
|
|
|
|
|
|
| |
When uploading the last part of a file, which was 640229 bytes, S3 rejected
that part: "Your proposed upload is smaller than the minimum allowed size"
I don't know what the minimum is, but the fix is just to include the last
part into the previous part. Since this can result in a part that's
double-sized, use half-sized parts normally.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
I'm a little stuck on getting the list of etags of the parts.
This seems to require taking the md5 of each part locally,
which doesn't get along well with lazily streaming in the part from the
file. It would need to read the file twice, or lose laziness and buffer a
whole part -- but parts might be quite large.
This seems to be a problem with the API provided; S3 is supposed to return
an etag, but that is not exposed. I have filed a bug:
https://github.com/aristidb/aws/issues/141
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently, initremote works, but not the other operations. They should be
fairly easy to add from this base.
Also, https://github.com/aristidb/aws/issues/119 blocks internet archive
support.
Note that since http-conduit is used, this also adds https support to S3.
Although git-annex encrypts everything anyway, so that may not be extremely
useful. It is not enabled by default, because existing S3 special remotes
have port=80 in their config. Setting port=443 will enable it.
This commit was sponsored by Daniel Brockman.
|
| |
|
|
|
|
|
| |
The assistant defaults to 1MiB chunk size for new S3 special remotes.
Which will work around a couple of bugs:
http://git-annex.branchable.com/bugs/S3_memory_leaks/
http://git-annex.branchable.com/bugs/S3_upload_not_using_multipart/
|
| | |
|
| |
|
|
| |
used. The initremote command now is used only to create new special remotes.
|
| |
|
|
| |
to the repository, by setting embedcreds=yes|no when running initremote.
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
| |
In particular, munge key filenames to comply with the IA's filename limits,
disable encryption, support their nonstandard way of creating buckets, and
allow x-amz-* headers to be specified in initremote to set item metadata.
Still TODO: initremote does not handle multiword metadata headers right.
|
| | |
|
|
|
Renamed Amazon_S3 page to just S3.
|
Joey Hess
anarcat
https://id.koumbit.net/anarcat
https://www.google.com/accounts/o8/id?id=AItOawnWvnTWY6LrcPB4BzYEBn5mRTpNhg5EtEg
Joey Hess
https://www.google.com/accounts/o8/id?id=AItOawnoUOqs_lbuWyZBqyU6unHgUduJwDDgiKY