summaryrefslogtreecommitdiff
path: root/Remote/S3.hs
Commit message (Collapse)AuthorAge
* plumb creds from webapp to initremoteGravatar Joey Hess2014-02-11
| | | | | Avoids abusing setting environment variables, which was always a hack and won't work on windows.
* add GETAVAILABILITY to external special remote protocolGravatar Joey Hess2014-01-13
| | | | | And some reworking of types, and added an annex-availability git config setting.
* assistant: Support repairing git remotes that are locally accessibleGravatar Joey Hess2013-10-27
| | | | | | | | (eg, on removable drives) gcrypt remotes are not yet handled. This commit was sponsored by Sören Brunk.
* S3: Try to ensure bucket name is valid for archive.org.Gravatar Joey Hess2013-10-16
|
* add remote fsck interfaceGravatar Joey Hess2013-10-11
| | | | | | | | | | | | | | | | | | | | Currently only implemented for local git remotes. May try to add support to git-annex-shell for ssh remotes later. Could concevably also be supported by some special remote, although that seems unlikely. Cronner user this when available, and when not falls back to fsck --fast --from remote git annex fsck --from does not itself use this interface. To do so, I would need to pass --fast and all other options that influence fsck on to the git annex fsck that it runs inside the remote. And that seems like a lot of work for a result that would be no better than cd remote; git annex fsck This may need to be revisited if git-annex-shell gets support, since it may be the case that the user cannot ssh to the server to run git-annex fsck there, but can run git-annex-shell there. This commit was sponsored by Damien Diederen.
* Support hot-swapping of removable drives containing gcrypt repositories.Gravatar Joey Hess2013-09-12
| | | | | | | | | | | To support this, a core.gcrypt-id is stored by git-annex inside the git config of a local gcrypt repository, when setting it up. That is compared with the remote's cached gcrypt-id. When different, a drive has been changed. git-annex then looks up the remote config for the uuid mapped from the core.gcrypt-id, and tweaks the configuration appropriately. When there is no known config for the uuid, it will refuse to use the remote.
* partially complete gcrypt remote (local send done; rest not)Gravatar Joey Hess2013-09-07
| | | | | | | | | | | | | | | | | | | | | | | | This is a git-remote-gcrypt encrypted special remote. Only sending files in to the remote works, and only for local repositories. Most of the work so far has involved making initremote work. A particular problem is that remote setup in this case needs to generate its own uuid, derivied from the gcrypt-id. That required some larger changes in the code to support. For ssh remotes, this will probably just reuse Remote.Rsync's code, so should be easy enough. And for downloading from a web remote, I will need to factor out the part of Remote.Git that does that. One particular thing that will need work is supporting hot-swapping a local gcrypt remote. I think it needs to store the gcrypt-id in the git config of the local remote, so that it can check it every time, and compare with the cached annex-uuid for the remote. If there is a mismatch, it can change both the cached annex-uuid and the gcrypt-id. That should work, and I laid some groundwork for it by already reading the remote's config when it's local. (Also needed for other reasons.) This commit was sponsored by Daniel Callahan.
* fix build (seems getGpgEncOpts got renamed to getGpgEncParams)Gravatar Joey Hess2013-09-04
|
* Allow public-key encryption of file content.Gravatar guilhem2013-09-03
| | | | | | | | | | | | With the initremote parameters "encryption=pubkey keyid=788A3F4C". /!\ Adding or removing a key has NO effect on files that have already been copied to the remote. Hence using keyid+= and keyid-= with such remotes should be used with care, and make little sense unless the point is to replace a (sub-)key by another. /!\ Also, a test case has been added to ensure that the cipher and file contents are encrypted as specified by the chosen encryption scheme.
* Store an annex-uuid file in the bucket when setting up a new S3 remote.Gravatar Joey Hess2013-04-27
|
* Automatically register public urls for files uploaded to the Internet Archive.Gravatar Joey Hess2013-04-25
|
* webapp: Display some additional information about a repository on its edit page.Gravatar Joey Hess2013-04-25
|
* S3: Dropping content from the Internet Archive doesn't work, but their API ↵Gravatar Joey Hess2013-04-25
| | | | indicates it does. Always refuse to drop from there.
* support enabling IA repositoriesGravatar Joey Hess2013-04-25
|
* connect existing meters to the transfer log for downloadsGravatar Joey Hess2013-04-11
| | | | | | | | | | | | | | Most remotes have meters in their implementations of retrieveKeyFile already. Simply hooking these up to the transfer log makes that information available. Easy peasy. This is particularly valuable information for encrypted remotes, which otherwise bypass the assistant's polling of temp files, and so don't have good progress bars yet. Still some work to do here (see progressbars.mdwn changes), but this is entirely an improvement from the lack of progress bars for encrypted downloads.
* webapp: Progess bar fixes for many types of special remotes.Gravatar Joey Hess2013-03-28
| | | | | | | | | | | | | There was confusion in different parts of the progress bar code about whether an update contained the total number of bytes transferred, or the number of bytes transferred since the last update. One way this bug showed up was progress bars that seemed to stick at zero for a long time. In order to fix it comprehensively, I add a new BytesProcessed data type, that is explicitly a total quantity of bytes, not a delta. Note that this doesn't necessarily fix every problem with progress bars. Particularly, buffering can now cause progress bars to seem to run ahead of transfers, reaching 100% when data is still being uploaded.
* add globallyAvailable to remotesGravatar Joey Hess2013-03-15
|
* split cost out into its own moduleGravatar Joey Hess2013-03-13
| | | | | Added a function to insert a new cost into a list, which could be used to asjust costs after a drag and drop.
* GnuPG options for symmetric encryption.Gravatar guilhem2013-03-11
|
* Special remotes now all rollback storage of keys that get modified during ↵Gravatar Joey Hess2013-01-09
| | | | the transfer, which can happen in direct mode.
* Fix transferring files to special remotes in direct mode.Gravatar Joey Hess2013-01-06
|
* type based git config handling for remotesGravatar Joey Hess2013-01-01
| | | | | Still a couple of places that use git config ad-hoc, but this is most of it done.
* whitespace fixesGravatar Joey Hess2012-12-13
|
* webapp: S3 and Glacier forms now have a select list of all ↵Gravatar Joey Hess2012-12-01
| | | | currently-supported AWS regions.
* avoid unnecessary MaybeGravatar Joey Hess2012-11-30
|
* webapp: Defaults to sharing box.com account info with friends, allowing ↵Gravatar Joey Hess2012-11-28
| | | | one-click enabling of the repository.
* Amazon Glacier special remote; 100% workingGravatar Joey Hess2012-11-20
|
* S3: Added progress display for uploading and downloading.Gravatar Joey Hess2012-11-18
|
* S3: upload progress displayGravatar Joey Hess2012-11-18
|
* better streaming while encrypting/decryptingGravatar Joey Hess2012-11-18
| | | | | | Both the directory and webdav special remotes used to have to buffer the whole file contents before it could be decrypted, as they read from chunks. Now the chunks are streamed through gpg with no buffering.
* webapp: support box.comGravatar Joey Hess2012-11-17
|
* encrypted webdav workingGravatar Joey Hess2012-11-16
|
* update to dav 0.1, and basic uploading is working!Gravatar Joey Hess2012-11-15
|
* factor out CredsGravatar Joey Hess2012-11-14
|
* where indentingGravatar Joey Hess2012-11-11
|
* change s3 creds cachingGravatar Joey Hess2012-09-26
| | | | | | | | Rather than store decrypted creds in the environment, store them in the creds cache file. This way, a single git-annex can have multiple S3 remotes using different creds.
* store S3 creds in a 600 mode file inside the local git repoGravatar Joey Hess2012-09-26
|
* unify typesGravatar Joey Hess2012-09-21
|
* add a progress callback to storeKey, and threaded it all the way throughGravatar Joey Hess2012-09-19
| | | | | | | | Transfer info files are updated when the callback is called, updating the number of bytes transferred. Left unused p variables at every place the callback should be used. Which is rather a lot..
* add support for readonly remotesGravatar Joey Hess2012-08-26
| | | | | | | Currently only the web special remote is readonly, but it'd be possible to also have readonly drives, or other remotes. These are handled in the assistant by only downloading from them, and never trying to upload to them.
* tweak field nameGravatar Joey Hess2012-08-26
|
* fix build warningGravatar Joey Hess2012-08-16
|
* Merge branch 'master' into assistantGravatar Joey Hess2012-08-16
|\ | | | | | | | | Conflicts: debian/changelog
| * S3: Add fileprefix setting.Gravatar Joey Hess2012-08-09
| |
* | add routes to pause/start/cancel transfersGravatar Joey Hess2012-08-08
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit includes a paydown on technical debt incurred two years ago, when I didn't know that it was bad to make custom Read and Show instances for types. As the routes need Read and Show for Transfer, which includes a Key, and deriving my own Read instance of key was not practical, I had to finally clean that up. So the compact Key read and show functions are now file2key and key2file, and Read and Show are now derived instances. Changed all code that used the old instances, compiler checked. (There were a few places, particularly in Command.Unused, and the test suite where the Show instance continue to be used for legitimate comparisons; ie show key_x == show key_y (though really in a bloom filter))
* | add a path field to remotesGravatar Joey Hess2012-07-22
|/ | | | | Also broke out some helper functions around constructing remotes, to be used later.
* record transfer information on local git remotesGravatar Joey Hess2012-07-01
| | | | | | | | | | | | | | | In order to record a semi-useful filename associated with the key, this required plumbing the filename all the way through to the remotes' storeKey and retrieveKeyFile. Note that there is potential for deadlock here, narrowly avoided. Suppose the repos are A and B. A sends file foo to B, and at the same time, B gets file foo from A. So, A locks its upload transfer info file, and then locks B's download transfer info file. At the same time, B is taking the two locks in the opposite order. This is only not a deadlock because the lock code does not wait, and aborts. So one of A or B's transfers will be aborted and the other transfer will continue. Whew!
* noopGravatar Joey Hess2012-04-21
|
* thread through original key to retrieveEnctyptedGravatar Joey Hess2012-03-04
| | | | | Allows showing progress bar for this last case of the directory special remote.
* Add progress bar display to the directory special remote.Gravatar Joey Hess2012-03-04
| | | | | | | So far I've only written progress bars for sending files, not yet receiving. No longer uses external cp at all. ByteString IO is fast enough.