summaryrefslogtreecommitdiff
path: root/Remote/Hook.hs
Commit message (Collapse)AuthorAge
* plumb creds from webapp to initremoteGravatar Joey Hess2014-02-11
| | | | | Avoids abusing setting environment variables, which was always a hack and won't work on windows.
* avoid needing a build-dep on hxt for Data.AssocListGravatar Joey Hess2014-01-14
|
* Fix a long-standing bug that could cause the wrong index file to be used ↵Gravatar Joey Hess2014-01-14
| | | | when committing to the git-annex branch, if GIT_INDEX_FILE is set in the environment. This typically resulted in git-annex branch log files being committed to the master branch and later showing up in the work tree. (These log files can be safely removed.)
* add GETAVAILABILITY to external special remote protocolGravatar Joey Hess2014-01-13
| | | | | And some reworking of types, and added an annex-availability git config setting.
* external special remotes mostly implemented (untested)Gravatar Joey Hess2013-12-26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This has not been tested at all. It compiles! The only known missing things are support for encryption, and for get/set of special remote configuration, and of key state. (The latter needs separate work to add a new per-key log file to store that state.) Only thing I don't much like is that initremote needs to be passed both type=external and externaltype=foo. It would be better to have just type=foo Most of this is quite straightforward code, that largely wrote itself given the types. The only tricky parts were: * Need to lock the remote when using it to eg make a request, because in theory git-annex could have multiple threads that each try to use a remote at the same time. I don't think that git-annex ever does that currently, but better safe than sorry. * Rather than starting up every external special remote program when git-annex starts, they are started only on demand, when first used. This will avoid slowdown, especially when running fast git-annex query commands. Once started, they keep running until git-annex stops, currently, which may not be ideal, but it's hard to know a better time to stop them. * Bit of a chicken and egg problem with caching the cost of the remote, because setting annex-cost in the git config needs the remote to already be set up. Managed to finesse that. This commit was sponsored by Lukas Anzinger.
* gcrypt, bup: Fix bug that prevented using these special remotes with ↵Gravatar Joey Hess2013-11-02
| | | | | | | | | encryption=pubkey. I think both of these are all that's affected, but I went ahead and fixed all the remotes that set their config to M.empty to instead store the actual config. Who knows what will expect it to be actually present in future, the Remote instance of getGpgEncParams came to..
* assistant: Support repairing git remotes that are locally accessibleGravatar Joey Hess2013-10-27
| | | | | | | | (eg, on removable drives) gcrypt remotes are not yet handled. This commit was sponsored by Sören Brunk.
* add remote fsck interfaceGravatar Joey Hess2013-10-11
| | | | | | | | | | | | | | | | | | | | Currently only implemented for local git remotes. May try to add support to git-annex-shell for ssh remotes later. Could concevably also be supported by some special remote, although that seems unlikely. Cronner user this when available, and when not falls back to fsck --fast --from remote git annex fsck --from does not itself use this interface. To do so, I would need to pass --fast and all other options that influence fsck on to the git annex fsck that it runs inside the remote. And that seems like a lot of work for a result that would be no better than cd remote; git annex fsck This may need to be revisited if git-annex-shell gets support, since it may be the case that the user cannot ssh to the server to run git-annex fsck there, but can run git-annex-shell there. This commit was sponsored by Damien Diederen.
* hlintGravatar Joey Hess2013-09-25
|
* Support hot-swapping of removable drives containing gcrypt repositories.Gravatar Joey Hess2013-09-12
| | | | | | | | | | | To support this, a core.gcrypt-id is stored by git-annex inside the git config of a local gcrypt repository, when setting it up. That is compared with the remote's cached gcrypt-id. When different, a drive has been changed. git-annex then looks up the remote config for the uuid mapped from the core.gcrypt-id, and tweaks the configuration appropriately. When there is no known config for the uuid, it will refuse to use the remote.
* partially complete gcrypt remote (local send done; rest not)Gravatar Joey Hess2013-09-07
| | | | | | | | | | | | | | | | | | | | | | | | This is a git-remote-gcrypt encrypted special remote. Only sending files in to the remote works, and only for local repositories. Most of the work so far has involved making initremote work. A particular problem is that remote setup in this case needs to generate its own uuid, derivied from the gcrypt-id. That required some larger changes in the code to support. For ssh remotes, this will probably just reuse Remote.Rsync's code, so should be easy enough. And for downloading from a web remote, I will need to factor out the part of Remote.Git that does that. One particular thing that will need work is supporting hot-swapping a local gcrypt remote. I think it needs to store the gcrypt-id in the git config of the local remote, so that it can check it every time, and compare with the cached annex-uuid for the remote. If there is a mismatch, it can change both the cached annex-uuid and the gcrypt-id. That should work, and I laid some groundwork for it by already reading the remote's config when it's local. (Also needed for other reasons.) This commit was sponsored by Daniel Callahan.
* Allow public-key encryption of file content.Gravatar guilhem2013-09-03
| | | | | | | | | | | | With the initremote parameters "encryption=pubkey keyid=788A3F4C". /!\ Adding or removing a key has NO effect on files that have already been copied to the remote. Hence using keyid+= and keyid-= with such remotes should be used with care, and make little sense unless the point is to replace a (sub-)key by another. /!\ Also, a test case has been added to ensure that the cipher and file contents are encrypted as specified by the chosen encryption scheme.
* hook special remote: Added combined hook program support.Gravatar Joey Hess2013-05-21
|
* connect existing meters to the transfer log for downloadsGravatar Joey Hess2013-04-11
| | | | | | | | | | | | | | Most remotes have meters in their implementations of retrieveKeyFile already. Simply hooking these up to the transfer log makes that information available. Easy peasy. This is particularly valuable information for encrypted remotes, which otherwise bypass the assistant's polling of temp files, and so don't have good progress bars yet. Still some work to do here (see progressbars.mdwn changes), but this is entirely an improvement from the lack of progress bars for encrypted downloads.
* webapp: Progess bar fixes for many types of special remotes.Gravatar Joey Hess2013-03-28
| | | | | | | | | | | | | There was confusion in different parts of the progress bar code about whether an update contained the total number of bytes transferred, or the number of bytes transferred since the last update. One way this bug showed up was progress bars that seemed to stick at zero for a long time. In order to fix it comprehensively, I add a new BytesProcessed data type, that is explicitly a total quantity of bytes, not a delta. Note that this doesn't necessarily fix every problem with progress bars. Particularly, buffering can now cause progress bars to seem to run ahead of transfers, reaching 100% when data is still being uploaded.
* add globallyAvailable to remotesGravatar Joey Hess2013-03-15
|
* split cost out into its own moduleGravatar Joey Hess2013-03-13
| | | | | Added a function to insert a new cost into a list, which could be used to asjust costs after a drag and drop.
* GnuPG options for symmetric encryption.Gravatar guilhem2013-03-11
|
* Special remotes now all rollback storage of keys that get modified during ↵Gravatar Joey Hess2013-01-09
| | | | the transfer, which can happen in direct mode.
* Fix transferring files to special remotes in direct mode.Gravatar Joey Hess2013-01-06
|
* type based git config handling for remotesGravatar Joey Hess2013-01-01
| | | | | Still a couple of places that use git config ad-hoc, but this is most of it done.
* whitespace fixesGravatar Joey Hess2012-12-13
|
* avoid unnecessary MaybeGravatar Joey Hess2012-11-30
|
* better streaming while encrypting/decryptingGravatar Joey Hess2012-11-18
| | | | | | Both the directory and webdav special remotes used to have to buffer the whole file contents before it could be decrypted, as they read from chunks. Now the chunks are streamed through gpg with no buffering.
* where indentingGravatar Joey Hess2012-11-11
|
* !! removalGravatar Joey Hess2012-10-21
|
* unify typesGravatar Joey Hess2012-09-21
|
* add a progress callback to storeKey, and threaded it all the way throughGravatar Joey Hess2012-09-19
| | | | | | | | Transfer info files are updated when the callback is called, updating the number of bytes transferred. Left unused p variables at every place the callback should be used. Which is rather a lot..
* add support for readonly remotesGravatar Joey Hess2012-08-26
| | | | | | | Currently only the web special remote is readonly, but it'd be possible to also have readonly drives, or other remotes. These are handled in the assistant by only downloading from them, and never trying to upload to them.
* tweak field nameGravatar Joey Hess2012-08-26
|
* add routes to pause/start/cancel transfersGravatar Joey Hess2012-08-08
| | | | | | | | | | | | | | | | This commit includes a paydown on technical debt incurred two years ago, when I didn't know that it was bad to make custom Read and Show instances for types. As the routes need Read and Show for Transfer, which includes a Key, and deriving my own Read instance of key was not practical, I had to finally clean that up. So the compact Key read and show functions are now file2key and key2file, and Read and Show are now derived instances. Changed all code that used the old instances, compiler checked. (There were a few places, particularly in Command.Unused, and the test suite where the Show instance continue to be used for legitimate comparisons; ie show key_x == show key_y (though really in a bloom filter))
* add a path field to remotesGravatar Joey Hess2012-07-22
| | | | | Also broke out some helper functions around constructing remotes, to be used later.
* switch from System.Cmd.Utils to System.ProcessGravatar Joey Hess2012-07-18
| | | | | | | | | | | | | | | | | | Test suite now passes with -threaded! I traced back all the hangs with -threaded to System.Cmd.Utils. It seems it's just crappy/unsafe/outdated, and should not be used. System.Process seems to be the cool new thing, so converted all the code to use it instead. In the process, --debug stopped printing commands it runs. I may try to bring that back later. Note that even SafeSystem was switched to use System.Process. Since that was a modified version of code from System.Cmd.Utils, it needed to be converted too. I also got rid of nearly all calls to forkProcess, and all calls to executeFile, which I'm also doubtful about working well with -threaded.
* record transfer information on local git remotesGravatar Joey Hess2012-07-01
| | | | | | | | | | | | | | | In order to record a semi-useful filename associated with the key, this required plumbing the filename all the way through to the remotes' storeKey and retrieveKeyFile. Note that there is potential for deadlock here, narrowly avoided. Suppose the repos are A and B. A sends file foo to B, and at the same time, B gets file foo from A. So, A locks its upload transfer info file, and then locks B's download transfer info file. At the same time, B is taking the two locks in the opposite order. This is only not a deadlock because the lock code does not wait, and aborts. So one of A or B's transfers will be aborted and the other transfer will continue. Whew!
* avoid ByteString.Char8 where not neededGravatar Joey Hess2012-06-20
| | | | | Its truncation behavior is a red flag, so avoid using it in these places where only raw ByteStrings are used, without looking at the data inside.
* Preserve parent environment when running hooks of the hook special remote.Gravatar Joey Hess2012-06-04
|
* Fix use of several config settingsGravatar Joey Hess2012-05-05
| | | | | | | annex.ssh-options, annex.rsync-options, annex.bup-split-options. And adjust types to avoid the bugs that broke several config settings recently. Now "annex." prefixing is enforced at the type level.
* rationalize getConfigGravatar Joey Hess2012-03-22
| | | | | | | | | | getConfig got a remote-specific config, and this confusing name caused it to be used a couple of places that only were interested in global configs. Rename to getRemoteConfig and make getConfig only get global configs. There are no behavior changes here, but remote.<name>.annex-web-options never actually worked (and per-remote web options is a very unlikely to be useful case so I didn't make it work), so fix the documentation for it.
* nukes another 15 lines thanks to ifMGravatar Joey Hess2012-03-15
|
* thread through original key to retrieveEnctyptedGravatar Joey Hess2012-03-04
| | | | | Allows showing progress bar for this last case of the directory special remote.
* whereis: Prints the urls of files that the web special remote knows about.Gravatar Joey Hess2012-02-14
|
* fsck --from remote --fastGravatar Joey Hess2012-01-20
| | | | | | | Avoids expensive file transfers, at the expense of checking file size and/or contents. Required some reworking of the remote code.
* add tmp flag parameter to retrieveKeyFileGravatar Joey Hess2012-01-19
|
* type alias cleanupGravatar Joey Hess2011-12-31
|
* better filtering out of special remotesGravatar Joey Hess2011-12-31
|
* Fix the hook special remote, which bitrotted a while ago.Gravatar Joey Hess2011-12-20
|
* add support for using hashDirLower in addition to hashDirMixedGravatar Joey Hess2011-11-28
| | | | | | | | | | Supporting multiple directory hash types will allow converting to a different one, without a flag day. gitAnnexLocation now checks which of the possible locations have a file. This means more statting of files. Several places currently use gitAnnexLocation and immediately check if the returned file exists; those need to be optimised.
* factored out some useful error catching methodsGravatar Joey Hess2011-11-10
|
* safer inannex checkingGravatar Joey Hess2011-11-09
| | | | | | | | git-annex-shell inannex now returns always 0, 1, or 100 (the last when it's unclear if content is currently in the index due to it currently being moved or dropped). (Actual locking code still not yet written.)
* reorder repo parameters lastGravatar Joey Hess2011-11-08
| | | | | | | | | | | | | Many functions took the repo as their first parameter. Changing it consistently to be the last parameter allows doing some useful things with currying, that reduce boilerplate. In particular, g <- gitRepo is almost never needed now, instead use inRepo to run an IO action in the repo, and fromRepo to get a value from the repo. This also provides more opportunities to use monadic and applicative combinators.