summaryrefslogtreecommitdiff
path: root/Remote/Rsync.hs
Commit message (Collapse)AuthorAge
* split cost out into its own moduleGravatar Joey Hess2013-03-13
| | | | | Added a function to insert a new cost into a list, which could be used to asjust costs after a drag and drop.
* GnuPG options for symmetric encryption.Gravatar guilhem2013-03-11
|
* build fixGravatar Joey Hess2013-02-15
|
* optimise sending to encrypted rsyncGravatar Joey Hess2013-02-15
| | | | | | With an encrypted rsync remote, the encrpyted file can be renamed, rather than being copied, in crippled filesystem mode. This gets back to just as fast as non-crippled mode for this very common case.
* rsync special remote support for crippled filesystem modeGravatar Joey Hess2013-02-15
| | | | | | | | | | Cannot make a hard link, have to copy. I did find a way to make it work without setting up a tree, just using --include and --exclude. But it needs the same hash directories to be used on both sides, which is normally not the case. Still, I hope one day I will convert non-bare repos to use the same hash dirs as everything else, and then this will get more efficient.
* Special remotes now all rollback storage of keys that get modified during ↵Gravatar Joey Hess2013-01-09
| | | | the transfer, which can happen in direct mode.
* Fix transferring files to special remotes in direct mode.Gravatar Joey Hess2013-01-06
|
* type based git config handling for remotesGravatar Joey Hess2013-01-01
| | | | | Still a couple of places that use git config ad-hoc, but this is most of it done.
* whitespace fixesGravatar Joey Hess2012-12-13
|
* avoid unnecessary MaybeGravatar Joey Hess2012-11-30
|
* rsync: Fix bug introduced in last release that broke encrypted rsync special ↵Gravatar Joey Hess2012-11-27
| | | | remotes.
* better streaming while encrypting/decryptingGravatar Joey Hess2012-11-18
| | | | | | Both the directory and webdav special remotes used to have to buffer the whole file contents before it could be decrypted, as they read from chunks. Now the chunks are streamed through gpg with no buffering.
* where indentingGravatar Joey Hess2012-11-11
|
* rsync special remote: Include annex-rsync-options when running rsync to test ↵Gravatar Joey Hess2012-10-28
| | | | | | | a key's presence. Also, use the new withQuietOutput function to avoid running the shell to /dev/null stderr in two other places.
* unify typesGravatar Joey Hess2012-09-21
|
* hook rsync special remote up to the progress reportingGravatar Joey Hess2012-09-20
| | | | | | | | Easy! Note that with an encrypted remote, rsync will be sending a little more data than the key size, so displayed progress may get to 100% slightly quicker than it should. I doubt this is a big enough effect to worry about.
* add a progress callback to storeKey, and threaded it all the way throughGravatar Joey Hess2012-09-19
| | | | | | | | Transfer info files are updated when the callback is called, updating the number of bytes transferred. Left unused p variables at every place the callback should be used. Which is rather a lot..
* renamed RsyncFile -> RsyncGravatar Joey Hess2012-09-19
|
* add support for readonly remotesGravatar Joey Hess2012-08-26
| | | | | | | Currently only the web special remote is readonly, but it'd be possible to also have readonly drives, or other remotes. These are handled in the assistant by only downloading from them, and never trying to upload to them.
* tweak field nameGravatar Joey Hess2012-08-26
|
* add a path field to remotesGravatar Joey Hess2012-07-22
| | | | | Also broke out some helper functions around constructing remotes, to be used later.
* switch from System.Cmd.Utils to System.ProcessGravatar Joey Hess2012-07-18
| | | | | | | | | | | | | | | | | | Test suite now passes with -threaded! I traced back all the hangs with -threaded to System.Cmd.Utils. It seems it's just crappy/unsafe/outdated, and should not be used. System.Process seems to be the cool new thing, so converted all the code to use it instead. In the process, --debug stopped printing commands it runs. I may try to bring that back later. Note that even SafeSystem was switched to use System.Process. Since that was a modified version of code from System.Cmd.Utils, it needed to be converted too. I also got rid of nearly all calls to forkProcess, and all calls to executeFile, which I'm also doubtful about working well with -threaded.
* avoid --no-inplaceGravatar Joey Hess2012-07-10
| | | | | Not available on systems with shoddy getopts. Should not be necessary, as that's rsync's default.
* record transfer information on local git remotesGravatar Joey Hess2012-07-01
| | | | | | | | | | | | | | | In order to record a semi-useful filename associated with the key, this required plumbing the filename all the way through to the remotes' storeKey and retrieveKeyFile. Note that there is potential for deadlock here, narrowly avoided. Suppose the repos are A and B. A sends file foo to B, and at the same time, B gets file foo from A. So, A locks its upload transfer info file, and then locks B's download transfer info file. At the same time, B is taking the two locks in the opposite order. This is only not a deadlock because the lock code does not wait, and aborts. So one of A or B's transfers will be aborted and the other transfer will continue. Whew!
* avoid ByteString.Char8 where not neededGravatar Joey Hess2012-06-20
| | | | | Its truncation behavior is a red flag, so avoid using it in these places where only raw ByteStrings are used, without looking at the data inside.
* fishy commitGravatar Joey Hess2012-06-14
|
* use createAnnexDirectory when setting up tmp dirGravatar Joey Hess2012-06-05
|
* rsync shellescape disable optionGravatar Joey Hess2012-05-02
| | | | | | | | Rsync special remotes can be configured with shellescape=no to avoid shell quoting that is normally done when using rsync over ssh. This is known to be needed for certian rsync hosting providers (specificially hidrive.strato.com) that use rsync over ssh but do not pass it through the shell.
* rationalize getConfigGravatar Joey Hess2012-03-22
| | | | | | | | | | getConfig got a remote-specific config, and this confusing name caused it to be used a couple of places that only were interested in global configs. Rename to getRemoteConfig and make getConfig only get global configs. There are no behavior changes here, but remote.<name>.annex-web-options never actually worked (and per-remote web options is a very unlikely to be useful case so I didn't make it work), so fix the documentation for it.
* nukes another 15 lines thanks to ifMGravatar Joey Hess2012-03-15
|
* thread through original key to retrieveEnctyptedGravatar Joey Hess2012-03-04
| | | | | Allows showing progress bar for this last case of the directory special remote.
* whereis: Prints the urls of files that the web special remote knows about.Gravatar Joey Hess2012-02-14
|
* remove Utility.Conditional and use IfElseGravatar Joey Hess2012-01-24
| | | | | | | This drops the >>! and >>? with the nice low fixity. IfElse does have undocumented >>=>>! and >>=>>? operators, but I deem that too fishy. Anyway, using whenM and unlessM is easier; I sometimes mixed the operators up.
* fsck --from remote --fastGravatar Joey Hess2012-01-20
| | | | | | | Avoids expensive file transfers, at the expense of checking file size and/or contents. Required some reworking of the remote code.
* optimise fsck --from normal git remotesGravatar Joey Hess2012-01-19
| | | | | | | | | | For a local git remote, can symlink the file. For a git remote using rsync, can preseed any local content. There are a few reasons to use fsck --from on a normal git remote. One is if it's using gitosis or similar, and you don't have shell access to run git annex locally. Another reason could be if you just want to fsck certian files of a bare remote.
* add tmp flag parameter to retrieveKeyFileGravatar Joey Hess2012-01-19
|
* optimise fsck --from rsync special remoteGravatar Joey Hess2012-01-19
| | | | | When a file is present locally, the remote's version can be rsynced to a copy of it, which will avoid wasting a lot of bandwidth.
* after is a better name for observe_Gravatar Joey Hess2012-01-03
|
* factor observe_Gravatar Joey Hess2012-01-03
|
* type alias cleanupGravatar Joey Hess2011-12-31
|
* better filtering out of special remotesGravatar Joey Hess2011-12-31
|
* Improve deletion of files from rsync special remotes. Closes: #652849Gravatar Joey Hess2011-12-21
| | | | | | Rsync is only run once, with include / exclude rules used to specify exactly what to delete. This is faster, and avoids ugly error messages from rsync, and doesn't fail if the content already got deleted somehow.
* more partial function removalGravatar Joey Hess2011-12-15
| | | | | Left a few Prelude.head's in where it was checked not null and too hard to remove, etc.
* factor out untilTrueGravatar Joey Hess2011-12-02
|
* convert rsync special backend to using both hash directory typesGravatar Joey Hess2011-12-02
|
* add support for using hashDirLower in addition to hashDirMixedGravatar Joey Hess2011-11-28
| | | | | | | | | | Supporting multiple directory hash types will allow converting to a different one, without a flag day. gitAnnexLocation now checks which of the possible locations have a file. This means more statting of files. Several places currently use gitAnnexLocation and immediately check if the returned file exists; those need to be optimised.
* Avoid excessive escaping for rsync special remotes that are not accessed ↵Gravatar Joey Hess2011-11-18
| | | | | | | | | over ssh. This is actually tricky, 45bbf210a1210172c7c7b87879ed74f7c8ccbdba added the escaping because it's needed for rsync that does go over ssh. So I had to detect whether the remote's rsync url will use ssh or not, and vary the escaping.
* factored out some useful error catching methodsGravatar Joey Hess2011-11-10
|
* safer inannex checkingGravatar Joey Hess2011-11-09
| | | | | | | | git-annex-shell inannex now returns always 0, 1, or 100 (the last when it's unclear if content is currently in the index due to it currently being moved or dropped). (Actual locking code still not yet written.)
* reorder repo parameters lastGravatar Joey Hess2011-11-08
| | | | | | | | | | | | | Many functions took the repo as their first parameter. Changing it consistently to be the last parameter allows doing some useful things with currying, that reduce boilerplate. In particular, g <- gitRepo is almost never needed now, instead use inRepo to run an IO action in the repo, and fromRepo to get a value from the repo. This also provides more opportunities to use monadic and applicative combinators.