summaryrefslogtreecommitdiff
path: root/Remote
Commit message (Collapse)AuthorAge
* add a path field to remotesGravatar Joey Hess2012-07-22
| | | | | Also broke out some helper functions around constructing remotes, to be used later.
* add back debug loggingGravatar Joey Hess2012-07-19
| | | | | | | | | | | | | Make Utility.Process wrap the parts of System.Process that I use, and add debug logging to them. Also wrote some higher-level code that allows running an action with handles to a processes stdin or stdout (or both), and checking its exit status, all in a single function call. As a bonus, the debug logging now indicates whether the process is being run to read from it, feed it data, chat with it (writing and reading), or just call it for its side effect.
* switch from System.Cmd.Utils to System.ProcessGravatar Joey Hess2012-07-18
| | | | | | | | | | | | | | | | | | Test suite now passes with -threaded! I traced back all the hangs with -threaded to System.Cmd.Utils. It seems it's just crappy/unsafe/outdated, and should not be used. System.Process seems to be the cool new thing, so converted all the code to use it instead. In the process, --debug stopped printing commands it runs. I may try to bring that back later. Note that even SafeSystem was switched to use System.Process. Since that was a modified version of code from System.Cmd.Utils, it needed to be converted too. I also got rid of nearly all calls to forkProcess, and all calls to executeFile, which I'm also doubtful about working well with -threaded.
* avoid --no-inplaceGravatar Joey Hess2012-07-10
| | | | | Not available on systems with shoddy getopts. Should not be necessary, as that's rsync's default.
* pass associatedfile and remoteuuid to git-annex-shellGravatar Joey Hess2012-07-02
| | | | | | | | This *almost* works. Along the way, I noticed that the --uuid parameter was being accidentially passed after the --, so that has never been actually used by git-annex-shell to verify it's running in the expected repository. Oops. Fixed.
* record transfer information on local git remotesGravatar Joey Hess2012-07-01
| | | | | | | | | | | | | | | In order to record a semi-useful filename associated with the key, this required plumbing the filename all the way through to the remotes' storeKey and retrieveKeyFile. Note that there is potential for deadlock here, narrowly avoided. Suppose the repos are A and B. A sends file foo to B, and at the same time, B gets file foo from A. So, A locks its upload transfer info file, and then locks B's download transfer info file. At the same time, B is taking the two locks in the opposite order. This is only not a deadlock because the lock code does not wait, and aborts. So one of A or B's transfers will be aborted and the other transfer will continue. Whew!
* pointlessnessGravatar Joey Hess2012-06-29
|
* Better fix for unavailable local remotesGravatar Joey Hess2012-06-26
| | | | | | Not including such remotes turned out to have other consequences, including annex-truselevel git config being ignored. Instead, add guards before each operation that might try to operate on such a repo.
* Avoid ugly failure mode when moving content from a local repository that is ↵Gravatar Joey Hess2012-06-26
| | | | | | | | | | | | | | | | | | | not available. Prelude.undefined error message was introduced by bb4f31a0ee496ffb83d31cc56f8827e47605d763. It seems best to filter out local repositories that cannot be accessed from the list of remotes, rather than keeping them in and making every thing that uses the list have to deal with remotes that may have an unknown location. Besides fixing the error message, this also makes unavailable local remotes' names not be shown in various messages, including in git annex status output. Also, move --to an unavailable local repository now avoids some ugly errors like "changeWorkingDirectory: does not exist".
* avoid ByteString.Char8 where not neededGravatar Joey Hess2012-06-20
| | | | | Its truncation behavior is a red flag, so avoid using it in these places where only raw ByteStrings are used, without looking at the data inside.
* fishy commitGravatar Joey Hess2012-06-14
|
* use createAnnexDirectory when setting up tmp dirGravatar Joey Hess2012-06-05
|
* Preserve parent environment when running hooks of the hook special remote.Gravatar Joey Hess2012-06-04
|
* suppress "(Recording state in git)" message when committing change to remote ↵Gravatar Joey Hess2012-05-20
| | | | | | | | | | | | | | | state This was shown redundantly for a tricky reason -- while it runs inside a doSideAction block that would appear to supress it, the action being run is in a different state monad; for the remote, and so the suppression doesn't work. Always suppressing the message when committing to a local remote is ok do to though -- it mirrors the /dev/nulling of the git annex shell commit output. And it turns out that any time there is a git-annex branch state change to commit on the remote, the local repo has also had a similar change made, and so the message has been shown already.
* Add support for core.worktree, and fix support for GIT_WORK_TREE and GIT_DIR.Gravatar Joey Hess2012-05-18
| | | | | | | The environment needs to override git-config. Changed when git config is read, and avoid rereading it once it's been read. chdir for both worktree settings.
* Clean up handling of git directory and git worktree.Gravatar Joey Hess2012-05-18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Baked into the code was an assumption that a repository's git directory could be determined by adding ".git" to its work tree (or nothing for bare repos). That fails when core.worktree, or GIT_DIR and GIT_WORK_TREE are used to separate the two. This was attacked at the type level, by storing the gitdir and worktree separately, so Nothing for the worktree means a bare repo. A complication arose because we don't learn where a repository is bare until its configuration is read. So another Location type handles repositories that have not had their config read yet. I am not entirely happy with this being a Location type, rather than representing them entirely separate from the Git type. The new code is not worse than the old, but better types could enforce more safety. Added support for core.worktree. Overriding it with -c isn't supported because it's not really clear what to do if a git repo's config is read, is not bare, and is then overridden to bare. What is the right git directory in this case? I will worry about this if/when someone has a use case for overriding core.worktree with -c. (See Git.Config.updateLocation) Also removed and renamed some functions like gitDir and workTree that misused git's terminology. One minor regression is known: git annex add in a bare repository does not print a nice error message, but runs git ls-files in a way that fails earlier with a less nice error message. This is because before --work-tree was always passed to git commands, even in a bare repo, while now it's not.
* Fix use of several config settingsGravatar Joey Hess2012-05-05
| | | | | | | annex.ssh-options, annex.rsync-options, annex.bup-split-options. And adjust types to avoid the bugs that broke several config settings recently. Now "annex." prefixing is enforced at the type level.
* rsync shellescape disable optionGravatar Joey Hess2012-05-02
| | | | | | | | Rsync special remotes can be configured with shellescape=no to avoid shell quoting that is normally done when using rsync over ssh. This is known to be needed for certian rsync hosting providers (specificially hidrive.strato.com) that use rsync over ssh but do not pass it through the shell.
* refactorGravatar Joey Hess2012-04-29
|
* Added shared cipher mode to encryptable special remotes.Gravatar Joey Hess2012-04-29
| | | | | | This option avoids gpg key distribution, at the expense of flexability, and with the requirement that all clones of the git repository be equally trusted.
* Add annex.httpheaders and annex.httpheader-command config settingsGravatar Joey Hess2012-04-22
| | | | | | Allow custom headers to be sent with all HTTP requests. (Requested by the Internet Archive)
* noopGravatar Joey Hess2012-04-21
|
* in which I discover voidGravatar Joey Hess2012-04-21
| | | | void :: Functor f => f a -> f () -- ah, of course that's useful :)
* honor core.sharedRepository when making all the other files in the annexGravatar Joey Hess2012-04-21
| | | | Lock files, directories, etc.
* Directory special remotes now check annex.diskreserve.Gravatar Joey Hess2012-04-20
|
* use LANGUAGE CPP pragma, avoids running cpp on all the other sourcesGravatar Joey Hess2012-04-17
|
* cabal file now autodetects whether S3 support is available.Gravatar Joey Hess2012-04-14
|
* bup: Properly handle key names with spaces or other things that are not ↵Gravatar Joey Hess2012-04-11
| | | | | | | | legal git refs. Continue using the key name as bup ref name, to preserve backwards compatability, unless it is an illegal git ref. In that case, use a sha256 of the key name instead.
* rationalize getConfigGravatar Joey Hess2012-03-22
| | | | | | | | | | getConfig got a remote-specific config, and this confusing name caused it to be used a couple of places that only were interested in global configs. Rename to getRemoteConfig and make getConfig only get global configs. There are no behavior changes here, but remote.<name>.annex-web-options never actually worked (and per-remote web options is a very unlikely to be useful case so I didn't make it work), so fix the documentation for it.
* fun with symbolsGravatar Joey Hess2012-03-17
| | | | | | Nothing at all on hackage is using <&&> or <||>. (Also, <&&> should short-circuit on failure.)
* nukes another 15 lines thanks to ifMGravatar Joey Hess2012-03-15
|
* Work around a bug in rsync (IMHO) introduced by openSUSE's SIP patch.Gravatar Joey Hess2012-03-12
| | | | | | | | | | | | | | | | | | | | | | | | | openSUSE patches rsync with a patch adding SIP protocol support. https://gist.github.com/2026167 With this patch, running rsync with no hostname parameter is apparently supposed to list SIP hosts on the network. Practically, it does nothing and exits 0. git-annex uses rsync in a very special way to allow git-annex-shell to be run on the remote host, and so did not need to specify a hostname, or a file to transfer as a rsync parameter. So it sent ":", a degenerate case of "host:file". But the patch cannot differentiate ":" with no host parameter (a bug in the SIP patch surely). Results were that getting files failed, as rsync seemed to succeed, but the requested file failed to arrive. Also I think that sending files will make git-annex think a file has been transferred to the remote when really rsync does nothing. The workaround for this buggy rsync patch is to use "dummy:" as the hostname.
* add remote start and stop hooksGravatar Joey Hess2012-03-04
| | | | | | Locking is used, so that, if there are multiple git-annex processes using a remote concurrently, the stop hook is only run by the last process that uses it.
* better chunked file retrievalGravatar Joey Hess2012-03-04
| | | | | | Avoids opening every chunk at once, instead streaming them in. Not done for encrypted file retrieval yet.
* thread through original key to retrieveEnctyptedGravatar Joey Hess2012-03-04
| | | | | Allows showing progress bar for this last case of the directory special remote.
* add progress display when receiving filesGravatar Joey Hess2012-03-04
| | | | | | That was actually really easy. But, when getting a file from an encrypted directory special remote, no meter can be shown, because the total file size is not known.
* Add progress bar display to the directory special remote.Gravatar Joey Hess2012-03-04
| | | | | | | So far I've only written progress bars for sending files, not yet receiving. No longer uses external cp at all. ByteString IO is fast enough.
* tweakGravatar Joey Hess2012-03-03
|
* Directory special remotes now support chunking files written to themGravatar Joey Hess2012-03-03
| | | | | | Avoiding writing files larger than a specified size is useful on certian things. For example, box.com has a file size limit of 100 mb. Could also be useful on really crappy removable media.
* do a cleanup commit after moving data from or to a git remoteGravatar Joey Hess2012-02-25
| | | | | | | | Added Annex.cleanup, which is a general purpose interface for adding actions to run at the end. Remotes with the old git-annex-shell will commit every time, and have no commit command, so hide stderr when running the commit command.
* whereis: Prints the urls of files that the web special remote knows about.Gravatar Joey Hess2012-02-14
|
* oopsGravatar Joey Hess2012-02-14
|
* rework git check-attr interfaceGravatar Joey Hess2012-02-13
| | | | | | | | | | | | | | | Now gitattributes are looked up, efficiently, in only the places that really need them, using the same approach used for cat-file. The old CheckAttr code seemed very fragile, in the way it streamed files through git check-attr. I actually found that cad8824852aa0623dc41eac02a9e2bae47d88ec4 was still deadlocking with ghc 7.4, at the end of adding a lot of files. This should fix that problem, and avoid future ones. The best part is that this removes withAttrFilesInGit and withNumCopies, which were complicated Seek methods, as well as simplfying the types for several other Seek methods that had a Backend tupled in.
* When checking that an url has a key, verify that the Content-Length, if ↵Gravatar Joey Hess2012-02-10
| | | | | | | | available, matches the size of the key. If there's no Content-Length, or the key has no size, this check is not done, but it should happen most of the time, and protect against web content that has changed.
* S3: Fix irrefutable pattern failure when accessing encrypted S3 credentials.Gravatar Joey Hess2012-02-08
|
* refactorGravatar Joey Hess2012-02-07
| | | | Wow, triple monadic lift!
* IO exception reworkGravatar Joey Hess2012-02-03
| | | | | | ghc 7.4 comaplains about use of System.IO.Error to catch exceptions. Ok, use Control.Exception, with variants specialized to only catch IO exceptions.
* faster local-local droppingGravatar Joey Hess2012-01-28
| | | | | | Dropping a key from a local remote ran git-annex-shell unnecessarily. Now git-annex-shell is never used when acting on a local remote.
* Avoid repeated location log commits when a remote is receiving files.Gravatar Joey Hess2012-01-28
| | | | | | | | | Done by adding a oneshot mode, in which location log changes are written to the journal, but not committed. Taking advantage of git-annex's existing ability to recover in this situation. This is used by git-annex-shell and other places where changes are made to a remote's location log.
* Revert "Avoid creating ~/.bup when initializing a bup remote"Gravatar Joey Hess2012-01-28
| | | | | | | | | | | | | | This reverts commit 6da40100c99a097401fb130b63edc05b18f61249. On closer examinaton, this change is wrong. The bup special remote can be configured with "buprepo=", which makes it use the default ~/.bup repo. This change makes it use a different temp dir each time, which I'm sure would not be appreciated by anyone with that configuration. Bup insisting in creating ~/.bup even when using a different repo does seem like a bug in *something*, but I'm leaning toward the bug being in bup itself.