summaryrefslogtreecommitdiff
path: root/Annex.hs
Commit message (Collapse)AuthorAge
* indentation foo, and a new coding style page. no code changesGravatar Joey Hess2012-10-28
|
* deal with mtl/monads-tf conflictGravatar Joey Hess2012-10-24
| | | | | | I had been using -ignore-package monads-tf to deal with this, but the XMPP library uses monads-tf, so that also ignores it. Instead, use PackageImports to force use of mtl in my own code.
* add ConfigMonitor threadGravatar Joey Hess2012-10-20
| | | | | | | | | | | | | | | | | | | | Monitors git-annex branch for changes, which are noticed by the Merger thread whenever the branch ref is changed (either due to an incoming push, or a local change), and refreshes cached config values for modified config files. Rate limited to run no more often than once per minute. This is important because frequent git-annex branch changes happen when files are being added, or transferred, etc. A primary use case is that, when preferred content changes are made, and get pushed to remotes, the remotes start honoring those settings. Other use cases include propigating repository description and trust changes to remotes, and learning when a remote has added a new special remote, so the webapp can present the GUI to enable that special remote locally. Also added a uuid.log cache. All other config files already had caches.
* Preferred content path matching bugfix.Gravatar Joey Hess2012-10-17
| | | | | | | When in a subdir, both the normal filepath, and the filepath relative to the top of the git repo are needed for matching. The former for key lookup, and the latter for include/exclude to match against. Previously, key lookup didn't work in this situation.
* add AssumeNotPresent parameter to limitsGravatar Joey Hess2012-10-05
| | | | | | | | | | | | | Solves the issue with preferred content expressions and dropping that I mentioned yesterday. My solution was to add a parameter to specify a set of repositories where content should be assumed not to be present. When deciding whether to drop, it can put the current repository in, and then if the expression fails to match, the content can be dropped. Using yesterday's example "(not copies=trusted:2) and (not in=usbdrive)", when the local repo is one of the 2 trusted copies, the drop check will see only 1 trusted copy, so the expression matches, and so the content will not be dropped.
* added preferred-content log, and allow editing it with vicfgGravatar Joey Hess2012-10-04
| | | | | | | | | | | | | | This includes a full parser for the boolean expressions in the log, that compiles them into Matchers. Those matchers are not used yet. A complication is that matching against an expression should never crash git-annex with an error. Instead, vicfg checks that the expressions parse. If a bad expression (or an expression understood by some future git-annex version) gets into the log, it'll be ignored. Most of the code in Limit couldn't fail anyway, but I did have to make limitCopies check its parameter first, and return an error if it's bad, rather than erroring at runtime.
* group, ungroup: New commands to indicate groups of repositories.Gravatar Joey Hess2012-10-01
|
* pointlessnessGravatar Joey Hess2012-06-29
|
* run event handlers all in the same Annex monadGravatar Joey Hess2012-06-04
| | | | | | | | | | | | | | | | Uses a MVar again, as there seems no other way to thread the state through inotify events. This is a rather unsatisfactory result. I had wanted to run them in the same monad so that the git queue could be used to coleasce git commands and speed things up. But, that led to fragility: If several files are added, and one is removed before queue flush, git add will fail to add any of them. So, the queue is still explicitly flushed after each add for now. TODO: Investigate using git add --ignore-errors. This would need to be done in Command.Add. And, git add still exits nonzero with it, so would need to avoid crashing on queue flush.
* Add support for core.worktree, and fix support for GIT_WORK_TREE and GIT_DIR.Gravatar Joey Hess2012-05-18
| | | | | | | The environment needs to override git-config. Changed when git config is read, and avoid rereading it once it's been read. chdir for both worktree settings.
* fix test suite buildGravatar Joey Hess2012-04-30
|
* Added shared cipher mode to encryptable special remotes.Gravatar Joey Hess2012-04-29
| | | | | | This option avoids gpg key distribution, at the expense of flexability, and with the requirement that all clones of the git repository be equally trusted.
* display "Recording state in git..." when staging the journalGravatar Joey Hess2012-04-27
| | | | | | | | A bit tricky to avoid printing it twice in a row when there are queued git commands to run and journal to stage. Added a generic way to run an action that may output multiple side messages, with only the first displayed.
* cache parsed core.sharedrepositoryGravatar Joey Hess2012-04-21
|
* do a cleanup commit after moving data from or to a git remoteGravatar Joey Hess2012-02-25
| | | | | | | | Added Annex.cleanup, which is a general purpose interface for adding actions to run at the end. Remotes with the old git-annex-shell will commit every time, and have no commit command, so hide stderr when running the commit command.
* Added a annex.queuesize settingGravatar Joey Hess2012-02-15
| | | | | | | | | | useful when adding hundreds of thousands of files on a system with plenty of memory. git add gets quite slow in such a large repository, so if the system has more than the ~32 mb of memory the queue can use by default, it's a useful optimisation to increase the queue size, in order to decrease the number of times git add is run.
* rework git check-attr interfaceGravatar Joey Hess2012-02-13
| | | | | | | | | | | | | | | Now gitattributes are looked up, efficiently, in only the places that really need them, using the same approach used for cat-file. The old CheckAttr code seemed very fragile, in the way it streamed files through git check-attr. I actually found that cad8824852aa0623dc41eac02a9e2bae47d88ec4 was still deadlocking with ghc 7.4, at the end of adding a lot of files. This should fix that problem, and avoid future ones. The best part is that this removes withAttrFilesInGit and withNumCopies, which were complicated Seek methods, as well as simplfying the types for several other Seek methods that had a Backend tupled in.
* switch to the strict state monadGravatar Joey Hess2012-01-29
| | | | | | | | | | I had not realized what a memory leak the lazy state monad could be, although I have not seen much evidence of actual leaking in git-annex. However, if running git-annex on a great many files, this could matter. The additional Utility.State.changeState adds even more strictness, avoiding a problem I saw in github-backup where repeatedly modifying state built up a huge pile of thunks.
* ssh connection cachingGravatar Joey Hess2012-01-20
| | | | | | | | | | | Ssh connection caching is now enabled automatically by git-annex. Only one ssh connection is made to each host per git-annex run, which can speed some things up a lot, as well as avoiding repeated password prompts. Concurrent git-annex processes also share ssh connections. Cached ssh connections are shut down when git-annex exits. Note: The rsync special remote does not yet participate in the ssh connection caching.
* Add annex-trustlevel configuration settings, which can be used to override ↵Gravatar Joey Hess2012-01-09
| | | | | | | | | | | | | the trust level of a remote. This overrides the trust.log, and is overridden by the command-line trust parameters. It would have been nicer to have Logs.Trust.trustMap just look up the configuration for all remotes, but a dependency loop prevented that (Remotes depends on Logs.Trust in several ways). So instead, look up the configuration when building remotes, storing it in the same forcetrust field used for the command-line trust parameters.
* more command-specific optionsGravatar Joey Hess2012-01-06
| | | | | | | | | | | | | | | | | | | | | Made --from and --to command-specific options. Added generic storage for values of command-specific options, which allows removing some of the special case fields in AnnexState. (Also added generic storage for command-specific flags, although there are not yet any.) Note that this storage uses a Map, so repeatedly looking up the same value is slightly more expensive than looking up an AnnexState field. But, the value can be looked up once in the seek stage, transformed as necessary, and passed in a closure to the start stage, and this avoids that overhead. Still, I'm hesitant to use this for things like force or fast flags. It's probably best to reserve it for flags that are only used by a few commands, or options like --from and --to that it's important only be allowed to be used with commands that implement them, to avoid user confusion.
* type aliasGravatar Joey Hess2011-12-31
|
* type alias cleanupGravatar Joey Hess2011-12-31
|
* Merge branch 'new-monad-control'Gravatar Joey Hess2011-12-24
|\
* | Format strings can be specified using the new --find option, to control what ↵Gravatar Joey Hess2011-12-22
| | | | | | | | is output by git annex find.
* | fix commentGravatar Joey Hess2011-12-20
| |
* | use Common in a few more modulesGravatar Joey Hess2011-12-20
| |
* | split out three modules from GitGravatar Joey Hess2011-12-13
| | | | | | | | | | Constructors and configuration make sense in separate modules. A separate Git.Types is needed to avoid cycles.
| * Merge branch 'master' into new-monad-controlGravatar Joey Hess2011-12-11
| |\ | |/ |/| | | | | Conflicts: git-annex.cabal
* | Fix caching of decrypted ciphers, which failed when drop had to check ↵Gravatar Joey Hess2011-12-08
| | | | | | | | multiple different encrypted special remotes.
| * the Annex newtype is backGravatar Joey Hess2011-12-06
| | | | | | | | | | | | | | Thanks to Bas van Dijk for providing the instance declarations I needed. Grody stuff. Bas is talking about perhaps providing utility functions that contain the ugly parts, so this code may be able to be removed using a future version of monad-control.
| * adjust to build with monad-control-0.3Gravatar Joey Hess2011-12-05
|/ | | | | | | | | I had to, I hope temporarily, lose my nice Annex newtype, and use a type synonym. This because I cannot find a way to derive a MonadBaseControl instance of the Annex newtype. I've emailed Bas van Dijk in hope he can help get the newtype back. Otherwise appears to build & work.
* find: Support --print0Gravatar Joey Hess2011-11-22
| | | | | | | | | | | It would be nice if command-specific options were supported. The first difficulty is that which command is being called is not known until after getopt; but that could be worked around by finding the first non-dashed parameter. Storing the settings without putting them in the annex monad is the next difficulty; it could perhaps be handled by making the seek stage pass applicable settings into the start stage (and from there on to perform as needed). But that still leaves a problem, what data type to use to represent the options between getopt and seek?
* golfGravatar Joey Hess2011-11-12
|
* cleanupGravatar Joey Hess2011-11-09
|
* reorder repo parameters lastGravatar Joey Hess2011-11-08
| | | | | | | | | | | | | Many functions took the repo as their first parameter. Changing it consistently to be the last parameter allows doing some useful things with currying, that reduce boilerplate. In particular, g <- gitRepo is almost never needed now, instead use inRepo to run an IO action in the repo, and fromRepo to get a value from the repo. This also provides more opportunities to use monadic and applicative combinators.
* The fromkey command now takes the key as its first parameter. The --key ↵Gravatar Joey Hess2011-10-31
| | | | option is no longer used.
* improve config reading when operating on remote on same hostGravatar Joey Hess2011-10-27
| | | | | | | | | Before the config was read each time onLocal was called, and entirely redundantly since it's read for same-host remotes on startup. Also a minor bug fix: When rsyncing to a same-host remote, use the rsync-options from the repository that the user ran git-annex in, not those of the receiving repository.
* factor out common importsGravatar Joey Hess2011-10-03
| | | | no code changes
* refactor catfile codeGravatar Joey Hess2011-09-28
| | | | split into generic IO code, and a thin Annex wrapper
* rework annex-ignore handlingGravatar Joey Hess2011-09-18
| | | | | | | | Only one place need to filter the list of remotes for ignored remotes: keyPossibilities. Make the full list available to everything else. This allows getting rid of the special case handing for --from and --to to make ignored remotes not be ignored with those options.
* refactor --exclude to use Utility.MatcherGravatar Joey Hess2011-09-18
| | | | | This should change no behavior, but opens the poissibility to use the matcher for other sorts of limits on which files git-annex processes.
* remove optimize subcommand; use --auto insteadGravatar Joey Hess2011-09-15
| | | | | | | | | | | | | | | | | | get, drop: Added --auto option, which decides whether to get/drop content as needed to work toward the configured numcopies. The problem with bundling it up in optimize was that I then found I wanted to run an optmize that did not drop files, only got them. Considered adding a --only-get switch to it, but that seemed wrong. Instead, let's make existing subcommands optionally smarter. Note that the only actual difference between drop and drop --auto is that the latter does not even try to drop a file if it knows of not enough copies, and does not print any error messages about files it was unable to drop. It might be nice to make get avoid asking git for attributes when not in auto mode. For now it always asks for attributes.
* basic json supportGravatar Joey Hess2011-09-01
| | | | | | | | | | | | | | | | | | This includes a generic JSONStream library built on top of Text.JSON (somewhat hackishly). It would be possible to stream out a single json document describing all actions, but it's probably better for consumers if they can expect one json document per line, so I did it that way instead. Output from external programs used for transferring files is not currently hidden when outputting json, which probably makes it not very useful there. This may be dealt with if there is demand for json output for --get or --move to be parsable. The version, status, and find subcommands have hand-crafted output and don't do json. The whereis subcommand needs to be modified to produce useful json.
* generalize quiet flag to output typeGravatar Joey Hess2011-09-01
| | | | This will allow adding other styles of output.
* code simplification thanks to applicative functorsGravatar Joey Hess2011-08-25
|
* avoid the functorGravatar Joey Hess2011-08-21
| | | | fmap = liftM
* drop an unnecessart liftIOGravatar Joey Hess2011-08-19
| | | | the liftM on its own can lift all the way into IO.
* make Annex an opaque data typeGravatar Joey Hess2011-08-19
| | | | | | | | | Was a type alias; using newtype has the benefit that type errors will show "Annex foo" rather than two lines of internal type nonsense. Yay! There should be no other effects to size or runtime. I've tried to do this at least twice before (each time I read RWH chapter 10); finally understood how to this time.. sorta.
* remove unused backend machineryGravatar Joey Hess2011-07-05
| | | | | | | | | | | | | The only remaining vestiage of backends is different types of keys. These are still called "backends", mostly to avoid needing to change user interface and configuration. But everything to do with storing keys in different backends was gone; instead different types of remotes are used. In the refactoring, lots of code was moved out of odd corners like Backend.File, to closer to where it's used, like Command.Drop and Command.Fsck. Quite a lot of dead code was removed. Several data structures became simpler, which may result in better runtime efficiency. There should be no user-visible changes.