summaryrefslogtreecommitdiff
path: root/Annex.hs
Commit message (Collapse)AuthorAge
* converted MetaData, eliminating a global value from Annex state .. beautifulGravatar Joey Hess2015-07-12
|
* better memoize core.sharedrepository handlingGravatar Joey Hess2015-05-19
| | | | | It was memoized, but that was not used consistently. Move it to Types.GitConfig so it will auto-memoize.
* use lock pools throughout git-annexGravatar Joey Hess2015-05-19
| | | | | | | | | | | | | The one exception is in Utility.Daemon. As long as a process only daemonizes once, which seems reasonable, and as long as it avoids calling checkDaemon once it's already running as a daemon, the fcntl locking gotchas won't be a problem there. Annex.LockFile has it's own separate lock pool layer, which has been renamed to LockCache. This is a persistent cache of locks that persist until closed. This is not quite done; lockContent stil needs to be converted.
* Merge branch 'master' into concurrentprogressGravatar Joey Hess2015-05-12
|\ | | | | | | | | | | | | | | | | | | | | | | Conflicts: Command/Fsck.hs Messages.hs Remote/Directory.hs Remote/Git.hs Remote/Helper/Special.hs Types/Remote.hs debian/changelog git-annex.cabal
| * refactorGravatar Joey Hess2015-04-30
| |
* | get, move, copy, mirror: Concurrent downloads and uploads are now supported!Gravatar Joey Hess2015-04-10
|/ | | | | | | | | | | This works, and seems fairly robust. Clean get of 20 files at -J3. At -J10, there are some messages about ssh multiplexing, probably due to a race spinning up the ssh connection cacher. But, it manages to get all the files ok regardless. The progress bars are a scrambled mess though, due to bugs in ascii-progress, which I've already filed. Particularly this one: https://github.com/yamadapc/haskell-ascii-progress/issues/8
* use defGravatar Joey Hess2015-04-03
|
* --auto is no longer a global option; only get, drop, and copy accept it.Gravatar Joey Hess2015-03-25
| | | | Not a behavior change unless you were passing it to a command that ignored it.
* Submodules are now supported by git-annex!Gravatar Joey Hess2015-03-02
| | | | | | | | | | | | | | | | | | Seems to work, but still experimental until it's been tested more. When repositories are on filesystems not supporting symlinks, the .git dir symlink trick cannot be used. Since we're going to be in direct mode anyway, the .git dir symlink is not strictly needed. However, I have not fixed the code that creates new annex symlinks to handle this case -- the committed symlinks will be wrong. git annex sync happens to currently fail in a submodule using direct mode, because there's no HEAD ref. That also needs to be dealt with to get this fully working in crippled filesystems. Leaving http://github.com/datalad/datalad/issues/44 open until these issues are dealt with.
* update my email address and homepage urlGravatar Joey Hess2015-01-21
|
* remove ifdef that is always trueGravatar Joey Hess2015-01-15
| | | | cabal file has a depends on exceptions 0.6.0
* handle sync's use of setCurrentDirectory to work with relative pathsGravatar Joey Hess2015-01-06
| | | | | | I think this is the last problimatic setCurrentDirectory. I also audited for extrnal commands that git-annex might run with cwd = foo, and did not find any that were passed any FilePath that might be absolute.
* Switch to using relative paths to the git repository.Gravatar Joey Hess2015-01-06
| | | | | | | | | | | | | | | | | | | | | | | | | | | This allows the git repository to be moved while git-annex is running in it, with fewer problems. On Windows, this avoids some of the problems with the absurdly small MAX_PATH of 260 bytes. In particular, git-annex repositories should work in deeper/longer directory structures than before. See http://git-annex.branchable.com/bugs/__34__git-annex:_direct:_1_failed__34___on_Windows/ There are several possible ways this change could break git-annex: 1. If it changes its working directory while it's running, that would be Bad News. Good news everyone! git-annex never does so. It would also break thread safety, so all such things were stomped out long ago. 2. parentDir "." -> "" which is not a valid path. I had to fix one instace of this, and I should probably wipe all calls to parentDir out of the git-annex code base; it was never a good idea. 3. Things like relPathDirToFile require absolute input paths, and code assumes that the git repo path is absolute and passes it to it as-is. In the case of relPathDirToFile, I converted it to not make this assumption. Currently, the test suite has 16 failures.
* try to avoid Windows MAX_PATH limit, by using \\?\ prefix on git repo pathGravatar Joey Hess2015-01-06
|
* Urls can now be claimed by remotes. This will allow creating, for example, a ↵Gravatar Joey Hess2014-12-08
| | | | external special remote that handles magnet: and *.torrent urls.
* When accessing a local remote, shut down git-cat-file processes afterwards, ↵Gravatar Joey Hess2014-08-20
| | | | | | | | | | | | | | | to ensure that remotes on removable media can be unmounted. Closes: #758630 This does mean that eg, copying multiple files to a local remote will become slightly slower, since it now restarts git-cat-file after each copy. Should not be significant slowdown. The reason git-cat-file is run on the remote at all is to update its location log. In order to add an item to it, it needs to get the current content of the log. Finding a way to avoid needing to do that would be a good path to avoiding this slowdown if it does become a problem somehow. This commit was sponsored by Evan Deaubl.
* unify exception handling into Utility.ExceptionGravatar Joey Hess2014-08-07
| | | | | | | | | | | | | | | | | | | | Removed old extensible-exceptions, only needed for very old ghc. Made webdav use Utility.Exception, to work after some changes in DAV's exception handling. Removed Annex.Exception. Mostly this was trivial, but note that tryAnnex is replaced with tryNonAsync and catchAnnex replaced with catchNonAsync. In theory that could be a behavior change, since the former caught all exceptions, and the latter don't catch async exceptions. However, in practice, nothing in the Annex monad uses async exceptions. Grepping for throwTo and killThread only find stuff in the assistant, which does not seem related. Command.Add.undo is changed to accept a SomeException, and things that use it for rollback now catch non-async exceptions, rather than only IOExceptions.
* fix for Windows file timestamp timezone madnessGravatar Joey Hess2014-06-12
| | | | | | | | | | | | | | | | | | | | | | On Windows, changing the time zone causes the apparent mtime of files to change. This confuses git-annex, which natually thinks this means the files have actually been modified (since THAT'S WHAT A MTIME IS FOR, BILL <sheesh>). Work around this stupidity, by using the inode sentinal file to detect if the timezone has changed, and calculate a TSDelta, which will be applied when generating InodeCaches. This should add no overhead at all on unix. Indeed, I sped up a few things slightly in the refactoring. Seems to basically work! But it has a big known problem: If the timezone changes while the assistant (or a long-running command) runs, it won't notice, since it only checks the inode cache once, and so will use the old delta for all new inode caches it generates for new files it's added. Which will result in them seeming changed the next time it runs. This commit was sponsored by Vincent Demeester.
* allow building with old versions of exceptions before MonadMask was split outGravatar Joey Hess2014-05-28
|
* Use exceptions in place of deprecated MonadCatchIO-transformersGravatar Ben Gamari2014-05-28
|
* factor out getRemoteGitConfigGravatar Joey Hess2014-05-16
|
* fix build failure with flags -Webapp -QuviGravatar Fraser Tweedale2014-04-08
|
* reorg matcher types; no non-type code changesGravatar Joey Hess2014-03-29
|
* add desktop notificationsGravatar Joey Hess2014-03-22
| | | | | | | Motivation: Hook scripts for nautilus or other file managers need to provide the user with feedback that a file is being downloaded. This commit was sponsored by THM Schoemaker.
* clean up cleanup action enumerationGravatar Joey Hess2014-03-13
|
* Fix zombie leak and general inneficiency when copying files to a local git repo.Gravatar Joey Hess2014-03-06
| | | | | | Benchmarking this with 1000 small files being copied, the time reduced from 15.98s to 14.64s -- an 8% improvement in the non-data-transfer overhead of git-annex copy.
* pre-commit-annex hook script to automatically extract metadata from lots of ↵Gravatar Joey Hess2014-03-02
| | | | | | | | | | | | | | | | | | types of files Using the extract(1) program to do the heavy lifting. Decided to make git-annex run pre-commit-annex when committing. Since git-annex pre-commit also runs it, it'll be run when git commit is run too, via the pre-commit hook. This basically gives back the pre-commit hook that git-annex took away. The implementation avoids repeatedly looking for the hook script when the assistant is running and committing repeatedly; only checks if the hook is available once. To make the script simpler, made git-annex metadata -s field?=value only set a field when it's not already got a value. This commit was sponsored by bak.
* Probe for quvi version at run time.Gravatar Joey Hess2014-02-28
| | | | | Overhead: git annex addurl runs quvi --version once. And more bloat to Annex state..
* metacata command can now operate on many files at onceGravatar Joey Hess2014-02-13
|
* use locking on WindowsGravatar Joey Hess2014-01-28
| | | | This is all the easy cases, where there was already a separate lock file.
* add "unused" preferred content expressionGravatar Joey Hess2014-01-22
| | | | | | | With a really nice optimisation that keeps it from having any overhead in normal operation! This commit was sponsored by Ulises Vitulli.
* numcopies cleanup, part 2Gravatar Joey Hess2014-01-21
| | | | This includes several bug fixes.
* reorganize numcopies code (no behavior changes)Gravatar Joey Hess2014-01-21
| | | | | | | Move stuff into Logs.NumCopies. Add a NumCopies newtype. Better names for various serialization classes that are specific to one thing or another.
* global numcopies settingGravatar Joey Hess2014-01-20
| | | | | | | | | | | | | | | | | | | | | | | * numcopies: New command, sets global numcopies value that is seen by all clones of a repository. * The annex.numcopies git config setting is deprecated. Once the numcopies command is used to set the global number of copies, any annex.numcopies git configs will be ignored. * assistant: Make the prefs page set the global numcopies. This global numcopies setting is needed to let preferred content expressions operate on numcopies. It's also convenient, because typically if you want git-annex to preserve N copies of files in a repo, you want it to do that no matter which repo it's running in. Making it global avoids needing to warn the user about gotchas involving inconsistent annex.numcopies settings. (See changes to doc/numcopies.mdwn.) Added a new variety of git-annex branch log file, that holds only 1 value. Will probably be useful for other stuff later. This commit was sponsored by Nicolas Pouillard.
* fix inversion of control in CommandSeek (no behavior changes)Gravatar Joey Hess2014-01-20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | I've been disliking how the command seek actions were written for some time, with their inversion of control and ugly workarounds. The last straw to fix it was sync --content, which didn't fit the Annex [CommandStart] interface well at all. I have not yet made it take advantage of the changed interface though. The crucial change, and probably why I didn't do it this way from the beginning, is to make each CommandStart action be run with exceptions caught, and if it fails, increment a failure counter in annex state. So I finally remove the very first code I wrote for git-annex, which was before I had exception handling in the Annex monad, and so ran outside that monad, passing state explicitly as it ran each CommandStart action. This was a real slog from 1 to 5 am. Test suite passes. Memory usage is lower than before, sometimes by a couple of megabytes, and remains constant, even when running in a large repo, and even when repeatedly failing and incrementing the error counter. So no accidental laziness space leaks. Wall clock speed is identical, even in large repos. This commit was sponsored by an anonymous bitcoiner.
* improve matcher data type to allow matching Keys, instead of just files (no ↵Gravatar Joey Hess2014-01-18
| | | | behavior changes)
* fix reversion in relative paths to local remotes of direct mode reposGravatar Joey Hess2013-11-26
| | | | | | 52ad9a1528bc51f65411aca263def97615367943 broke support for local remotes from direct mode repos, because the relative path was taken to be from the gitdir, rather than from the work tree.
* restart on upgrade now fully workingGravatar Joey Hess2013-11-22
|
* fix standalone build of this moduleGravatar Joey Hess2013-11-22
|
* automatically set and unset core.bare when switching to/from direct modeGravatar Joey Hess2013-11-05
|
* support direct mode repositories with core.bare=true (not yet default)Gravatar Joey Hess2013-11-05
| | | | | | | | | | | Direct mode repositories can now have core.bare=true set, to prevent accidentally running git commands that try to operate on the work tree, and so do the wrong thing. This is not yet the default, and it causes known problems for git-annex sync due to receive.denyCurrentBranch not working in bare repositories. This commit was sponsored by Richard Hartmann.
* cleanupGravatar Joey Hess2013-10-18
|
* Send a git-annex user-agent when downloading urls.Gravatar Joey Hess2013-09-28
| | | | | | | | | Overridable with --user-agent option. Not yet done for S3 or WebDAV due to limitations of libraries used -- nether allows a user-agent header to be specified. This commit sponsored by Michael Zehrer.
* gitignore support for the assistant and watcherGravatar Joey Hess2013-08-02
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Requires git 1.8.4 or newer. When it's installed, a background git check-ignore process is run, and used to efficiently check ignores whenever a new file is added. Thanks to Adam Spiers, for getting the necessary support into git for this. A complication is what to do about files that are gitignored but have been checked into git anyway. git commands assume the ignore has been overridden in this case, and not need any more overriding to commit a changed version. However, for the assistant to do the same, it would have to run git ls-files to check if the ignored file is in git. This is somewhat expensive. Or it could use the running git-cat-file process to query the file that way, but that requires transferring the whole file content over a pipe, so it can be quite expensive too, for files that are not git-annex symlinks. Now imagine if the user knows that a file or directory tree will be getting frequent changes, and doesn't want the assistant to sync it, so gitignores it. The assistant could overload the system with repeated ls-files checks! So, I've decided that the assistant will not automatically commit changes to files that are gitignored. This is a tradeoff. Hopefully it won't be a problem to adjust .gitignore settings to not ignore files you want the assistant to autocommit, or to manually git annex add files that are listed in .gitignore. (This could be revisited if git-annex gets access to an interface to check the content of the index w/o forking a git command. This could be libgit2, or perhaps a separate git cat-file --batch-check process, so it wouldn't need to ship over the whole file content.) This commit was sponsored by Francois Marier. Thanks!
* Make --numcopies override annex.numcopies set in .gitattributes.Gravatar Joey Hess2013-07-09
|
* assistant: Work around git-cat-file's not reloading the index after files ↵Gravatar Joey Hess2013-05-25
| | | | | | are staged. Argh.
* refactorGravatar Joey Hess2013-05-24
|
* Switch to MonadCatchIO-transformers for better handling of state while ↵Gravatar Joey Hess2013-05-19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | catching exceptions. As seen in this bug report, the lifted exception handling using the StateT monad throws away state changes when an action throws an exception. http://git-annex.branchable.com/bugs/git_annex_fork_bombs_on_gpg_file/ .. Which can result in cached values being redundantly calculated, or other possibly worse bugs when the annex state gets out of sync with reality. This switches from a StateT AnnexState to a ReaderT (MVar AnnexState). All changes to the state go via the MVar. So when an Annex action is running inside an exception handler, and it makes some changes, they immediately go into affect in the MVar. If it then throws an exception (or even crashes its thread!), the state changes are still in effect. The MonadCatchIO-transformers change is actually only incidental. I could have kept on using lifted-base for the exception handling. However, I'd have needed to write a new instance of MonadBaseControl for the new monad.. and I didn't write the old instance.. I begged Bas and he kindly sent it to me. Happily, MonadCatchIO-transformers is able to derive a MonadCatchIO instance for my monad. This is a deep level change. It passes the test suite! What could it break? Well.. The most likely breakage would be to code that runs an Annex action in an exception handler, and *wants* state changes to be thrown away. Perhaps the state changes leaves the state inconsistent, or wrong. Since there are relatively few places in git-annex that catch exceptions in the Annex monad, and the AnnexState is generally just used to cache calculated data, this is unlikely to be a problem. Oh yeah, this change also makes Assistant.Types.ThreadedMonad a bit redundant. It's now entirely possible to run concurrent Annex actions in different threads, all sharing access to the same state! The ThreadedMonad just adds some extra work on top of that, with its own MVar, and avoids such actions possibly stepping on one-another's toes. I have not gotten rid of it, but might try that later. Being able to run concurrent Annex actions would simplify parts of the Assistant code.
* start one git-cat-file per index fileGravatar Joey Hess2013-05-15
| | | | | | | This reverts a5031031f0d596b2381a785925beb574d90a862e and properly fixes the issue discussed there. This makes git-annex behave much nicer in direct mode.
* turn on PackageImports globallyGravatar Joey Hess2013-04-13
| | | | | | | This will make it easier to use the Evil Splicer, when it needs to add package qualified imports And there's no real downside.