summaryrefslogtreecommitdiff
path: root/Annex
Commit message (Collapse)AuthorAge
* better readProcessGravatar Joey Hess2012-07-19
|
* add back debug loggingGravatar Joey Hess2012-07-19
| | | | | | | | | | | | | Make Utility.Process wrap the parts of System.Process that I use, and add debug logging to them. Also wrote some higher-level code that allows running an action with handles to a processes stdin or stdout (or both), and checking its exit status, all in a single function call. As a bonus, the debug logging now indicates whether the process is being run to read from it, feed it data, chat with it (writing and reading), or just call it for its side effect.
* switch from System.Cmd.Utils to System.ProcessGravatar Joey Hess2012-07-18
| | | | | | | | | | | | | | | | | | Test suite now passes with -threaded! I traced back all the hangs with -threaded to System.Cmd.Utils. It seems it's just crappy/unsafe/outdated, and should not be used. System.Process seems to be the cool new thing, so converted all the code to use it instead. In the process, --debug stopped printing commands it runs. I may try to bring that back later. Note that even SafeSystem was switched to use System.Process. Since that was a modified version of code from System.Cmd.Utils, it needed to be converted too. I also got rid of nearly all calls to forkProcess, and all calls to executeFile, which I'm also doubtful about working well with -threaded.
* more debuggingGravatar Joey Hess2012-07-18
|
* avoid ByteString.Char8 where not neededGravatar Joey Hess2012-06-20
| | | | | Its truncation behavior is a red flag, so avoid using it in these places where only raw ByteStrings are used, without looking at the data inside.
* fishy commitGravatar Joey Hess2012-06-14
|
* hlintGravatar Joey Hess2012-06-12
|
* crazy optimisationGravatar Joey Hess2012-06-10
| | | | Crazy like a fox..
* queue size fixGravatar Joey Hess2012-06-10
| | | | | Increase queue size for update-index actions, because otherwise they'll never be flushed.
* refactor and function name cleanupGravatar Joey Hess2012-06-08
| | | | (oops, I had a calcMerge and a calc_merge!)
* make watch use the queueGravatar Joey Hess2012-06-07
| | | | | May not work. Certianly needs to flush the queue from time to time when only symlink changes are being made.
* extend Git.Queue to be able to queue more than simple git commandsGravatar Joey Hess2012-06-07
| | | | | | While I was in there, I noticed and fixed a bug in the queue size calculations. It was never encountered only because Queue.add was only ever run with 1 file in the list.
* close the git add raceGravatar Joey Hess2012-06-06
| | | | | | | | | | | | | | | | | There's a race adding a new file to the annex: The file is moved to the annex and replaced with a symlink, and then we git add the symlink. If someone comes along in the meantime and replaces the symlink with something else, such as a new large file, we add that instead. Which could be bad.. This race is fixed by avoiding using git add, instead the symlink is directly staged into the index. It would be nice to make `git annex add` use this same technique. I have not done so yet because it currently runs git update-index once per file, which would slow does `git annex add`. A future enhancement would be to extend the Git.Queue to include the ability to run update-index with a list of Streamers.
* factor out nukeFileGravatar Joey Hess2012-06-06
|
* Merge branch 'master' into watchGravatar Joey Hess2012-06-06
|\
| * factor out generic update-index code from unionmerge codeGravatar Joey Hess2012-06-06
| |
* | flush the git queue when a new type of action is being added to itGravatar Joey Hess2012-06-04
|/ | | | | | | | This allows the queue to be used in a single process for multiple possibly conflicting commands, like add and rm, without running them out of order. This assumes that running the same git subcommand with different parameters cannot itself conflict.
* Clean up handling of git directory and git worktree.Gravatar Joey Hess2012-05-18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Baked into the code was an assumption that a repository's git directory could be determined by adding ".git" to its work tree (or nothing for bare repos). That fails when core.worktree, or GIT_DIR and GIT_WORK_TREE are used to separate the two. This was attacked at the type level, by storing the gitdir and worktree separately, so Nothing for the worktree means a bare repo. A complication arose because we don't learn where a repository is bare until its configuration is read. So another Location type handles repositories that have not had their config read yet. I am not entirely happy with this being a Location type, rather than representing them entirely separate from the Git type. The new code is not worse than the old, but better types could enforce more safety. Added support for core.worktree. Overriding it with -c isn't supported because it's not really clear what to do if a git repo's config is read, is not bare, and is then overridden to bare. What is the right git directory in this case? I will worry about this if/when someone has a use case for overriding core.worktree with -c. (See Git.Config.updateLocation) Also removed and renamed some functions like gitDir and workTree that misused git's terminology. One minor regression is known: git annex add in a bare repository does not print a nice error message, but runs git ls-files in a way that fails earlier with a less nice error message. This is because before --work-tree was always passed to git commands, even in a bare repo, while now it's not.
* Fix use of several config settingsGravatar Joey Hess2012-05-05
| | | | | | | annex.ssh-options, annex.rsync-options, annex.bup-split-options. And adjust types to avoid the bugs that broke several config settings recently. Now "annex." prefixing is enforced at the type level.
* display "Recording state in git..." when staging the journalGravatar Joey Hess2012-04-27
| | | | | | | | A bit tricky to avoid printing it twice in a row when there are queued git commands to run and journal to stage. Added a generic way to run an action that may output multiple side messages, with only the first displayed.
* uninit: Clear annex.uuid from .git/config. Closes: #670639Gravatar Joey Hess2012-04-27
|
* Add annex.httpheaders and annex.httpheader-command config settingsGravatar Joey Hess2012-04-22
| | | | | | Allow custom headers to be sent with all HTTP requests. (Requested by the Internet Archive)
* noopGravatar Joey Hess2012-04-21
|
* in which I discover voidGravatar Joey Hess2012-04-21
| | | | void :: Functor f => f a -> f () -- ah, of course that's useful :)
* cache parsed core.sharedrepositoryGravatar Joey Hess2012-04-21
|
* honor core.sharedRepository when making all the other files in the annexGravatar Joey Hess2012-04-21
| | | | Lock files, directories, etc.
* better file mode setting codeGravatar Joey Hess2012-04-21
|
* Support git's core.sharedRepository configurationGravatar Joey Hess2012-04-21
| | | | | | This is incomplete, it does not honor it yet for hash directories and other annex bookkeeping files. Some of that is not needed for a bare repo; some of it may be.
* inverted logicGravatar Joey Hess2012-04-20
|
* export a more generalized checkDiskSpaceGravatar Joey Hess2012-04-20
|
* Rewrote free disk space checking codeGravatar Joey Hess2012-03-22
| | | | | Moving the portability handling into a small C library cleans up things a lot, avoiding the pain of unpacking structs from inside haskell code.
* use new getConfigGravatar Joey Hess2012-03-22
|
* rationalize getConfigGravatar Joey Hess2012-03-22
| | | | | | | | | | getConfig got a remote-specific config, and this confusing name caused it to be used a couple of places that only were interested in global configs. Rename to getRemoteConfig and make getConfig only get global configs. There are no behavior changes here, but remote.<name>.annex-web-options never actually worked (and per-remote web options is a very unlikely to be useful case so I didn't make it work), so fix the documentation for it.
* status: Prints available local disk space, or shows if git-annex doesn't know.Gravatar Joey Hess2012-03-21
|
* Improve detection of inability to check free disk space.Gravatar Joey Hess2012-03-21
| | | | | | | | Don't check if configure indicated checks won't work. This should fix a FTBFS on mipsel, where configure correctly detects the checks won't work, while garbage is returned for disk space info at git-annex runtime. It also means that, when built via cabal, disk space checks are not enabled, unfortunatly.
* added ifM and nuked 11 lines of codeGravatar Joey Hess2012-03-14
| | | | no behavior changes
* getKeysPresent is now fully lazyGravatar Joey Hess2012-03-11
| | | | | | | | | | | | .. Allowing it to be used by things in constant space! Random statistics: git annex status has gone from taking 239 mb of memory and 26 seconds in a repo, to 8 mb and 13 seconds. The trick here is the unsafeInterleaveIO, and the form of the function's recursion, which I cribbed heavily from System.IO.HVFS.Utils.recurseDirStat. The difference is, this one goes to a limited depth and avoids statting everything.
* status: Fixed to run in nearly constant space.Gravatar Joey Hess2012-03-11
| | | | | | | | Before, it leaked space due to caching lists of keys. Now all necessary data about keys is calculated as they stream in. The "nearly constant" is due to getKeysPresent, which builds up a lot of [] thunks as it traverses .git/annex/objects/. Will deal with it later.
* syscall optimisationGravatar Joey Hess2012-03-06
|
* configure: Check if ssh connection caching is supported by the installed ↵Gravatar Joey Hess2012-02-25
| | | | version of ssh and default annex.sshcaching accordingly.
* improve alwayscommit=false modeGravatar Joey Hess2012-02-25
| | | | | | | | | | | | | | Now changes are staged into the branch's index, but not committed, which avoids growing a large journal. And sync and merge always explicitly commit, ensuring that even when they do nothing else, they commit the staged changes. Added a flag file to indicate that the branch's journal contains uncommitted changes. (Could use git ls-files, but don't want to run that every time.) In the future, this ability to have uncommitted changes staged in the journal might be used on remotes after a series of oneshot commands.
* add annex.alwayscommit optionGravatar Joey Hess2012-02-25
| | | | | | To avoid commits of data to the git-annex branch after each command is run, set annex.alwayscommit=false. Its data will then be committed less frequently, when a merge or sync is done.
* Deal with NFS problem that caused a failure to remove a directory when ↵Gravatar Joey Hess2012-02-24
| | | | | | | | | | | | | | removing content from the annex. I was able to reproduce this on linux using the kernel's nfs server and mounting localhost:/. Determined that removing the directory fails when the just-deleted file in it was locked. Considered dropping the lock before removing the directory, but this would complicate parts of the code that should not need to worry about locking. So instead, ignore the failure to remove the directory in this case. While I was at it, made it attempt to remove both levels of hash directories, in case they're empty.
* hlintGravatar Joey Hess2012-02-16
|
* Added a annex.queuesize settingGravatar Joey Hess2012-02-15
| | | | | | | | | | useful when adding hundreds of thousands of files on a system with plenty of memory. git add gets quite slow in such a large repository, so if the system has more than the ~32 mb of memory the queue can use by default, it's a useful optimisation to increase the queue size, in order to decrease the number of times git add is run.
* tweakGravatar Joey Hess2012-02-14
|
* fix memory leak when staging the journalGravatar Joey Hess2012-02-14
| | | | | | The list of files had to be retained until the end so it could be deleted. Also, a list of update-index lines was generated and only then fed into it. Now everything streams in constant space.
* Fixed a memory leak due to excessive strictness when committing journal files.Gravatar Joey Hess2012-02-14
| | | | | | When hashing the files, the entire list of shas was read strictly. That was entirely unnecessary, since there's a cleanup action run after they're consumed.
* rework git check-attr interfaceGravatar Joey Hess2012-02-13
| | | | | | | | | | | | | | | Now gitattributes are looked up, efficiently, in only the places that really need them, using the same approach used for cat-file. The old CheckAttr code seemed very fragile, in the way it streamed files through git check-attr. I actually found that cad8824852aa0623dc41eac02a9e2bae47d88ec4 was still deadlocking with ghc 7.4, at the end of adding a lot of files. This should fix that problem, and avoid future ones. The best part is that this removes withAttrFilesInGit and withNumCopies, which were complicated Seek methods, as well as simplfying the types for several other Seek methods that had a Backend tupled in.
* Fix teardown of stale cached ssh connections.Gravatar Joey Hess2012-02-09
|