summaryrefslogtreecommitdiff
path: root/Git
Commit message (Collapse)AuthorAge
* avoid the dashed ssh hostname class of security holesGravatar Joey Hess2017-08-17
| | | | | | | | | | | | | | | | | | | | | | | | Security fix: Disallow hostname starting with a dash, which would get passed to ssh and be treated an option. This could be used by an attacker who provides a crafted ssh url (for eg a git remote) to execute arbitrary code via ssh -oProxyCommand. No CVE has yet been assigned for this hole. The same class of security hole recently affected git itself, CVE-2017-1000117. Method: Identified all places where ssh is run, by git grep '"ssh"' Converted them all to use a SshHost, if they did not already, for specifying the hostname. SshHost was made a data type with a smart constructor, which rejects hostnames starting with '-'. Note that git-annex already contains extensive use of Utility.SafeCommand, which fixes a similar class of problem where a filename starting with a dash gets passed to a program which treats it as an option. This commit was sponsored by Jochen Bartl on Patreon.
* fix failing quickcheck propertiesGravatar Joey Hess2017-06-17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | QuickCheck 2.10 found a counterexample eg "\929184" broke the property. As far as I can tell, Git.Filename is matching how git handles encoding of strange high unicode characters in filenames for display. Git does not display high unicode characters, and instead displays the C-style escaped form of each byte. This is ambiguous, but since git is not unicode aware, it doesn't need to roundtrip parse it. So, making Git.FileName's roundtrip test only chars < 256 seems fine. Utility.Format.format uses encode_c, in order to mimic git, so that's ok. Utility.Format.gen uses decode_c, but only so that stuff like "\n" in the format string is handled. If the format string contains C-style octal escapes, they will be converted to ascii characters, and not combined into unicode characters, but that should not be a problem. If the user wants unicode characters, they can include them in the format string, without escaping them. Finally, decode_c is used by Utility.Gpg.secretKeys, because gpg --with-colons hex-escapes some characters in particular ':' and '\\'. gpg passes unicode through, so this use of decode_c is not a problem. This commit was sponsored by Henrik Riomar on Patreon.
* adeiu, MissingHGravatar Joey Hess2017-05-16
| | | | | | | | | | | | | | | | Removed dependency on MissingH, instead depending on the split library. After laying groundwork for this since 2015, it was mostly straightforward. Added Utility.Tuple and Utility.Split. Eyeballed System.Path.WildMatch while implementing the same thing. Since MissingH's progress meter display was being used, I re-implemented my own. Bonus: Now progress is displayed for transfers of files of unknown size. This commit was sponsored by Shane-o on Patreon.
* fix GIT_SSH_COMMAND -n parameterGravatar Joey Hess2017-03-20
| | | | | | | | | It was being passed to sh, not to the command, oops. Noticed because it broke the test suite on OSX, where sh -n silently does nothing. Would also break on Linux when eg posh was being used as the shell; bash ignores the -n. This commit was supported by the NSF-funded DataLad project.
* super tricky shell command generation hackGravatar Joey Hess2017-03-17
| | | | | | | | | | GIT_SSH_COMMAND was not working correctly with git-annex get, because when used in rsync -e, there were additional parameters appended at the end, which the GIT_SSH_COMMAND should not see. Fixed by constructing the shell command differently. This commit was supported by the NSF-funded DataLad project.
* fix over-shell-escapeGravatar Joey Hess2017-03-17
| | | | Seems I had one time too many.
* Support GIT_SSH and GIT_SSH_COMMANDGravatar Joey Hess2017-03-17
| | | | | | | | | | | | | | | | | | | | They are handled close the same as they are by git. However, unlike git, git-annex sometimes needs to pass the -n parameter when using these. So, this has the potential for breaking some setup, and perhaps there ought to be a ANNEX_USE_GIT_SSH=1 needed to use these. But I'd rather avoid that if possible, so let's see if anyone complains. Almost all places where "ssh" was run have been changed to support the env vars. Anything still calling sshOptions does not support them. In particular, rsync special remotes don't. Seems that annex-rsync-transport already gives sufficient control there. (Fixed in passing: Remote.Helper.Ssh.toRepo used to extract remoteAnnexSshOptions and pass them to sshOptions, which was redundant since sshOptions also extracts those.) This commit was sponsored by Jeff Goeke-Smith on Patreon.
* status: Pass --ignore-submodules=when option on to git status.Gravatar Joey Hess2017-02-20
| | | | | | | Didn't make --ignore-submodules without a value be handled because I can't see a way to make optparse-applicative parse that. I've opened a bug requesting a way to do that: https://github.com/pcapriotti/optparse-applicative/issues/243
* adjust: Fix behavior when used in a repository that contains submodules.Gravatar Joey Hess2017-02-20
| | | | | Also fixed the LsFiles parser to not assume its output has a fixed width type field.
* post-recive hook to make updateInstead work in direct mode and adjusted branchesGravatar Joey Hess2017-02-17
| | | | | | | | * Added post-recieve hook, which makes updateInstead work with direct mode and adjusted branches. * init: Set up the post-receive hook. This commit was sponsored by Fernando Jimenez on Patreon.
* unused: Improved memory use significantly when there are a lot of ↵Gravatar Joey Hess2017-01-31
| | | | | | | | | | | differences between branches. Argh, didn't need an accumulator here! I think I use accumulators a lot more than I need to when recusively processing lists.. This commit was sponsored by Jeff Goeke-Smith on Patreon.
* Some optimisations to string splitting code.Gravatar Joey Hess2017-01-31
| | | | | | | | | | | | | | | | | | | Turns out that Data.List.Utils.split is slow and makes a lot of allocations. Here's a much simpler single character splitter that behaves the same (even in wacky corner cases) while running in half the time and 75% the allocations. As well as being an optimisation, this helps move toward eliminating use of missingh. (Data.List.Split.splitOn is nearly as slow as Data.List.Utils.split and allocates even more.) I have not benchmarked the effect on git-annex, but would not be surprised to see some parsing of eg, large streams from git commands run twice as fast, and possibly in less memory. This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
* fix lack of laziness streaming large diffsGravatar Joey Hess2017-01-31
| | | | | | | | A commit last year that made a partial function use Maybe unfortunately caused the whole input to need to be consumed, breaking streaming. So, revert it. This commit was sponsored by Nick Daly on Patreon.
* Always use filesystem encoding for all file and handle reads and writes.Gravatar Joey Hess2016-12-24
| | | | | This is a big scary change. I have convinced myself it should be safe. I hope!
* Avoid backtraces on expected failures when built with ghc 8; only use ↵Gravatar Joey Hess2016-11-15
| | | | | | | | | | | | | backtraces for unexpected errors. ghc 8 added backtraces on uncaught errors. This is great, but git-annex was using error in many places for a error message targeted at the user, in some known problem case. A backtrace only confuses such a message, so omit it. Notably, commands like git annex drop that failed due to eg, numcopies, used to use error, so had a backtrace. This commit was sponsored by Ethan Aubin.
* restart coprocess in raw modeGravatar Joey Hess2016-11-01
| | | | | | | | | | | Restarting a crashing git process could result in filename encoding issues when not in a unicode locale, as the restarted processes's handles were not read in raw mode. Since rawMode is always used when starting a coprocess, didn't bother to parameterise it and just always enable it for simplicity. This commit was sponsored by Jake Vosloo on Patreon.
* Fix reversion in 6.20161012 that prevented adding files with a space in ↵Gravatar Joey Hess2016-10-31
| | | | their name.
* Assistant, repair: Fix ignoring of git fsck errors due to duplicate file ↵Gravatar Joey Hess2016-10-31
| | | | entries in tree objects.
* Assistant, repair: Improved filtering out of git fsck lines about duplicate ↵Gravatar Joey Hess2016-10-18
| | | | file entries in tree objects.
* fix tree graft-in bugGravatar Joey Hess2016-10-11
| | | | | | | | | | | When adding a tree like a/b/c/d when a/b already exists, fixes the bug that the tree that got created was a/b/a/b/c/d Just need to flatten out the top N directories of the tree that's being grafted in, so we get the c/d part. This was complicated by the Tree data type being a rose tree rather than a regular tree. This commit was sponsored by Nick Daly on Patreon.
* sync: Fix bug in adjusted branch merging that could cause recently added ↵Gravatar Joey Hess2016-10-10
| | | | | | | | | | | files to be lost when updating the adjusted branch. The modification flag was not being set when making modifications deep in a tree, so parent trees were not updated to contain the modified tree. Seems to have exposed another bug where the wrong filename gets grafted in. This commit was sponsored by Brock Spratlen on Patreon.
* Avoid using a lot of memory when large objects are present in the git repositoryGravatar Joey Hess2016-10-05
| | | | | | | | | | | | | | | | | | | | | | .. and have to be checked to see if they are a pointed to an annexed file. Cases where such memory use could occur included, but were not limited to: - git commit -a of a large unlocked file (in v5 mode) - git-annex adjust when a large file was checked into git directly Generally, any use of catKey was a potential problem. Fix by using git cat-file --batch-check to check size before catting. This adds another git batch process, which is included in the CatFileHandle for simplicity. There could be performance impact, anywhere catKey is used. Particularly likely to affect adjusted branch generation speed, and operations on unlocked files in v6 mode. Hopefully since the --batch-check and --batch read the same data, disk buffering will avoid most overhead. Leaving only the overhead of talking to the process over the pipe and whatever computation --batch-check needs to do. This commit was sponsored by Bruno BEAUFILS on Patreon.
* Optimisations to git-annex branch query and setting, avoiding repeated ↵Gravatar Joey Hess2016-09-29
| | | | | | | | | | | | | | | | | | | | | | copies of the environment. Speeds up commands like "git-annex find --in remote" by over 50%. Profiling showed that adjustGitEnv was 21% of the time and 37% of the allocations of that command. It copied the environment each time with getEnvironment. The only repeated use of adjustGitEnv is in withIndexFile, which tends to be run at least once per file. So, it was optimised by keeping a cache of the environment, which can be reused. There could be other better ways to optimise this. Maybe get the while environment once at startup. But, then it would have to be serialized back out each time running a child process, so I doubt that would be a net win. It might be better to cache a version of the environment that is pre-modified to use .git-annex/index. But, profiling doesn't show that modifying the enviroment is taking any significant time.
* fix bugs in handing of deep branches with sync and adjusted branchesGravatar Joey Hess2016-09-21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * sync: Previously, when run in a branch with a slash in its name, such as "foo/bar", the sync branch was "synced/bar". That conflicted with the sync branch used for branch "bar", so has been changed to "synced/foo/bar". * adjust: Previously, when adjusting a branch with a slash in its name, such as "foo/bar", the adjusted branch was "adjusted/bar(unlocked)". That conflicted with the adjusted branch used for branch "bar", so has been changed to "adjusted/foo/bar(unlocked)" * Also, running sync in an adjusted branch did not correctly sync changes back to the parent branch when it had a slash in its name. This bug has been fixed. Eliminate use of Git.Ref.under and Git.Ref.basename; using Git.Ref.underBase and Git.Ref.base make everything handle deep branches correctly. Probably noone was adjusting deep branches, and v6 is still experimental anyway, so I'm not going to worry about the mess that was left by that bug. In the case of git-annex sync, using a fixed git-annex with an old unfixed one will mean they use different sync branches for a deep branch, and so they may stop syncing until the old one is upgraded. However, that's only a problem when syncing between repositories without going via a central bare repository. Added a warning about this to the CHANGELOG, but it's probably not going to affect many people at all. This commit was sponsored by Riku Voipio.
* improve commentGravatar Joey Hess2016-09-21
|
* Assistant, repair: Filter out git fsck lines about duplicate file entries in ↵Gravatar Joey Hess2016-09-05
| | | | tree objects.
* Windows: Handle shebang in external special remote program.Gravatar Joey Hess2016-09-05
|
* --branch, stage 2Gravatar Joey Hess2016-07-20
| | | | | | | | Show branch:file that is being operated on. I had to make ActionItem a type and not a type class because withKeyOptions' passed two different types of values when using the type class, and I could not get the type checker to accept that.
* New url for git-remote-gcrypt, now maintained by spwhitton.Gravatar Joey Hess2016-07-05
|
* Pass -S to git commit-tree when commit.gpgsign is set and when making a ↵Gravatar Joey Hess2016-06-02
| | | | non-automatic commit, in order to preserve current behavior when used with git 1.9, which has stopped doing this itself.
* minor typo fixes throughoutGravatar Yaroslav Halchenko2016-06-02
| | | | | problematic flexibility
* Windows: Avoid terminating git-annex branch lines with \r\n when union merging.Gravatar Joey Hess2016-05-27
|
* git's handing of relative GIT_INDEX_FILE is more insane than I thought; ↵Gravatar Joey Hess2016-05-22
| | | | | | | | | | | | | | | always make absolute This is actually worse than I thought; when git is being run with a detached work tree, GIT_INDEX_FILE is treated as a path relative to CWD, instead of the normal behavior of relative the top of the work tree. This seems to make it basically impossible for any program that wants to use GIT_INDEX_FILE to use anything other than an absolute path to it; there are too many configurations to keep straight that can change how git interprets what should be a simple relative path to a file. (I have complained to the git developers.)
* unify handling of unusual GIT_INDEX_FILE relative pathGravatar Joey Hess2016-05-17
| | | | This is probably a git bug that stuck in its interface.
* Work around git bug in handling of relative path to GIT_INDEX_FILE when in a ↵Gravatar Joey Hess2016-05-17
| | | | | | | | subdirectory of the repository. This affected git annex view. It turns out that some other places that use GIT_INDEX_FILE were already working around the bug. I removed the workaround from Annex.Branch since the new workaround will do.
* Propigate GIT_DIR and GIT_WORK_TREE environment to external special remotes.Gravatar Joey Hess2016-05-06
| | | | | | | | | Since git-annex unsets these when started, they have to be explicitly propigated. Also, this makes --git-dir and --work-tree settings be reflected in the environment. The need for this came up in https://github.com/DanielDent/git-annex-remote-rclone/issues/3
* more windows path fixesGravatar Joey Hess2016-05-04
| | | | | Let git-style filepaths be looked up in the removeset, even though windows-style filepaths are probably being fed into it.
* more windows path fixesGravatar Joey Hess2016-05-04
| | | | | beneathSubTree can be called with both windows-style and git-style paths, so needs to normalize to windows-style.
* avoid hardcoded slashes; broke on windowsGravatar Joey Hess2016-05-03
|
* --allow-unrelated-histories will be in git 2.9.0, not 2.8.2Gravatar Joey Hess2016-04-27
| | | | Also, I had the logic backwards in the version comparison.
* use --allow-unrelated-histories for nowGravatar Joey Hess2016-04-22
| | | | | | | | I'd prefer to use the env var, but let's use what git currently supports. Revert this when the env var gets supported. Note that the version checking assumes git 2.8.2 will get support for the switch.
* assistant: Deal with upcoming git's refusal to merge unrelated histories by ↵Gravatar Joey Hess2016-04-22
| | | | | | | | | | | | | default git 2.8.1 (or perhaps 2.9.0) is going to prevent git merge from merging in unrelated branches. Since the webapp's pairing etc features often combine together repositories with unrelated histories, work around this behavior change by setting GIT_MERGE_ALLOW_UNRELATED_HISTORIES when the assistant merges. Note though that this is not done for git annex sync's merges, so it will follow git's default or configured behavior.
* fix build with ghc 7.6.3Gravatar Joey Hess2016-04-08
|
* avoid withWorkTreeRelated affecting annex symlink calculationGravatar Joey Hess2016-04-08
|
* run out of tree merge with --no-ffGravatar Joey Hess2016-04-06
| | | | | | This is how direct mode does it too, and somehow, for reasons that currently escape me, this makes git merge not care if it's run with an empty work tree.
* sync: Show output of git commit.Gravatar Joey Hess2016-04-05
| | | | | | Rationalle: User might have hook scripts whose output they want to see. Also, git commit output may tell the user they forgot to add a file. The output is not too ugly when there's nothing to commit.
* Upgrading a direct mode repository to v6 has changed to enter an adjusted ↵Gravatar Joey Hess2016-04-04
| | | | | | | | unlocked branch. This makes the direct mode to v6 upgrade able to be performed in one clone of a repository without affecting other clones, which can continue using v5 and direct mode.
* fix parsing of commit with no parentsGravatar Joey Hess2016-03-31
|
* add reflog messagesGravatar Joey Hess2016-03-31
|
* Merge branch 'master' into adjustedbranchGravatar Joey Hess2016-03-29
|\