summaryrefslogtreecommitdiff
path: root/Command
Commit message (Collapse)AuthorAge
* version: Added "dependency versions" line.Gravatar Joey Hess2017-04-07
| | | | This commit was sponsored by Anthony DeRobertis on Patreon.
* git annex add -u now supported, analagous to git add -uGravatar Joey Hess2017-04-07
| | | | | | | | | | | | Unlike git add -u, git annex add -u does not update the index for files removed from the working tree. But then, "git add ." stages removals, and "git annex add ." does not, so that's an existing divergence. Seems that --update --batch would need to run git ls-files once per line of batch input, which would surely be too slow, so just throw an error for that. This commit was supported by the NSF-funded DataLad project.
* enableremote: Fix re-enabling of existing gcrypt remotes, so that eg, ↵Gravatar Joey Hess2017-04-07
| | | | | | | | | | | | encryption key changes take effect. They were silently ignored, a reversion introduced in 6.20160527. I don't like this regular git remote special case in enableremote, but I can't see a way to get rid of it. So, check if the existing remote is a Remote.Git This commit was sponsored by Trenton Cronholm on Patreon.
* enableremote: When enabling a non-special remote, param=value parameters ↵Gravatar Joey Hess2017-04-07
| | | | | | can't be used, so error out if any are provided. This commit was sponsored by Riku Voipio.
* Added remote.<name>.annex-push and remote.<name>.annex-pullGravatar Joey Hess2017-04-05
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The former can be useful to make remotes that don't get fully synced with local changes, which comes up in a lot of situations. The latter was mostly added for symmetry, but could be useful (though less likely to be). Implementing `remote.<name>.annex-pull` was a bit tricky, as there's no one place where git-annex pulls/fetches from remotes. I audited all instances of "fetch" and "pull". A few cases were left not checking this config: * Git.Repair can try to pull missing refs from a remote, and if the local repo is corrupted, that seems a reasonable thing to do even though the config would normally prevent it. * Assistant.WebApp.Gpg and Remote.Gcrypt and Remote.Git do fetches as part of the setup process of a remote. The config would probably not be set then, and having the setup fail seems worse than honoring it if it is already set. I have not prevented all the code that does a "merge" from merging branches from remotes with remote.<name>.annex-pull=false. That could perhaps be done, but it would need a way to map from branch name to remote name, and the way refspecs work makes that hard to get really correct. So if the user fetches manually, the git-annex branch will get merged, for example. Anther way of looking at/justifying this is that the setting is called "annex-pull", not "annex-merge". This commit was supported by the NSF-funded DataLad project.
* fix windows buildGravatar Joey Hess2017-04-05
|
* add missing "do"Gravatar Joey Hess2017-04-03
| | | | Unsure how it got committed in an uncompilable state before..
* multicast: New command, uses uftp to multicast annexed files, for eg a ↵Gravatar Joey Hess2017-03-30
| | | | | | classroom setting. This commit was supported by the NSF-funded DataLad project.
* sync --content-of=pathGravatar Joey Hess2017-03-20
| | | | | | | For when you want to sync only some files' contents, not the whole working tree. This commit was sponsored by Anthony DeRobertis on Patreon.
* Support GIT_SSH and GIT_SSH_COMMANDGravatar Joey Hess2017-03-17
| | | | | | | | | | | | | | | | | | | | They are handled close the same as they are by git. However, unlike git, git-annex sometimes needs to pass the -n parameter when using these. So, this has the potential for breaking some setup, and perhaps there ought to be a ANNEX_USE_GIT_SSH=1 needed to use these. But I'd rather avoid that if possible, so let's see if anyone complains. Almost all places where "ssh" was run have been changed to support the env vars. Anything still calling sshOptions does not support them. In particular, rsync special remotes don't. Seems that annex-rsync-transport already gives sufficient control there. (Fixed in passing: Remote.Helper.Ssh.toRepo used to extract remoteAnnexSshOptions and pass them to sshOptions, which was redundant since sshOptions also extracts those.) This commit was sponsored by Jeff Goeke-Smith on Patreon.
* Drop support for building with old versions of directory, feed, and http-types.Gravatar Joey Hess2017-03-10
|
* minor cleanupGravatar Joey Hess2017-03-10
|
* use ActionItem rather than StringGravatar Joey Hess2017-03-10
| | | | | This changes fsck -A warnings to include the name of the key, which is a bit redundant in one case, but was missing in another case.
* AssociatedFile newtypeGravatar Joey Hess2017-03-10
| | | | | | To prevent any further mistakes like 1a497cefb47557f0b4788c606f9071be422b2511 This commit was sponsored by Francois Marier on Patreon.
* fix fsck bug introduced in 1a497cefb47557f0b4788c606f9071be422b2511Gravatar Joey Hess2017-03-10
| | | | | Got two Maybe FilePaths crossed. Test suite caught it. Slightly improved types to avoid this mistake.
* fsck -q: When a file has bad content, include the name of the file in the ↵Gravatar Joey Hess2017-03-08
| | | | | | warning message. This commit was sponsored by Alexander Thompson on Patreon.
* status: Propigate nonzero exit code from git status.Gravatar Joey Hess2017-03-02
|
* improve display of checking known urlsGravatar Joey Hess2017-02-28
| | | | Display it as a separate action, so it ends with a newline
* remove old debug printGravatar Joey Hess2017-02-28
|
* squelch a couple of warnings about moveAnnex return codeGravatar Joey Hess2017-02-28
|
* inheritable annex.securehashesonlyGravatar Joey Hess2017-02-27
| | | | | | | | | | | | | | | * init: When annex.securehashesonly has been set with git-annex config, copy that value to the annex.securehashesonly git config. * config --set: As well as setting value in git-annex branch, set local gitconfig. This is needed especially for annex.securehashesonly, which is read only from local gitconfig and not the git-annex branch. doc/todo/sha1_collision_embedding_in_git-annex_keys.mdwn has the rationalle for doing it this way. There's no perfect solution; this seems to be the least-bad one. This commit was supported by the NSF-funded DataLad project.
* make fsck check annex.securehashesonly, and new tip for working around SHA1 ↵Gravatar Joey Hess2017-02-27
| | | | | | collisions with git-annex This commit was sponsored by andrea rota.
* annex.securehashesonlyGravatar Joey Hess2017-02-27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Cryptographically secure hashes can be forced to be used in a repository, by setting annex.securehashesonly. This does not prevent the git repository from containing files with insecure hashes, but it does prevent the content of such files from being pulled into .git/annex/objects from another repository. We want to make sure that at no point does git-annex accept content into .git/annex/objects that is hashed with an insecure key. Here's how it was done: * .git/annex/objects/xx/yy/KEY/ is kept frozen, so nothing can be written to it normally * So every place that writes content must call, thawContent or modifyContent. We can audit for these, and be sure we've considered all cases. * The main functions are moveAnnex, and linkToAnnex; these were made to check annex.securehashesonly, and are the main security boundary for annex.securehashesonly. * Most other calls to modifyContent deal with other files in the KEY directory (inode cache etc). The other ones that mess with the content are: - Annex.Direct.toDirectGen, in which content already in the annex directory is moved to the direct mode file, so not relevant. - fix and lock, which don't add new content - Command.ReKey.linkKey, which manually unlocks it to make a copy. * All other calls to thawContent appear safe. Made moveAnnex return a Bool, so checked all callsites and made them deal with a failure in appropriate ways. linkToAnnex simply returns LinkAnnexFailed; all callsites already deal with it failing in appropriate ways. This commit was sponsored by Riku Voipio.
* fix up Read instance incompatability caused by recent commitGravatar Joey Hess2017-02-24
| | | | | | | | | | | | | | | | | | | 2f868db90c7ba16eee901b9b1472b1e1a889dd93 changed the Read instance for Key. I've checked all uses of that instance (by removing it and seeing what breaks), and they're all limited to the webapp, except one. That is GitAnnexDistribution's Read instance. So, 2f868db90c7ba16eee901b9b1472b1e1a889dd93 would have broken upgrades of git-annex from downloads.kitenet.net. Once the .info files there got updated for a new release, old releases would have failed to parse them and never upgraded. To fix this, I found a way to make the .info files that contain GitAnnexDistribution values be readable by the old version of git-annex. This commit was sponsored by Ewen McNeill.
* add KeyVariety typeGravatar Joey Hess2017-02-24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Where before the "name" of a key and a backend was a string, this makes it a concrete data type. This is groundwork for allowing some varieties of keys to be disabled in file2key, so git-annex won't use them at all. Benchmarks ran in my big repo: old git-annex info: real 0m3.338s user 0m3.124s sys 0m0.244s new git-annex info: real 0m3.216s user 0m3.024s sys 0m0.220s new git-annex find: real 0m7.138s user 0m6.924s sys 0m0.252s old git-annex find: real 0m7.433s user 0m7.240s sys 0m0.232s Surprising result; I'd have expected it to be slower since it now parses all the key varieties. But, the parser is very simple and perhaps sharing KeyVarieties uses less memory or something like that. This commit was supported by the NSF-funded DataLad project.
* noCommit for PostReceiveGravatar Joey Hess2017-02-23
| | | | | | | | | | | | | | | This was noticed because it broke the datalad test suite, which pushed to the remote and then fetched to check if it had received the expected branches. Auto-init caused the git-annex branch on the remote to diverge, breaking that test. https://github.com/datalad/datalad/issues/1319#issuecomment-281649518 The auto-init still happens, it's staged in the journal, and will be commited by some later git-annex command when it runs. Which is fine, it's the same as that later command doing the auto-init. This commit was supported by the NSF-funded DataLad project
* status: Pass --ignore-submodules=when option on to git status.Gravatar Joey Hess2017-02-20
| | | | | | | Didn't make --ignore-submodules without a value be handled because I can't see a way to make optparse-applicative parse that. I've opened a bug requesting a way to do that: https://github.com/pcapriotti/optparse-applicative/issues/243
* sync hack to make updateInstead work on eg FATGravatar Joey Hess2017-02-17
| | | | | | | | | | | | sync: When syncing with a local repository located on a crippled filesystem, run the post-receive hook there, since it wouldn't get run otherwise. This makes pushing to repos on FAT-formatted removable drives update them when receive.denyCurrentBranch=updateInstead. Made Remote.Git export onLocal, which was cleaned up to not have so many caveats about its use. This commit was sponsored by Jeff Goeke-Smith on Patreon.
* post-recive hook to make updateInstead work in direct mode and adjusted branchesGravatar Joey Hess2017-02-17
| | | | | | | | * Added post-recieve hook, which makes updateInstead work with direct mode and adjusted branches. * init: Set up the post-receive hook. This commit was sponsored by Fernando Jimenez on Patreon.
* make git-annex:git-annex push quiet againGravatar Joey Hess2017-02-17
| | | | | | | | | | | | Recent changes had a side effect of displaying errors in the fairly common case when this push fails. Since the synced/git-annex push is always forced, those errors are noise, so hide again. This means 3 separate pushes are done now, where before it only made 2. A bit more expensive, but ssh connection caching eliminates most of the costs. This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
* make query commands not output extraneous messagesGravatar Joey Hess2017-02-16
| | | | | | config group groupwanted numcopies schedule wanted required: Avoid displaying extraneous messages about repository auto-init, git-annex branch merging, etc, when being used to get information.
* sync: Improve integration with receive.denyCurrentBranch=updateInsteadGravatar Joey Hess2017-02-15
| | | | | | | | | | By displaying error messages from the remote then it fails to update its checked out branch. Error messages in the default receive.denyCurrentBranch are still suppressed, which matches user expectations. This commit was sponsored by Nick Daly on Patreon.
* Run ssh with -n whenever input is not being piped into itGravatar Joey Hess2017-02-15
| | | | | | | | | | | | | | | | | | | | ... to avoid it consuming stdin that it shouldn't. This fixes git-annex-checkpresentkey --batch remote, which didn't output results for all keys passed into it. Other git-annex commands that communicate with a remote over ssh may also have been consuming stdin that they shouldn't have, which could have impacted using them in eg, shell scripts. For example, a shell script reading files from stdin and passing them to git annex drop would be impacted by this bug, whenever git annex drop ran git-annex-shell checkpresent, it would consume part/all of the stdin that the shell script was supposed to consume. Fixed by adding a ConsumeStdin parameter to Annex.Ssh.sshOptions, which is used throughout git-annex to run ssh (in order for ssh connection caching to work). Every call site was checked to see if it used CreatePipe for stdin, and if not was marked NoConsumeStdin.
* forgot to compile last commit; fix mistakesGravatar Joey Hess2017-02-15
|
* sync, merge: Fail when the current branch has no commits yet, instead of not ↵Gravatar Joey Hess2017-02-14
| | | | | | | | | | | | | | | | | merging in anything from remotes and appearing to succeed. At first I wanted to make it go ahead and merge into the newborn branch, so made it use Git.Branch.currentUnsafe to get the current branch. But that failed: fatal: ambiguous argument 'refs/heads/master..refs/heads/synced/master': unknown revision or path not in the working tree. A whole nother code path to handle merging into newborn branches seemed excessive, so went with displaying a warning and propigating failure status. This commit was sponsored by Brock Spratlen on Patreon.
* correct spelling mistakesGravatar Edward Betts2017-02-12
|
* import --reinject-duplicatesGravatar Joey Hess2017-02-09
| | | | | | | | | This is the same as running git annex reinject --known, followed by git-annex import. The advantage to having it in one command is that it only has to hash each file once; the two commands have to hash the imported files a second time. This commit was sponsored by Shane-o on Patreon.
* Make import --deduplicate and --skip-duplicates only hash once, not twiceGravatar Joey Hess2017-02-09
| | | | | | | | | | | | | | | | | | | | import: --deduplicate and --skip-duplicates were implemented inneficiently; they unncessarily hashed each file twice. They have been improved to only hash once. The new approach is to lock down (minimally) and hash files, and then reuse that information when importing them. This was rather tricky, especially in detecting changes to files while they are being imported. The output of import changed slightly. While before it silently skipped over files with eg --skip-duplicates, now it shows each file as it starts to act on it. Since every file is hashed first thing, it would otherwise not be clear what file import is chewing on. (Actually, it wasn't clear before when any of the duplicates switches were used.) This commit was sponsored by Alexander Thompson on Patreon.
* import: Changed how --deduplicate, --skip-duplicates, and --clean-duplicates ↵Gravatar Joey Hess2017-02-07
| | | | | | | | | | | | | | | | | | | | | determine if a file is a duplicate Before, only content known to be present somewhere was considered a duplicate. Now, any content that has been annexed before will be considered a duplicate, even if all annexed copies of the data have been lost. Note that --clean-duplicates and --deduplicate still check numcopies, so won't delete duplicate files unless there's an annexed copy. This makes import use the same method as reinject --known. The man page already said that duplicate meant "its content is either present in the local repository already, or git-annex knows of another repository that contains it, or it was present in the annex before but has been removed now". So, this is really only bringing the implementation into line with the man page. This commit was sponsored by Jochen Bartl on Patreon.
* initremote: When a uuid= parameter is passed, use the specified UUID for the ↵Gravatar Joey Hess2017-02-07
| | | | | | | new special remote, instead of generating a UUID. This can be useful in some situations, eg when the same data can be accessed via two different special remote backends.
* add SetupStage parameter to RemoteType.setupGravatar Joey Hess2017-02-07
| | | | | | | | | | | | | | | | | Most remotes have an idempotent setup that can be reused for enableremote, but in a few cases, it needs to tell which, and whether a UUID was provided to setup was used. This is groundwork for making initremote be able to provide a UUID. It should not change any behavior. Note that it would be nice to make the UUID always be provided to setup, and make setup not need to generate and return a UUID. What prevented this simplification is Remote.Git.gitSetup, which needs to reuse the UUID of the git remote when setting it up, and so has to return that UUID. This commit was sponsored by Thom May on Patreon.
* assistant: Make --autostart --foreground wait for the children it starts.Gravatar Joey Hess2017-02-07
| | | | | | Before, the --foreground was ignored when autostarting. This commit was sponsored by Denis Dzyubenko on Patreon.
* wormhole pairing appid flag day 2021-12-31Gravatar Joey Hess2017-02-03
| | | | | | | | | | | | Wormhole pairing will start to provide an appid to wormhole on 2021-12-31. An appid can't be provided now because Debian stable is going to ship a older version of git-annex that does not provide an appid. Assumption is that by 2021-12-31, this version of git-annex will be shipped in a Debian stable release. If that turns out to not be the case, this change will need to be cherry-picked into the git-annex in Debian stable, or its wormhole pairing will break. This commit was sponsored by Thomas Hochstein on Patreon.
* make sync --no-commit override annex.annex.autocommitGravatar Joey Hess2017-02-03
|
* New annex.synccontent config settingGravatar Joey Hess2017-02-03
| | | | | | | | | | | | | | | | .. which can be set to true to make git annex sync default to --content. This may become the default at some point in the future. As well as being configuable by git config, it can be configured by git-annex config to control the default behavior in all clones of a repository. Had to add a separate --no-content switch to we can tell if it's been explicitly set, and should override annex.synccontent. If --content was the default, this complication would not be necessary. This commit was sponsored by Jake Vosloo on Patreon.
* annex.autocommit can be configured via git-annex configGravatar Joey Hess2017-02-03
| | | | | | | | | | | | | | | | | ... to control the default behavior in all clones of a repository. This includes a new Configurable data type, so the GitConfig type indicates which values can be configured this way. The implementation should be quite efficient; the config log is only read once, and only when a Configurable value has not already been set by git-config. Indeed, it would be nice in the future to extend this, so that git-config is itself only read on demand. Some commands may not need to look at the git configuration at all. This commit was sponsored by Trenton Cronholm on Patreon.
* unused: When large files are checked right into git, avoid buffering their ↵Gravatar Joey Hess2017-01-31
| | | | | | | | | contents in memory. This makes it a little bit slower since it has to check file size, but worth it to fix a potential memory use problem. This commit was sponsored by Fernando Jimenez on Patreon.
* Some optimisations to string splitting code.Gravatar Joey Hess2017-01-31
| | | | | | | | | | | | | | | | | | | Turns out that Data.List.Utils.split is slow and makes a lot of allocations. Here's a much simpler single character splitter that behaves the same (even in wacky corner cases) while running in half the time and 75% the allocations. As well as being an optimisation, this helps move toward eliminating use of missingh. (Data.List.Split.splitOn is nearly as slow as Data.List.Utils.split and allocates even more.) I have not benchmarked the effect on git-annex, but would not be surprised to see some parsing of eg, large streams from git commands run twice as fast, and possibly in less memory. This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
* make git annex config settings editable in vicfgGravatar Joey Hess2017-01-30
| | | | This commit was sponsored by Shane-o on Patreon.
* config: New command for storing configuration in the git-annex branch.Gravatar Joey Hess2017-01-30
| | | | | | | | | | | Any config names can be set using this; git-annex commands will only look at specific ones that make sense and are worth the overhead of querying the branch. This might also be useful for storing whatever other config-type stuff the user might want to shove into the git-annex branch. This commit was sponsored by Jochen Bartl on Patreon.