summaryrefslogtreecommitdiff
path: root/Assistant
Commit message (Collapse)AuthorAge
* Added remote.<name>.annex-push and remote.<name>.annex-pullGravatar Joey Hess2017-04-05
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The former can be useful to make remotes that don't get fully synced with local changes, which comes up in a lot of situations. The latter was mostly added for symmetry, but could be useful (though less likely to be). Implementing `remote.<name>.annex-pull` was a bit tricky, as there's no one place where git-annex pulls/fetches from remotes. I audited all instances of "fetch" and "pull". A few cases were left not checking this config: * Git.Repair can try to pull missing refs from a remote, and if the local repo is corrupted, that seems a reasonable thing to do even though the config would normally prevent it. * Assistant.WebApp.Gpg and Remote.Gcrypt and Remote.Git do fetches as part of the setup process of a remote. The config would probably not be set then, and having the setup fail seems worse than honoring it if it is already set. I have not prevented all the code that does a "merge" from merging branches from remotes with remote.<name>.annex-pull=false. That could perhaps be done, but it would need a way to map from branch name to remote name, and the way refspecs work makes that hard to get really correct. So if the user fetches manually, the git-annex branch will get merged, for example. Anther way of looking at/justifying this is that the setting is called "annex-pull", not "annex-merge". This commit was supported by the NSF-funded DataLad project.
* AssociatedFile newtypeGravatar Joey Hess2017-03-10
| | | | | | To prevent any further mistakes like 1a497cefb47557f0b4788c606f9071be422b2511 This commit was sponsored by Francois Marier on Patreon.
* remove unused importGravatar Joey Hess2017-03-08
|
* assistant: Add 1/200th second delay between checking each file in the full ↵Gravatar Joey Hess2017-03-06
| | | | | | | | | | | | | | | | | | transfer scan, to avoid using too much CPU. The slowdown is not going to be large in typical small-ish repos. And it does not seem to matter if the assistant reacts a little bit slower in situations involving the expensive scan, since: a) Those situations typically involve getting back in sync after something has changed on a remote, often after a disconnect of some duration. So taking a few seconds more is not noticable. b) If the scan finds things that it needs to do, it will start blocking anyway after 10 transfers are queued (due to use of queueTransferWhenSmall). So, only the speed of finding the first 10 transfers will be impacted by this change. This commit was sponsored by Jochen Bartl on Patreon.
* fix up Read instance incompatability caused by recent commitGravatar Joey Hess2017-02-24
| | | | | | | | | | | | | | | | | | | 2f868db90c7ba16eee901b9b1472b1e1a889dd93 changed the Read instance for Key. I've checked all uses of that instance (by removing it and seeing what breaks), and they're all limited to the webapp, except one. That is GitAnnexDistribution's Read instance. So, 2f868db90c7ba16eee901b9b1472b1e1a889dd93 would have broken upgrades of git-annex from downloads.kitenet.net. Once the .info files there got updated for a new release, old releases would have failed to parse them and never upgraded. To fix this, I found a way to make the .info files that contain GitAnnexDistribution values be readable by the old version of git-annex. This commit was sponsored by Ewen McNeill.
* add KeyVariety typeGravatar Joey Hess2017-02-24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Where before the "name" of a key and a backend was a string, this makes it a concrete data type. This is groundwork for allowing some varieties of keys to be disabled in file2key, so git-annex won't use them at all. Benchmarks ran in my big repo: old git-annex info: real 0m3.338s user 0m3.124s sys 0m0.244s new git-annex info: real 0m3.216s user 0m3.024s sys 0m0.220s new git-annex find: real 0m7.138s user 0m6.924s sys 0m0.252s old git-annex find: real 0m7.433s user 0m7.240s sys 0m0.232s Surprising result; I'd have expected it to be slower since it now parses all the key varieties. But, the parser is very simple and perhaps sharing KeyVarieties uses less memory or something like that. This commit was supported by the NSF-funded DataLad project.
* include problem PairData in error messageGravatar Joey Hess2017-02-13
| | | | | | | | | | Users occasionally report this error firing, and I can't see why, so include the rejected PairData in the error message. This is safe even if it contains evil escape characters, because showing it displays them in escaped form. This commit was sponsored by Bruno BEAUFILS on Patreon.
* Make import --deduplicate and --skip-duplicates only hash once, not twiceGravatar Joey Hess2017-02-09
| | | | | | | | | | | | | | | | | | | | import: --deduplicate and --skip-duplicates were implemented inneficiently; they unncessarily hashed each file twice. They have been improved to only hash once. The new approach is to lock down (minimally) and hash files, and then reuse that information when importing them. This was rather tricky, especially in detecting changes to files while they are being imported. The output of import changed slightly. While before it silently skipped over files with eg --skip-duplicates, now it shows each file as it starts to act on it. Since every file is hashed first thing, it would otherwise not be clear what file import is chewing on. (Actually, it wasn't clear before when any of the duplicates switches were used.) This commit was sponsored by Alexander Thompson on Patreon.
* add SetupStage parameter to RemoteType.setupGravatar Joey Hess2017-02-07
| | | | | | | | | | | | | | | | | Most remotes have an idempotent setup that can be reused for enableremote, but in a few cases, it needs to tell which, and whether a UUID was provided to setup was used. This is groundwork for making initremote be able to provide a UUID. It should not change any behavior. Note that it would be nice to make the UUID always be provided to setup, and make setup not need to generate and return a UUID. What prevented this simplification is Remote.Git.gitSetup, which needs to reuse the UUID of the git remote when setting it up, and so has to return that UUID. This commit was sponsored by Thom May on Patreon.
* annex.autocommit can be configured via git-annex configGravatar Joey Hess2017-02-03
| | | | | | | | | | | | | | | | | ... to control the default behavior in all clones of a repository. This includes a new Configurable data type, so the GitConfig type indicates which values can be configured this way. The implementation should be quite efficient; the config log is only read once, and only when a Configurable value has not already been set by git-config. Indeed, it would be nice in the future to extend this, so that git-config is itself only read on demand. Some commands may not need to look at the git configuration at all. This commit was sponsored by Trenton Cronholm on Patreon.
* Some optimisations to string splitting code.Gravatar Joey Hess2017-01-31
| | | | | | | | | | | | | | | | | | | Turns out that Data.List.Utils.split is slow and makes a lot of allocations. Here's a much simpler single character splitter that behaves the same (even in wacky corner cases) while running in half the time and 75% the allocations. As well as being an optimisation, this helps move toward eliminating use of missingh. (Data.List.Split.splitOn is nearly as slow as Data.List.Utils.split and allocates even more.) I have not benchmarked the effect on git-annex, but would not be surprised to see some parsing of eg, large streams from git commands run twice as fast, and possibly in less memory. This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
* webapp: full wormhole pairing UI (untested)Gravatar Joey Hess2016-12-27
| | | | This commit was sponsored by Riku Voipio.
* clean up some importsGravatar Joey Hess2016-12-27
|
* mocked up wormhole pairing interface in webappGravatar Joey Hess2016-12-24
|
* Merge branch 'master' into no-xmppGravatar Joey Hess2016-12-24
|\
| * Always use filesystem encoding for all file and handle reads and writes.Gravatar Joey Hess2016-12-24
| | | | | | | | | | This is a big scary change. I have convinced myself it should be safe. I hope!
| * max authtoken length is 128Gravatar Joey Hess2016-11-30
| | | | | | | | | | It was stopping at 128, so the 512 was only incorrect, it didn't change behavior.
| * Merge branch 'master' into torGravatar Joey Hess2016-11-29
| |\
| * | unified AuthToken type between webapp and torGravatar Joey Hess2016-11-22
| | |
| | * Relicense 5 source files that are not part of the webapp from AGPL to GPL.Gravatar Joey Hess2016-11-21
| | | | | | | | | | | | | | | | | | | | | | | | Building w/o the webapp is not supposed to pull in any AGPLed files. I appear to have written all the code in these files; the only commit by anyone else is 0d555aa363482ed041db2d9c63da271ba7f4ced8 and is a spelling fix that is not copyrightable.
| * | remotedaemon: Fork to background by default. Added --foreground switch to ↵Gravatar Joey Hess2016-11-20
| |/ | | | | | | | | | | enable old behavior. Groundwork for tor hidden services, which the remotedaemon will serve.
| * Avoid backtraces on expected failures when built with ghc 8; only use ↵Gravatar Joey Hess2016-11-15
| | | | | | | | | | | | | | | | | | | | | | | | | | backtraces for unexpected errors. ghc 8 added backtraces on uncaught errors. This is great, but git-annex was using error in many places for a error message targeted at the user, in some known problem case. A backtrace only confuses such a message, so omit it. Notably, commands like git annex drop that failed due to eg, numcopies, used to use error, so had a backtrace. This commit was sponsored by Ethan Aubin.
| * fix tricky warning with ghc 8Gravatar Joey Hess2016-11-15
| | | | | | | | | | Whether Route was exported from Assistant.WebApp.Types or not depended on the version of ghc. So, explictly export it.
| * sync: Pass --allow-unrelated-histories to git merge when used with git git ↵Gravatar Joey Hess2016-11-15
| | | | | | | | | | | | | | | | | | | | | | | | 2.9.0 or newer. This makes merging a remote into a freshly created direct mode repository work the same as it works in indirect mode. The git-annex branches would get merged in any case by a sync, since that doesn't use git merge. This might need to be revisited later to better mirror git's behavior.
* | remove xmpp supportGravatar Joey Hess2016-11-14
|/ | | | | | | | | | | | I've long considered the XMPP support in git-annex a wart. It's nice to remove it. (This also removes the NetMessager, which was only used for XMPP, and the daemonstatus's desynced list (likewise).) Existing XMPP remotes should be ignored by git-annex. This commit was sponsored by Brock Spratlen on Patreon.
* convert TMVars that are never left empty into TVarsGravatar Joey Hess2016-09-30
| | | | | This is probably more efficient, and it avoids mistakenly leaving them empty.
* switch away from deprecated interfaceGravatar Joey Hess2016-09-05
| | | | | Again the new stuff works back to network-2.4, so no need to adjust cabal bounds.
* switch away from deprecated recvFromGravatar Joey Hess2016-09-05
| | | | | Network.Socket.ByteString is in network since before 2.4, so no need to adjust bounds.
* get, move, copy, mirror: Added --failed switch which retries failed copies/movesGravatar Joey Hess2016-08-03
| | | | | | | | | Note that get --from foo --failed will get things that a previous get --from bar tried and failed to get, etc. I considered making --failed only retry transfers from the same remote, but it was easier, and seems more useful, to not have the same remote requirement. Noisy due to some refactoring into Types/
* webapp: Escape unusual characters in ssh hostnames when generating mangled ↵Gravatar Joey Hess2016-07-19
| | | | hostnames. This allows IPv6 addresses to be used on filesystems not supporting : in filenames.
* filter out NoUUID remotes from syncDataRemotesGravatar Joey Hess2016-06-09
|
* only warn about missing dbus support on linuxGravatar Joey Hess2016-05-24
|
* Pass the various gnupg-options configs to gpg in several cases where they ↵Gravatar Joey Hess2016-05-23
| | | | | | | | | | | | were not before. Removed the instance LensGpgEncParams RemoteConfig because it encouraged code that does not take the RemoteGitConfig into account. RemoteType's setup was changed to take a RemoteGitConfig, although the only place that is able to provide a non-empty one is enableremote, when it's changing an existing remote. This led to several folow-on changes, and got RemoteGitConfig plumbed through.
* move keys db closure to AutoMergeGravatar Joey Hess2016-05-16
| | | | | This makes git-annex sync also do it, which makes sure that the keys db info is fresh when doing a sync --content.
* assistant: Fix race in v6 mode that caused downloaded file content to ↵Gravatar Joey Hess2016-05-16
| | | | | | | | | | | | sometimes not replace pointer files. The keys database handle needs to be closed after merging, because the smudge filter, in another process, updates the database. Old cached info can be read for a while from the open database handle; closing it ensures that the info written by the smudge filter is available. This is pretty horribly ad-hoc, and it's especially nasty that the transferrer closes the database every time.
* assistant: Fix bug that caused v6 pointer files to be annexed by the assistant.Gravatar Joey Hess2016-05-16
|
* stage pointer file not annex linkGravatar Joey Hess2016-05-16
|
* avoid warnings about not exported System.Directory.isSymbolicLinkGravatar Joey Hess2016-04-28
|
* Fix build with directory-1.2.6.2.Gravatar Joey Hess2016-04-28
| | | | | | It started exporting a isSymbolicLink which supports windows. But, git-annex does no use symlinks on windows yet and this conflicts with the function by the same name from unix-compat, so hide it.
* make assistant not write wrappers when GIT_ANNEX_PACKAGE_INSTALL is setGravatar Joey Hess2016-04-27
| | | | | | This way, the git-annex-standalone.deb, which does set GIT_ANNEX_APP_BASE, will not have the assistant install wrappers that it does not need to, since git-annex is installed system wide from a package.
* assistant: Deal with upcoming git's refusal to merge unrelated histories by ↵Gravatar Joey Hess2016-04-22
| | | | | | | | | | | | | default git 2.8.1 (or perhaps 2.9.0) is going to prevent git merge from merging in unrelated branches. Since the webapp's pairing etc features often combine together repositories with unrelated histories, work around this behavior change by setting GIT_MERGE_ALLOW_UNRELATED_HISTORIES when the assistant merges. Note though that this is not done for git annex sync's merges, so it will follow git's default or configured behavior.
* use more general moduleGravatar Joey Hess2016-04-21
|
* Preserve execute bits of unlocked files in v6 mode.Gravatar Joey Hess2016-04-14
| | | | | | | | | | | | | | When annex.thin is set, adding an object will add the execute bits to the work tree file, and this does mean that the annex object file ends up executable. This doesn't add any complexity that wasn't already present, because git annex add of an executable file has always ingested it so that the annex object ends up executable. But, since an annex object file can be executable or not, when populating an unlocked file from one, the executable bit is always added or removed to match the mode of the pointer file.
* update sync branch to the orig branch when in adjusted branchGravatar Joey Hess2016-02-29
|
* make assistant aware of adjusted branches when mergingGravatar Joey Hess2016-02-29
|
* push original branch when on adjusted branchGravatar Joey Hess2016-02-29
|
* make sync aware of adjusted branchesGravatar Joey Hess2016-02-29
| | | | | | | | | | So, it will pull and push the original branch, not the adjusted one. And, for merging, it will use updateAdjustedBranch (not implemented yet). Note that remaining uses of Git.Branch.current need to be checked too; for things that should act on the original branch, and not the adjusted branch.
* annex.largefiles can be configured in .gitattributes tooGravatar Joey Hess2016-02-02
| | | | | This is particulary useful for v6 repositories, since the .gitattributes configuration will apply in all clones of the repository.
* remove 3 build flagsGravatar Joey Hess2016-01-26
| | | | | | | | | | | | | * Removed the webapp-secure build flag, rolling it into the webapp build flag. * Removed the quvi and tahoe build flags, which only adds aeson to the core dependencies. * Removed the feed build flag, which only adds feed to the core dependencies. Build flags have cost in both code complexity and also make Setup configure have to work harder to find a usable set of build flags when some dependencies are missing.
* assistant: Use udisks2 dbus events to detect when disks are mounted, instead ↵Gravatar Joey Hess2016-01-22
| | | | of relying on gnome/kde stuff that is not stable.