aboutsummaryrefslogtreecommitdiff
path: root/Assistant/Threads
Commit message (Collapse)AuthorAge
* AssociatedFile newtypeGravatar Joey Hess2017-03-10
| | | | | | To prevent any further mistakes like 1a497cefb47557f0b4788c606f9071be422b2511 This commit was sponsored by Francois Marier on Patreon.
* remove unused importGravatar Joey Hess2017-03-08
|
* assistant: Add 1/200th second delay between checking each file in the full ↵Gravatar Joey Hess2017-03-06
| | | | | | | | | | | | | | | | | | transfer scan, to avoid using too much CPU. The slowdown is not going to be large in typical small-ish repos. And it does not seem to matter if the assistant reacts a little bit slower in situations involving the expensive scan, since: a) Those situations typically involve getting back in sync after something has changed on a remote, often after a disconnect of some duration. So taking a few seconds more is not noticable. b) If the scan finds things that it needs to do, it will start blocking anyway after 10 transfers are queued (due to use of queueTransferWhenSmall). So, only the speed of finding the first 10 transfers will be impacted by this change. This commit was sponsored by Jochen Bartl on Patreon.
* add KeyVariety typeGravatar Joey Hess2017-02-24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Where before the "name" of a key and a backend was a string, this makes it a concrete data type. This is groundwork for allowing some varieties of keys to be disabled in file2key, so git-annex won't use them at all. Benchmarks ran in my big repo: old git-annex info: real 0m3.338s user 0m3.124s sys 0m0.244s new git-annex info: real 0m3.216s user 0m3.024s sys 0m0.220s new git-annex find: real 0m7.138s user 0m6.924s sys 0m0.252s old git-annex find: real 0m7.433s user 0m7.240s sys 0m0.232s Surprising result; I'd have expected it to be slower since it now parses all the key varieties. But, the parser is very simple and perhaps sharing KeyVarieties uses less memory or something like that. This commit was supported by the NSF-funded DataLad project.
* include problem PairData in error messageGravatar Joey Hess2017-02-13
| | | | | | | | | | Users occasionally report this error firing, and I can't see why, so include the rejected PairData in the error message. This is safe even if it contains evil escape characters, because showing it displays them in escaped form. This commit was sponsored by Bruno BEAUFILS on Patreon.
* Make import --deduplicate and --skip-duplicates only hash once, not twiceGravatar Joey Hess2017-02-09
| | | | | | | | | | | | | | | | | | | | import: --deduplicate and --skip-duplicates were implemented inneficiently; they unncessarily hashed each file twice. They have been improved to only hash once. The new approach is to lock down (minimally) and hash files, and then reuse that information when importing them. This was rather tricky, especially in detecting changes to files while they are being imported. The output of import changed slightly. While before it silently skipped over files with eg --skip-duplicates, now it shows each file as it starts to act on it. Since every file is hashed first thing, it would otherwise not be clear what file import is chewing on. (Actually, it wasn't clear before when any of the duplicates switches were used.) This commit was sponsored by Alexander Thompson on Patreon.
* annex.autocommit can be configured via git-annex configGravatar Joey Hess2017-02-03
| | | | | | | | | | | | | | | | | ... to control the default behavior in all clones of a repository. This includes a new Configurable data type, so the GitConfig type indicates which values can be configured this way. The implementation should be quite efficient; the config log is only read once, and only when a Configurable value has not already been set by git-config. Indeed, it would be nice in the future to extend this, so that git-config is itself only read on demand. Some commands may not need to look at the git configuration at all. This commit was sponsored by Trenton Cronholm on Patreon.
* webapp: full wormhole pairing UI (untested)Gravatar Joey Hess2016-12-27
| | | | This commit was sponsored by Riku Voipio.
* clean up some importsGravatar Joey Hess2016-12-27
|
* Merge branch 'master' into no-xmppGravatar Joey Hess2016-12-24
|\
| * max authtoken length is 128Gravatar Joey Hess2016-11-30
| | | | | | | | | | It was stopping at 128, so the 512 was only incorrect, it didn't change behavior.
| * unified AuthToken type between webapp and torGravatar Joey Hess2016-11-22
| |
| * remotedaemon: Fork to background by default. Added --foreground switch to ↵Gravatar Joey Hess2016-11-20
| | | | | | | | | | | | enable old behavior. Groundwork for tor hidden services, which the remotedaemon will serve.
| * Avoid backtraces on expected failures when built with ghc 8; only use ↵Gravatar Joey Hess2016-11-15
| | | | | | | | | | | | | | | | | | | | | | | | | | backtraces for unexpected errors. ghc 8 added backtraces on uncaught errors. This is great, but git-annex was using error in many places for a error message targeted at the user, in some known problem case. A backtrace only confuses such a message, so omit it. Notably, commands like git annex drop that failed due to eg, numcopies, used to use error, so had a backtrace. This commit was sponsored by Ethan Aubin.
| * sync: Pass --allow-unrelated-histories to git merge when used with git git ↵Gravatar Joey Hess2016-11-15
| | | | | | | | | | | | | | | | | | | | | | | | 2.9.0 or newer. This makes merging a remote into a freshly created direct mode repository work the same as it works in indirect mode. The git-annex branches would get merged in any case by a sync, since that doesn't use git merge. This might need to be revisited later to better mirror git's behavior.
* | remove xmpp supportGravatar Joey Hess2016-11-14
|/ | | | | | | | | | | | I've long considered the XMPP support in git-annex a wart. It's nice to remove it. (This also removes the NetMessager, which was only used for XMPP, and the daemonstatus's desynced list (likewise).) Existing XMPP remotes should be ignored by git-annex. This commit was sponsored by Brock Spratlen on Patreon.
* switch away from deprecated recvFromGravatar Joey Hess2016-09-05
| | | | | Network.Socket.ByteString is in network since before 2.4, so no need to adjust bounds.
* get, move, copy, mirror: Added --failed switch which retries failed copies/movesGravatar Joey Hess2016-08-03
| | | | | | | | | Note that get --from foo --failed will get things that a previous get --from bar tried and failed to get, etc. I considered making --failed only retry transfers from the same remote, but it was easier, and seems more useful, to not have the same remote requirement. Noisy due to some refactoring into Types/
* only warn about missing dbus support on linuxGravatar Joey Hess2016-05-24
|
* move keys db closure to AutoMergeGravatar Joey Hess2016-05-16
| | | | | This makes git-annex sync also do it, which makes sure that the keys db info is fresh when doing a sync --content.
* assistant: Fix race in v6 mode that caused downloaded file content to ↵Gravatar Joey Hess2016-05-16
| | | | | | | | | | | | sometimes not replace pointer files. The keys database handle needs to be closed after merging, because the smudge filter, in another process, updates the database. Old cached info can be read for a while from the open database handle; closing it ensures that the info written by the smudge filter is available. This is pretty horribly ad-hoc, and it's especially nasty that the transferrer closes the database every time.
* assistant: Fix bug that caused v6 pointer files to be annexed by the assistant.Gravatar Joey Hess2016-05-16
|
* stage pointer file not annex linkGravatar Joey Hess2016-05-16
|
* assistant: Deal with upcoming git's refusal to merge unrelated histories by ↵Gravatar Joey Hess2016-04-22
| | | | | | | | | | | | | default git 2.8.1 (or perhaps 2.9.0) is going to prevent git merge from merging in unrelated branches. Since the webapp's pairing etc features often combine together repositories with unrelated histories, work around this behavior change by setting GIT_MERGE_ALLOW_UNRELATED_HISTORIES when the assistant merges. Note though that this is not done for git annex sync's merges, so it will follow git's default or configured behavior.
* use more general moduleGravatar Joey Hess2016-04-21
|
* Preserve execute bits of unlocked files in v6 mode.Gravatar Joey Hess2016-04-14
| | | | | | | | | | | | | | When annex.thin is set, adding an object will add the execute bits to the work tree file, and this does mean that the annex object file ends up executable. This doesn't add any complexity that wasn't already present, because git annex add of an executable file has always ingested it so that the annex object ends up executable. But, since an annex object file can be executable or not, when populating an unlocked file from one, the executable bit is always added or removed to match the mode of the pointer file.
* make assistant aware of adjusted branches when mergingGravatar Joey Hess2016-02-29
|
* make sync aware of adjusted branchesGravatar Joey Hess2016-02-29
| | | | | | | | | | So, it will pull and push the original branch, not the adjusted one. And, for merging, it will use updateAdjustedBranch (not implemented yet). Note that remaining uses of Git.Branch.current need to be checked too; for things that should act on the original branch, and not the adjusted branch.
* annex.largefiles can be configured in .gitattributes tooGravatar Joey Hess2016-02-02
| | | | | This is particulary useful for v6 repositories, since the .gitattributes configuration will apply in all clones of the repository.
* remove 3 build flagsGravatar Joey Hess2016-01-26
| | | | | | | | | | | | | * Removed the webapp-secure build flag, rolling it into the webapp build flag. * Removed the quvi and tahoe build flags, which only adds aeson to the core dependencies. * Removed the feed build flag, which only adds feed to the core dependencies. Build flags have cost in both code complexity and also make Setup configure have to work harder to find a usable set of build flags when some dependencies are missing.
* assistant: Use udisks2 dbus events to detect when disks are mounted, instead ↵Gravatar Joey Hess2016-01-22
| | | | of relying on gnome/kde stuff that is not stable.
* assistant: Added new name used for udisks2 dbus service in gnome 3.18.Gravatar Joey Hess2016-01-22
|
* remove 163 lines of code without changing anything except importsGravatar Joey Hess2016-01-20
|
* immediate queue flushing when annex.queuesize=1Gravatar Joey Hess2016-01-13
| | | | | | | | | Previously, it only flushed when the queue got larger than 1. Also, make the queue auto-flush when items are added, rather than needing to be flushed as a separate step. This simplifies the code and make it more efficient too, as it avoids needing to read the queue out of the state to check if it should be flushed.
* avoid confusing git with a modified ctime in clean filterGravatar Joey Hess2016-01-07
| | | | | | | Linking the file to the tmp dir was not necessary in the clean filter, and it caused the ctime to change, which caused git to think the file was changed. This caused git status to get slow as it kept re-cleaning unchanged files.
* use TopFilePath for associated filesGravatar Joey Hess2016-01-05
| | | | | | | | | | | | | | | Fixes several bugs with updates of pointer files. When eg, running git annex drop --from localremote it was updating the pointer file in the local repository, not the remote. Also, fixes drop ../foo when run in a subdir, and probably lots of other problems. Test suite drops from ~30 to 11 failures now. TopFilePath is used to force thinking about what the filepath is relative to. The data stored in the sqlite db is still just a plain string, and TopFilePath is a newtype, so there's no overhead involved in using it in DataBase.Keys.
* improve data typeGravatar Joey Hess2016-01-01
|
* fix OSX buildGravatar Joey Hess2015-12-28
|
* combine PendingAddChanges for the same file into oneGravatar Joey Hess2015-12-22
| | | | | | | | | | | | | In v6 unlocked mode, this fixes a problem that was making eg, echo > file cause the assistant to copy the file to the annex object, instead of hard linking it. That because 2 change events were seen (one for opening the file and one for closing) and processed together the file was then locked down twice. Which meant it had mutiple hard links, and so prevented linkAnnex from hard linking it. There might be scenarios where multiple events come in, but staggered such that a file gets locked down repeatedly, and it would still be copied to the annex object in that case.
* populate unlocked files with newly available content when ingestingGravatar Joey Hess2015-12-22
| | | | | | This can happen when ingesting a new file in either locked or unlocked mode, when some unlocked files in the repo use the same key, and the content was not locally available before.
* finish v6 support for assistantGravatar Joey Hess2015-12-22
| | | | Seems to basically work now!
* refactoringGravatar Joey Hess2015-12-22
| | | | no behavior changes
* wip v6 support for assistantGravatar Joey Hess2015-12-21
| | | | Files are not yet added to v6 repos in unlocked mode.
* reorgGravatar Joey Hess2015-12-15
|
* move InodeSentinal from direct mode code to its own moduleGravatar Joey Hess2015-12-09
| | | | | Will be used outside of direct mode for v6 unlocked files, and is already used outside of direct mode when adding files to annex.
* generalize catchHardwareFault to catchIOErrorTypeGravatar Joey Hess2015-12-06
|
* add VerifiedCopy data typeGravatar Joey Hess2015-10-08
| | | | | | | | | There should be no behavior changes in this commit, it just adds a more expressive data type and adjusts code that had been passing around a [UUID] or sometimes a Maybe Remote to instead use [VerifiedCopy]. Although, since some functions were taking two different [UUID] lists, there's some potential for me to have gotten it horribly wrong.
* --debug is passed along to git-annex-shell when git-annex is in debug mode.Gravatar Joey Hess2015-08-13
|
* Support monitoring systemd-networkd connectivity.Gravatar Sebastian Reuße2015-06-02
|
* remove Params constructor from Utility.SafeCommandGravatar Joey Hess2015-06-01
| | | | | | | | | | | | | | | | | | This removes a bit of complexity, and should make things faster (avoids tokenizing Params string), and probably involve less garbage collection. In a few places, it was useful to use Params to avoid needing a list, but that is easily avoided. Problems noticed while doing this conversion: * Some uses of Params "oneword" which was entirely unnecessary overhead. * A few places that built up a list of parameters with ++ and then used Params to split it! Test suite passes.