aboutsummaryrefslogtreecommitdiff
path: root/Utility
Commit message (Collapse)AuthorAge
* Windows: Fix bug in shell script shebang lookup code that caused a "delayed ↵Gravatar Joey Hess2017-03-13
| | | | | | | | | read on closed handle" error. The bug was that withFile closes the handle afterwards, but the content of the file was not read due to laziness. Using readFile avoids it. This commit was sponsored by Nick Daly on Patreon.
* Drop support for building with old versions of directory, feed, and http-types.Gravatar Joey Hess2017-03-10
|
* Drop support for building with old versions of dns and http-conduit.Gravatar Joey Hess2017-03-10
|
* relicense Utility/GPG.hs BSD as the rest of Utility isGravatar Joey Hess2017-03-10
| | | | | | | | | | The COPYRIGHT had Utility/DirWatcher* listed as GPL, but they were actually BSD licensed. No idea why I put the GPL on Utility/GPG.hs file originally. I wrote all of it, except for guilhem's small changes to it in a7f50205bb1ccd41d4a9ab4e11d984e643be755f, which seem too small to be independently copyrightable. I'm relicencing it BSD.
* Windows: Improve handling of shebang in external special remote program, ↵Gravatar Joey Hess2017-03-08
| | | | | | | | | | | | | | | | | | | searching for the program in the PATH. findShellCommand needs a full path to a file in order to check it for a shebang on Windows. It was being run with only the base name of the external special remote program, which would only work when it was in the current directory. This is why users in https://github.com/DanielDent/git-annex-remote-rclone/pull/10 and elsewhere were complaining that the previous improvements to git-annex didn't make git-remote-rclone work on Windows. Also, reworked checkearlytermination, which while it worked, seemed to rely on a race condition. And, improved its error messages. This commit was sponsored by Shane-o on Patreon.
* Removed support for building with the old cryptohash library.Gravatar Joey Hess2017-02-24
| | | | | Building with that library made git-annex not support SHA3; it's time for that to always be supported in case SHA2 dominoes.
* make curl show http errors to stderrGravatar Joey Hess2017-02-20
| | | | | | | | | | | | | * Run curl with -S, so HTTP errors are displayed, even when it's otherwise silent. * When downloading in --json or --quiet mode, use curl in preference to wget, since curl is able to display only errors to stderr, unlike wget. This does mean that downloadQuiet is only silent on stdout, not necessarily on stderr, which affects a couple other calls of it. For example, downloading the .git/config of a http remote may show an error message now, perhaps with slightly suboptimal formatting due to other output.
* improve layoutGravatar Joey Hess2017-02-20
|
* Run wget with -nv instead of -q, so it will display HTTP errors.Gravatar Joey Hess2017-02-20
| | | | | | This adds one extra line of output when a download is successful, after the progress bar. I don't much like that, but wget does not provide a way to show HTTP errors without it.
* simpler more generic processTranscript'Gravatar Joey Hess2017-02-15
| | | | | | | | This allows using functions that generate CreateProcess and passing the result to processTranscript', which is more flexible, and also simpler than the old interface. This commit was sponsored by Riku Voipio.
* Work around sqlite's incorrect handling of umask when creating databases.Gravatar Joey Hess2017-02-13
| | | | | | | | | Refactored some common code into initDb. This only deals with the problem when creating new databases. If a repo got bad permissions into it, it's up to the user to deal with it. This commit was sponsored by Ole-Morten Duesund on Patreon.
* correct spelling mistakesGravatar Edward Betts2017-02-12
|
* Improve pid locking code to work on filesystems that don't support hard links.Gravatar Joey Hess2017-02-10
| | | | | | | | | | Probing for hard link support in the pid locking code is redundant since git-annex init already probes that. But, it didn't seem worth threading that data through; the pid locking code runs at most once per git-annex process, and only on unusual filesystems. Optimising a single hard link and unlink isn't worth it. This commit was sponsored by Francois Marier on Patreon.
* wormhole pairing appid flag day 2021-12-31Gravatar Joey Hess2017-02-03
| | | | | | | | | | | | Wormhole pairing will start to provide an appid to wormhole on 2021-12-31. An appid can't be provided now because Debian stable is going to ship a older version of git-annex that does not provide an appid. Assumption is that by 2021-12-31, this version of git-annex will be shipped in a Debian stable release. If that turns out to not be the case, this change will need to be cherry-picked into the git-annex in Debian stable, or its wormhole pairing will break. This commit was sponsored by Thomas Hochstein on Patreon.
* Some optimisations to string splitting code.Gravatar Joey Hess2017-01-31
| | | | | | | | | | | | | | | | | | | Turns out that Data.List.Utils.split is slow and makes a lot of allocations. Here's a much simpler single character splitter that behaves the same (even in wacky corner cases) while running in half the time and 75% the allocations. As well as being an optimisation, this helps move toward eliminating use of missingh. (Data.List.Split.splitOn is nearly as slow as Data.List.Utils.split and allocates even more.) I have not benchmarked the effect on git-annex, but would not be surprised to see some parsing of eg, large streams from git commands run twice as fast, and possibly in less memory. This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
* work around ghc segfaultGravatar Joey Hess2016-12-30
| | | | | | | | | | | | | hSetEncoding of a closed handle segfaults. https://ghc.haskell.org/trac/ghc/ticket/7161 3b9d9a267b7c9247d36d9b622e1b836724ca5fb0 introduced the crash. In particular, stdin may get closed (by eg, getContents) and then trying to set its encoding will crash. We didn't need to adjust stdin's encoding anyway, but only stderr, to work around https://github.com/yesodweb/persistent/issues/474 Thanks to Mesar Hameed for assistance related to reproducing this bug.
* more windows build fixGravatar Joey Hess2016-12-30
|
* fix windows buildGravatar Joey Hess2016-12-30
|
* fix windows buildGravatar Joey Hess2016-12-30
|
* fix build with old ghcGravatar Joey Hess2016-12-30
|
* make this build under windowsGravatar Joey Hess2016-12-30
|
* improve description of password promptingGravatar Joey Hess2016-12-28
| | | | | | | | | Since the user does not know whether it will run su or sudo, indicate whether the password prompt will be for root or the user's password, when possible. I assume that programs like gksu that can prompt for either depending on system setup will make clear in their prompt what they're asking for.
* Support all common locations of the torrc file.Gravatar Joey Hess2016-12-28
|
* fix windows buildGravatar Joey Hess2016-12-28
|
* Merge branch 'no-xmpp'Gravatar Joey Hess2016-12-28
|\
* | whitespaceGravatar Joey Hess2016-12-28
| |
| * avoid leaving MVar emptyGravatar Joey Hess2016-12-27
| | | | | | | | Something might want to observe the code multiple times.
| * webapp: check that tor and magic wormhole are installedGravatar Joey Hess2016-12-24
| |
| * cleanupGravatar Joey Hess2016-12-24
|/
* Always use filesystem encoding for all file and handle reads and writes.Gravatar Joey Hess2016-12-24
| | | | | This is a big scary change. I have convinced myself it should be safe. I hope!
* really fix su commandGravatar Joey Hess2016-12-24
|
* fix su paramsGravatar Joey Hess2016-12-24
|
* include tor-annex in hidden service directory namesGravatar Joey Hess2016-12-21
| | | | | | To make it easier to manage/delete them etc. Backwards compatablity is preserved for existing tor configs.
* cleanupGravatar Joey Hess2016-12-20
|
* enable-tor: No longer needs to be run as root.Gravatar Joey Hess2016-12-20
| | | | | | When run by not root, su's to root automatically. This commit was sponsored by Brock Spratlen on Patreon.
* relocate tor socket out of /etcGravatar Joey Hess2016-12-20
| | | | | | | | | | | | | weasel explained that apparmor limits on what files tor can read do not apply to sockets (because they're not files). And apparently the problems I was seeing with hidden services not being accessible had to do with onion address propigation and not the location of the socket file. remotedaemon looks up the HiddenServicePort in torrc, so if it was previously configured with the socket in /etc, that will still work. This commit was sponsored by Denis Dzyubenko on Patreon.
* Fix build with directory-1.3.Gravatar Joey Hess2016-12-20
| | | | See https://github.com/haskell/directory/issues/66
* typoGravatar Joey Hess2016-12-18
|
* check if wormhole is installedGravatar Joey Hess2016-12-18
|
* p2p --pair with magic wormhole (untested)Gravatar Joey Hess2016-12-18
| | | | | | It builds. I have not tried to run it yet. :) This commit was sponsored by Jake Vosloo on Patreon.
* make sure False is returned on errorGravatar Joey Hess2016-12-17
|
* improve typesGravatar Joey Hess2016-12-17
|
* document a minor problemGravatar Joey Hess2016-12-17
|
* use PYTHONUNBUFFERED to force python to use sane stdout bufferingGravatar Joey Hess2016-12-17
| | | | | | | | | Works around https://github.com/warner/magic-wormhole/issues/108 See http://stackoverflow.com/questions/107705/disable-output-buffering for the gory details. Why a scripting language would chose a default stdout buffering that differs between terminal and piped output, and tends to introduce this kind of bug, I don't know.
* magic wormhole moduleGravatar Joey Hess2016-12-17
| | | | | | | | | | | | | | | | | | | This interacts with it using stdio, which is surprisingly hard. sendFile does not currently work, due to https://github.com/warner/magic-wormhole/issues/108 Parsing the output to find the magic code is done as robustly as possible, and should continue to work unless wormhole radically changes the format of its codes. Presumably it will never output something that looks like a wormhole code before the actual wormhole code; that would also break this. It would be better if there was a way to make wormhole not mix the code with other output, as requested in https://github.com/warner/magic-wormhole/issues/104 Only exchange of files/directories is supported. To exchange messages, https://github.com/warner/magic-wormhole/issues/99 would need to be resolved. I don't need message exchange however.
* Pass annex.web-options to wget and curl after other options, so that eg ↵Gravatar Joey Hess2016-12-13
| | | | --no-show-progress can be set by the user to disable the default --show-progress.
* Remove http-conduit (<2.2.0) constraintGravatar Alper Nebi Yasak2016-12-10
| | | | | | | | | | Since https://github.com/aristidb/aws/issues/206 is resolved, this constraint is no longer necessary. However, http-conduit (>=2.2.0) requires http-client (>=0.5.0) which introduces some breaking changes. This commit also implements those changes depending on the version. Fixes: https://git-annex.branchable.com/bugs/Build_with_aws_head_fails/ Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
* git-annex-shell, remotedaemon, git remote: Fix some memory DOS attacks.Gravatar Joey Hess2016-12-09
| | | | | | | | | | | | | | | | | | | | | The attacker could just send a very lot of data, with no \n and it would all be buffered in memory until the kernel killed git-annex or perhaps OOM killed some other more valuable process. This is a low impact security hole, only affecting communication between local git-annex and git-annex-shell on the remote system. (With either able to be the attacker). Only those with the right ssh key can do it. And, there are probably lots of ways to construct git repositories that make git use a lot of memory in various ways, which would have similar impact as this attack. The fix in P2P/IO.hs would have been higher impact, if it had made it to a released version, since it would have allowed DOSing the tor hidden service without needing to authenticate. (The LockContent and NotifyChanges instances may not be really exploitable; since the line is read and ignored, it probably gets read lazily and does not end up staying buffered in memory.)
* force sofar calculationGravatar Joey Hess2016-12-08
| | | | | This could avoid a memory leak. It would only happen when the meter didn't look at sofar.
* more p2p progress metersGravatar Joey Hess2016-12-07
| | | | | | | | | Display progress meter on send and receive from remote. Added a new hGetMetered that can read an exact number of bytes (or less), updating a meter as it goes. This commit was sponsored by Andreas on Patreon.