aboutsummaryrefslogtreecommitdiff
path: root/Utility
Commit message (Collapse)AuthorAge
* Escape ':' in file/directory names to avoid it being treated as a pathspec ↵Gravatar Joey Hess2013-08-01
| | | | | | | | | | | | | | | | | | | | | | | by some git commands A git pathspec is a filename, except when it starts with ':', it's taken to refer to a branch, etc. Rather than special case ':', any filename starting with anything unusual is prefixed with "./" This could have been a real mess to deal with, but luckily SafeCommand is already extensively used and so we know at the type level the difference between parameters that are files, and parameters that are command options. Testing did show that Git.Queue was not using SafeCommand on filenames fed to xargs. (Filenames starting with '-' worked before only because -- was used to separate filenames from options when calling eg git add.) The test suite now passes with filenames starting with ':'. However, I did not keep that change to it, because such filenames are probably not legal on windows, and I have enough ugly windows ifdefs in there as it is. This commit was sponsored by Otavio Salvador. Thanks!
* Fix a few bugs involving filenames that are at or near the filesystem's ↵Gravatar Joey Hess2013-07-30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | maximum filename length limit. Started with a problem when running addurl on a really long url, because the whole url is munged into the filename. Ended up doing a fairly extensive review for places where filenames could get too large, although it's hard to say I'm not missed any.. Backend.Url had a 128 character limit, which is fine when the limit is 255, but not if it's a lot shorter on some systems. So check the pathconf() limit. Note that this could result in fromUrl creating different keys for the same url, if run on systems with different limits. I don't see this is likely to cause any problems. That can already happen when using addurl --fast, or if the content of an url changes. Both Command.AddUrl and Backend.Url assumed that urls don't contain a lot of multi-byte unicode, and would fail to truncate an url that did properly. A few places use a filename as the template to make a temp file. While that's nice in that the temp file name can be easily related back to the original filename, it could lead to `git annex add` failing to add a filename that was at or close to the maximum length. Note that in Command.Add.lockdown, the template is still derived from the filename, just with enough space left to turn it into a temp file. This is an important optimisation, because the assistant may lock down a bunch of files all at once, and using the same template for all of them would cause openTempFile to iterate through the same set of names, looking for an unused temp file. I'm not very happy with the relatedTemplate hack, but it avoids that slowdown. Backend.WORM does not limit the filename stored in the key. I have not tried to change that; so git annex add will fail on really long filenames when using the WORM backend. It seems better to preserve the invariant that a WORM key always contains the complete filename, since the filename is the only unique material in the key, other than mtime and size. Since nobody has complained about add failing (I think I saw it once?) on WORM, probably it's ok, or nobody but me uses it. There may be compatability problems if using git annex addurl --fast or the WORM backend on a system with the 255 limit and then trying to use that repo in a system with a smaller limit. I have not tried to deal with those. This commit was sponsored by Alexander Brem. Thanks!
* importfeed: git-annex becomes a podcatcher in 150 LOCGravatar Joey Hess2013-07-28
|
* cleanupGravatar Joey Hess2013-07-20
|
* refactor and unify codeGravatar Joey Hess2013-07-19
| | | | This fixes several bugs in both modules.
* pluralize 1.1 kilobytes etcGravatar Joey Hess2013-07-19
|
* Display byte sizes with more precision.Gravatar Joey Hess2013-07-19
|
* catch does not exist error when adding a watchGravatar Joey Hess2013-07-17
| | | | | This could be thrown due to eg, the directory being moved or deleted, so the error should not be propigated.
* linux standalone auto-install iconsGravatar Joey Hess2013-07-09
|
* install to ~/.local/icons, not ~/iconsGravatar Joey Hess2013-07-09
| | | | | Apparently the Icon Theme Specification no longer matches reality, as implemented by XFCE and xdg-icon-resource.
* Install XDG desktop icon files.Gravatar Joey Hess2013-07-09
| | | | | | | | | | | | | | | | The icon files will be installed when running make install or cabal install. Did not try to run update-icon-caches, since I think it's debian specific, and dh_icons will take care of that for the Debian package. Using the favicon as a 16x16 icon. At 24x24 the svg displays pretty well, although the dotted lines are rather faint. The svg is ok at all higher resolutions. The standalone linux build auto-installs the desktop and autostart files when run. I have not made it auto-install the icon file too, because a) that would take more work to include them in the tarball and find them b) it would need to be an install to ~/.icons/, and I don't know if that really works!
* reorgGravatar Joey Hess2013-07-08
|
* Pass -f to curl when downloading a file with it, so it propigates failure.Gravatar Joey Hess2013-07-06
|
* Windows: Look for .exe extension when searching for a command in path.Gravatar Joey Hess2013-07-06
|
* fix a build failure on androidGravatar Joey Hess2013-06-27
|
* clean up build warnings with yesod 1.2, while still building with 1.1Gravatar Joey Hess2013-06-27
|
* webapp: Fix bug that caused the webapp to hang when built with yesod 1.2.Gravatar Joey Hess2013-06-27
|
* assistant: Fix bug that prevented adding files written by gnucash, and more ↵Gravatar Joey Hess2013-06-26
| | | | generally support adding hard links to files. However, other operations on hard links are still unsupported.
* simpler ifdef for linuxGravatar Joey Hess2013-06-21
|
* assistant: On Linux, the expensive transfer scan is run niced.Gravatar Joey Hess2013-06-20
| | | | | | | This is a compromise. I would like to nice every thread except for the webapp thread, but it's not practical to do so. That would need every thread to run as a bound thread, which could add significant overhead. And any forkIO would escape the nice level.
* fix regression test on windowsGravatar Joey Hess2013-06-18
|
* make withQuietOutput work on WindowsGravatar Joey Hess2013-06-17
|
* assistant: In direct mode, objects are now only dropped when all associated ↵Gravatar Joey Hess2013-06-15
| | | | files are unwanted. This avoids a repreated drop/get loop of a file that has a copy in an archive directory, and a copy not in an archive directory. (Indirect mode still has some buggy behavior in this area, since it does not keep track of associated files.) Closes: #712060
* got file descriptors mixed up in last commitGravatar Joey Hess2013-06-14
|
* Windows: Fix hang when adding several files at once.Gravatar Joey Hess2013-06-14
|
* Android: Fix use of cp command to not try to use features present only on ↵Gravatar Joey Hess2013-06-14
| | | | build system.
* Supports indirect mode on encfs in paranoia mode, and other filesystems that ↵Gravatar Joey Hess2013-06-10
| | | | do not support hard links, but do support symlinks and other POSIX filesystem features.
* now builds with both yesod 1.2 and 1.1Gravatar Joey Hess2013-06-03
|
* add liftH shim between yesod versions, to avoid needing zillions of ifdefsGravatar Joey Hess2013-06-03
|
* WIP yesod 1.2Gravatar Joey Hess2013-06-03
|
* avoid warnings when built with ghc 7.6Gravatar Joey Hess2013-06-02
|
* Android: Work around Android devices where the `am` command doesn't work.Gravatar Joey Hess2013-05-31
|
* Can now restart certain long-running git processes if they crash, and ↵Gravatar Joey Hess2013-05-31
| | | | | | | | | | | | | | | | | | | continue working. Fuzz tests have shown that git cat-file --batch sometimes stops running. It's not yet known why (no error message; repo seems ok). But this is something we can deal with in the CoProcess framework, since all 3 types of long-running git processes should be restartable if they fail. Note that, as implemented, only IO errors are caught. So an error thrown by the reveiver, when it sees something that is not valid output from git cat-file (etc) will not cause a restart. I don't want it to retry if git commands change their output or are just outputting garbage. This does mean that if the command did a partial output and crashed in the middle, it would still not be restarted. There is currently no guard against restarting a command repeatedly, if, for example, it crashes repeatedly on startup.
* remove debug printsGravatar Joey Hess2013-05-30
|
* fix a minor fd leakGravatar Joey Hess2013-05-27
|
* when xmpp connection fails, show the host(s) it tried to connect toGravatar Joey Hess2013-05-27
|
* fix warningGravatar Joey Hess2013-05-26
|
* more windows fixesGravatar Joey Hess2013-05-26
|
* typoGravatar Joey Hess2013-05-26
|
* fix windows buildGravatar Joey Hess2013-05-26
|
* fix handling of Not in the matcherGravatar Joey Hess2013-05-25
|
* Improve error handling when getting uuid of http remotes to auto-ignore, ↵Gravatar Joey Hess2013-05-25
| | | | like with ssh remotes.
* Fix bug in parsing of parens in some preferred content expressions. This ↵Gravatar Joey Hess2013-05-24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | fixes the behavior of the manual mode group. The current manual mode preferred content expression is: "present and (((exclude=*/archive/* and exclude=archive/*) or (not (copies=archive:1 or copies=smallarchive:1))) or (not copies=semitrusted+:1))" The old matcher misparsed this, to basically: OR (present and (...)) (not copies=semitrusted+:1)) The paren handling and indeed the whole conversion from tokens to the matcher was just wrong. The new way may not be the cleverest, but I think it is correct, and you can see how it pattern matches structurally against the expressions when parsing them. That expression is now parsed to: MAnd (MOp <function>) (MOr (MOr (MAnd (MOp <function>) (MOp <function>)) (MNot (MOr (MOp <function>) (MOp <function>)))) (MNot (MOp <function>))) Which appears correct, and behaves correct in testing. Also threw in a simplifier, so the final generated Matcher has less unnecessary clutter in it. Mostly so that I could more easily read & confirm them. Also, added a simple test of the Matcher to the test suite. There is a small chance of badly formed preferred content expressions behaving differently than before due to this rewrite.
* add two long-running XMPP push threads, no more inversion of controlGravatar Joey Hess2013-05-22
| | | | | | | | | I hope this will be easier to reason about, and less buggy. It was certianly easier to write! An immediate benefit is that with a traversable queue of push requests to select from, the threads can be a lot fairer about choosing which client to service next.
* better nukefileGravatar Joey Hess2013-05-21
| | | | | Fixed handling of case when file does not exist to work like it did before, and avoid an excess stat call.
* better nukeFileGravatar Joey Hess2013-05-21
| | | | This fixes a bug where a direct mode sync fails to delete broken symlinks.
* Allow building with gpg2.Gravatar Joey Hess2013-05-19
|
* Fix a zombie that could result when running a process like gpg to read and ↵Gravatar Joey Hess2013-05-19
| | | | write to it.
* Switch to MonadCatchIO-transformers for better handling of state while ↵Gravatar Joey Hess2013-05-19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | catching exceptions. As seen in this bug report, the lifted exception handling using the StateT monad throws away state changes when an action throws an exception. http://git-annex.branchable.com/bugs/git_annex_fork_bombs_on_gpg_file/ .. Which can result in cached values being redundantly calculated, or other possibly worse bugs when the annex state gets out of sync with reality. This switches from a StateT AnnexState to a ReaderT (MVar AnnexState). All changes to the state go via the MVar. So when an Annex action is running inside an exception handler, and it makes some changes, they immediately go into affect in the MVar. If it then throws an exception (or even crashes its thread!), the state changes are still in effect. The MonadCatchIO-transformers change is actually only incidental. I could have kept on using lifted-base for the exception handling. However, I'd have needed to write a new instance of MonadBaseControl for the new monad.. and I didn't write the old instance.. I begged Bas and he kindly sent it to me. Happily, MonadCatchIO-transformers is able to derive a MonadCatchIO instance for my monad. This is a deep level change. It passes the test suite! What could it break? Well.. The most likely breakage would be to code that runs an Annex action in an exception handler, and *wants* state changes to be thrown away. Perhaps the state changes leaves the state inconsistent, or wrong. Since there are relatively few places in git-annex that catch exceptions in the Annex monad, and the AnnexState is generally just used to cache calculated data, this is unlikely to be a problem. Oh yeah, this change also makes Assistant.Types.ThreadedMonad a bit redundant. It's now entirely possible to run concurrent Annex actions in different threads, all sharing access to the same state! The ThreadedMonad just adds some extra work on top of that, with its own MVar, and avoids such actions possibly stepping on one-another's toes. I have not gotten rid of it, but might try that later. Being able to run concurrent Annex actions would simplify parts of the Assistant code.
* fix the day's Windows permissions damageGravatar Joey Hess2013-05-14
|