| Commit message (Collapse) | Author | Age |
|---|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
base-4.9 split MonadFail from Monad. Strengthen some type signatures to
require MonadFail instead of just Monad, derive MonadFail in relevant
places, and change a partial pattern match inside STM to one that
explicitly calls error. (STM is not a MonadFail; the user must
explicitly specify the desired semantics if a pattern match doesn’t work
out. In this case, the failing branch of the pattern should never be
reached, so crashing is fine.)
|
| |
|
|
|
| |
This repository doesn’t contain the webapp. Remove dangling references
to it.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Much like Remote.P2P, there's a pool of connections to a peer, in order
to support concurrent operations.
Deals with old git-annex-ssh on the remote that does not support p2pstdio,
by only trying once to use it, and remembering if it's not supported.
Made p2pstdio send an AUTH_SUCCESS with its uuid, which serves the dual
purposes of something to detect to see that the connection is working,
and a way to verify that it's connected to the right uuid.
(There's a redundant uuid check since the uuid field is sent
by git_annex_shell, but I anticipate that being removed later when
the legacy git-annex-shell stuff gets removed.)
Not entirely happy with Remote.Git.runSsh's behavior
when the proto action fails. Running the fallback will work ok, but what
will we do when the fallbacks later get removed? It might be better to
try to reconnect, in case the connection got closed.
This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
|
| |
|
|
|
|
| |
Avoids ugly messages when forced ssh command is not git-annex-shell.
This commit was sponsored by Ole-Morten Duesund on Patreon.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Fourth or fifth try at this and finally found a way to make it work.
Absurd amount of busy-work forced on me by change in cabal's behavior.
Split up Utility modules that need posix stuff out of ones used by
Setup. Various other hacks around inability for Setup to use anything
that ifdefs a use of unix.
Probably lost a full day of my life to this.
This is how build systems make their users hate them. Just saying.
|
| |
|
|
|
|
|
|
|
|
|
| |
works."
This reverts commit c7c745d28868c8cd646d3e31dd8ba6a0b763f611.
No, still doesn't work when built with cabal. It did with stack; stack
must somehow make the unix package implicitly available.
With cabal, System.Posix.Process and System.Posix.Env are both missing.
|
| |
|
|
|
|
|
|
| |
Seems I had all the work in past commits to make this build, at least on
linux. I'm actually surprised it does, without a unix dep, Utility.Env
still builds ok somehow despite using System.Posix.Env.
This commit was sponsored by Fernando Jimenez on Patreon.
|
| |
|
|
|
|
|
|
|
|
|
| |
This avoids warnings from stack about the module not being listed in the
cabal file. So, the generated file is also renamed to Build/SysConfig.
Note that the setup program seems to be cached despite these changes; I
had to cabal clean to get cabal to update it so that Build/SysConfig was
written.
This commit was sponsored by Jochen Bartl on Patreon.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Now youtubeDlCheck downloads the beginning of the url's content and
checks if it's html, only when it is does it pass it off the youtube-dl
to check if it supports it.
This means more work is done for urls that youtube-dl does support,
but is probably more efficient for other urls, since it only downloads
the first chunk of content, while youtube-dl probably downloads more.
As well as the reported bug, this also fixes behavior when an url
was added with youtube-dl, but the url content has now changed from
a html page to something else. Remote.Web.checkKey used to wrongly
succeed in that situation, since youtube-dl said sure it can download
that something else.
This commit was supported by the NSF-funded DataLad project.
|
| |
|
|
|
|
|
|
| |
initremote, enableremote: Really support gpg subkeys suffixed with an
exclamation mark, which forces gpg to use a specific subkey. (Previous try
had a bug.)
This commit was sponsored by Jake Vosloo on Patreon.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fully working, including --fast/--relaxed.
Note that, while git-annex addurl --relaxed is not going to check
youtube-dl, I kept git annex importfeed --relaxed checking it.
Thinking is that, let's not break people's importfeed cron jobs, and
importfeed does not typically have to check a large number of new items,
so it's ok if it's a little bit slower when used with youtube playlist
feeds.
importfeed's behavior is also improved (?) when a feed has links in it
to non-media files. Before, those were skipped. Now, the content of the
link is downloaded. This had to be done, because trying to use
youtube-dl is slow, and if those were skipped, it would have to check
every time importfeed was run. While this behavior change may not be
desirable for some feeds, that intersperse links to web pages with
enclosures, it will be desirable for other feeds, that have
non-enclosure directy links to media files.
Remove old quvi modules.
This commit was sponsored by Øyvind Andersen Holm.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This will be used in youtube-dl integration, to tell when a html page has
been downloaded by addurl, in which case it is worth running youtube-dl
to see if it can extract media from it.
tagsoup is an almost free dependency, because yesod depends on it.
So, this only really adds a dep when git-annex is built without the
webapp.
I'd like this to as closely as possible match how browsers decide if a
page is html or not. Unfortunately, that is fairly heuristic, in order
to support malformed html. And, we don't want to falsely detect
something as html just because it has something that looks like a html
tag embedded somewhere in it. Probably any major video hosting site is
going to be serving html documents that at least start with a <html>
tag, so requiring that or a DOCTYPE should be good enough.
This commit was sponsored by Jeff Goeke-Smith on Patreon.
|
| |
|
|
| |
This commit was sponsored by Henrik Riomar on Patreon.
|
| |
|
|
|
|
|
|
| |
They need unix on non-windows, for Utility.Env, which Build.Configure uses,
but cabal can't express that in a custom-setup stanza.
To avoid this problem, Utility.Env would need to be moved into
unix-compat..
|
| |
|
|
|
|
|
|
|
|
|
| |
Windows needs the setenv package in custom-setup, but I don't want to
pull it in on unix, which would probably break some builds and need more
work. Instead, split out setEnv to a separate module.
Quite likely, unix-compat will get a portable environment layer, and
then both modules can be removed from here.
This commit was sponsored by Øyvind Andersen Holm.
|
| | |
|
| |
|
|
| |
Re-applying ac57659e61f9743aebd35258e89752ced0040f9f
|
| |
|
|
| |
This reverts commit ce44f20f128ad8740b9098ad86ed70156919cd43.
|
| |
|
|
|
|
| |
This reverts commit ac57659e61f9743aebd35258e89752ced0040f9f.
Too early for this; needs newer Win32 version. Le sigh.
|
| |
|
|
|
|
| |
This reverts commit afaf1799d47d88e1e0ace4f1bb128684761e11c1.
Yeah, too early for that too
|
| |
|
|
| |
Win32 now has its own getCurrentProcessId.
|
| |
|
|
|
|
|
|
|
|
|
| |
That version has my patches for the problems that Utility.PosixFiles
was working around, so am able to get rid of that module now.
This will later allow bringing back the custom-setup stanza in the cabal
file. It will need to depend on unix-compat 0.5 on all OS's, which I'm
not ready to do yet.
This commit was sponsored by Nick Daly on Patreon.
|
| |
|
|
|
| |
dropDrive needed since if splitPath splits out the drives, they would
appear different.
|
| |
|
|
| |
win32 upstream suggested a better name
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Windows: Fix reversion that caused the path used to link to annexed
content include the drive letter and full path, rather than being
relative. (`git annex fix` will fix up after this problem).
I've not identified the commit that brought the reversion (probably it
happened this spring when I was removing MisingH and last touched
Utility.Path). Likely commit c88dfda7e03cab4d96417518b6cfcffd54ad8f5d?
The problem is that relPathDirToFile got called two paths that had the
slashes different ways around. Since takeDrive includes the first slash,
this made two paths on the same drive seem different and it bailed.
(ifdefs around this to avoid doing extra work on non-windows)
This commit was sponsored by Jack Hill on Patreon.
|
| |
|
|
|
|
|
|
|
| |
Get ugly reversion out of CHANGELOG.
Also, relocated the windows stack.yaml to top, and updated windows build
instructions.
This commit was sponsored by Henrik Riomar on Patreon.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Code for terminating processes on Windows is not linking anymore;
made a warning be displayed instead. This breaks restarting the
assistant and git annex assistant --stop.
I hope to see the code added to the Win32 library, where it should fit
better and should avoid whatever problem is making the linker not like it
when included in git-annex. I opened an issue requesting its addition,
here: https://github.com/haskell/win32/issues/91
This commit was sponsored by Thomas Hochstein on Patreon.
|
| |
|
|
|
|
|
|
|
|
| |
This reverts commit d6d0b1fe20397bf073f11d579f5c0c38785e071a.
Neither way is working.. The other way failed:
.stack-work\dist\5f9bc736\build\git-annex\git-annex-tmp\Assistant.o:fake:(.text+0x6bb3): undefined reference to `terminatepid'
Seems that winprocess.c is not getting linked in.
|
| | |
|
| |
|
|
|
|
|
|
|
| |
Building with stack, it failed:
`_TerminateProcess' referenced in section `.text' of .stack-work\dist\5f9bc736\build\git-annex\git-annex-tmp\Utility\WinProcess.o: defined in discarded section `.text' of C:/Users/jenkins/AppData/Local/Programs/stack/i386-windows/ghc-8.0.2/mingw/bin/../lib/gcc/i686-w64-mingw32/5.2.0/../../../../i686-w64-mingw32/lib/../lib/libkernel32.a(dacgs01154.o)
This is a reversion of 6d66a81ca98f8579bf06f7fb724ed733670d39b9,
to try the other way to implement it, which will hopefully avoid the problem.
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
webdav: Checking if a non-existent file is present on Box.com triggered a
bug in its webdav support that generates an infinite series of redirects.
It seems to redirect foo to foo/ to foo/index.php to
foo/index.php/index.php ... Why a webdav endpoint would behave this way
who knows.
Deal with such problems by assuming such behavior means the file is not
present.
Can't simply disable following redirects, because the webdav endpoint could
legitimately be redirected to a new endpoint. So, when this happens
10 redirects have to be followed, before it gives up and assumes this means
the file does not exist.
This commit was supported by the NSF-funded DataLad project.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This avoids needing to deal with the complexity of partially transferred
files in the export. We'd not be able to resume uploading to such a file
anyway, so just avoid them.
The implementation in Remote.Directory is not completely ideal, because
it could leave the temp file hanging around in the export directory.
This only happens if it's killed with -9, or there's a power failure;
normally viaTmp cleans up after itself, even when interrupted. I could
not see a better way to do it though, since the export directory might
be the root of a filesystem.
Also some design thoughts on resuming, which depend on storeExport being
atomic.
This commit was sponsored by Fernando Jimenez on Partreon.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Security fix: Disallow hostname starting with a dash, which would get
passed to ssh and be treated an option. This could be used by an attacker
who provides a crafted ssh url (for eg a git remote) to execute arbitrary
code via ssh -oProxyCommand.
No CVE has yet been assigned for this hole.
The same class of security hole recently affected git itself,
CVE-2017-1000117.
Method: Identified all places where ssh is run, by git grep '"ssh"'
Converted them all to use a SshHost, if they did not already, for
specifying the hostname.
SshHost was made a data type with a smart constructor, which rejects
hostnames starting with '-'.
Note that git-annex already contains extensive use of Utility.SafeCommand,
which fixes a similar class of problem where a filename starting with a
dash gets passed to a program which treats it as an option.
This commit was sponsored by Jochen Bartl on Patreon.
|
| | |
|
| | |
|
| |
|
|
| |
to check if it exists. Some web servers take quite a long time to answer a HEAD request.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
By forking a worker process and only deleting the test directory once it exits.
This way, if a test leaves files open, they'll get closed when the worker
exits, so avoiding failure to delete open files on Windows, and failure to
delete directories due to NFS lock files.
If a test leaves a git worker process running, the closed pipes should
cause the worker to exit too, also avoiding the problem there. The 10
second sleep ought to give plenty of time for such worker processes to
exit, although this is of course a race.
Finally, even if test directory fails to be deleted still,
it won't appear as if the last test in the test suite failed; the error
will be displayed at the very end.
This commit was supported by the NSF-funded DataLad project.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
QuickCheck 2.10 found a counterexample eg "\929184" broke the property.
As far as I can tell, Git.Filename is matching how git handles encoding
of strange high unicode characters in filenames for display. Git does
not display high unicode characters, and instead displays the C-style
escaped form of each byte. This is ambiguous, but since git is not
unicode aware, it doesn't need to roundtrip parse it.
So, making Git.FileName's roundtrip test only chars < 256 seems fine.
Utility.Format.format uses encode_c, in order to mimic git, so that's
ok.
Utility.Format.gen uses decode_c, but only so that stuff like "\n"
in the format string is handled. If the format string contains C-style
octal escapes, they will be converted to ascii characters, and not
combined into unicode characters, but that should not be a problem.
If the user wants unicode characters, they can include them in the
format string, without escaping them.
Finally, decode_c is used by Utility.Gpg.secretKeys, because gpg
--with-colons hex-escapes some characters in particular ':' and '\\'.
gpg passes unicode through, so this use of decode_c is not a problem.
This commit was sponsored by Henrik Riomar on Patreon.
|
| |
|
|
|
|
|
|
| |
QuickCheck added an Arbitrary instance for CTime aka EpochTime. However,
while git-annex's instance disallowed times before the epoch, QuickCheck's
does not. So, rather than using its instance, convert from an Integer.
This commit was sponsored by Thomas Hochstein on Patreon.
|
| |
|
|
|
|
|
|
| |
Don't trust OSX FSEvents's eventFlagItemModified to be called when the last
writer of a file closes it; apparently that sometimes does not happen,
which prevented files from being quickly added.
This commit was sponsored by John Peloquin on Patreon.
|
| | |
|
Benjamin Barenblat
Joey Hess