diff options
| author |  Joey Hess <joeyh@joeyh.name> | 2015-03-27 10:22:32 -0400 |
|---|---|---|
| committer | Joey Hess <joeyh@joeyh.name> | 2015-03-27 10:22:36 -0400 |
| commit | b1483ee577dded9e00f8d82d52a806ee02888b2c (patch) | |
| tree | ad16d663918d043320587a5b2aa6539cd61e7397 | |
| parent | be1f27f20455080471472afbc48763e668c6b97b (diff) | |
Work around curl bug when asked to download an empty url to a file.
In this situation, curl -o exits successfully without creating the output
file.
There was already a workaround for curl file:/// but I did not realize this
also affected regular url downloads.
To fix it, pre-create the destination file before starting curl.
Since we cannot always know the size of an url before trying to download
it, let's always do this.
Note that since curl is told -C -, we have to consider if this
makes curl try to do a ranged download, which might fail on some servers
where a regular download would have succeeded. My testing indicates
this isn't a problem; since the file is empty, curl seems to not try to
do a ranged download.
Original report: https://github.com/datalad/datalad/issues/79
Curl bug report: https://github.com/bagder/curl/issues/183
| -rw-r--r-- | Utility/Url.hs | 11 | ||||
| -rw-r--r-- | debian/changelog | 1 |
2 files changed, 7 insertions, 5 deletions
diff --git a/Utility/Url.hs b/Utility/Url.hs index ddf5eea40..07be955e2 100644 --- a/Utility/Url.hs +++ b/Utility/Url.hs @@ -209,9 +209,6 @@ download' quiet url file uo = case parseURIRelaxed url of Just u | uriScheme u == "file:" -> do - -- curl does not create destination file - -- for an empty file:// url, so pre-create - writeFile file "" curl | otherwise -> ifM (inPath "wget") (wget , curl) _ -> return False @@ -240,8 +237,12 @@ download' quiet url file uo = - the remainder to download as the whole file, - and not indicating how much percent was - downloaded before the resume. -} - curl = go "curl" $ headerparams ++ quietopt "-s" ++ - [Params "-f -L -C - -# -o"] + curl = do + -- curl does not create destination file + -- if the url happens to be empty, so pre-create. + writeFile file "" + go "curl" $ headerparams ++ quietopt "-s" ++ + [Params "-f -L -C - -# -o"] go cmd opts = boolSystem cmd $ addUserAgent uo $ reqParams uo++opts++[File file, File url] quietopt s diff --git a/debian/changelog b/debian/changelog index c41a3c0b5..bb487279b 100644 --- a/debian/changelog +++ b/debian/changelog @@ -17,6 +17,7 @@ git-annex (5.20150318) UNRELEASED; urgency=medium * assistant: Committing a whole lot of files at once could overflow command-line length limits and cause the commit to fail. This only happened when using the assistant in an indirect mode repository. + * Work around curl bug when asked to download an empty url to a file. -- Joey Hess <id@joeyh.name> Thu, 19 Mar 2015 17:05:32 -0400 |