diff options
| author |  Joey Hess <joeyh@joeyh.name> | 2015-03-27 10:22:32 -0400 |
|---|---|---|
| committer | Joey Hess <joeyh@joeyh.name> | 2015-03-27 10:22:36 -0400 |
| commit | b1483ee577dded9e00f8d82d52a806ee02888b2c (patch) | |
| tree | ad16d663918d043320587a5b2aa6539cd61e7397 /Utility/Url.hs | |
| parent | be1f27f20455080471472afbc48763e668c6b97b (diff) | |
Work around curl bug when asked to download an empty url to a file.
In this situation, curl -o exits successfully without creating the output
file.
There was already a workaround for curl file:/// but I did not realize this
also affected regular url downloads.
To fix it, pre-create the destination file before starting curl.
Since we cannot always know the size of an url before trying to download
it, let's always do this.
Note that since curl is told -C -, we have to consider if this
makes curl try to do a ranged download, which might fail on some servers
where a regular download would have succeeded. My testing indicates
this isn't a problem; since the file is empty, curl seems to not try to
do a ranged download.
Original report: https://github.com/datalad/datalad/issues/79
Curl bug report: https://github.com/bagder/curl/issues/183
Diffstat (limited to 'Utility/Url.hs')
| -rw-r--r-- | Utility/Url.hs | 11 |
1 files changed, 6 insertions, 5 deletions
diff --git a/Utility/Url.hs b/Utility/Url.hs index ddf5eea40..07be955e2 100644 --- a/Utility/Url.hs +++ b/Utility/Url.hs @@ -209,9 +209,6 @@ download' quiet url file uo = case parseURIRelaxed url of Just u | uriScheme u == "file:" -> do - -- curl does not create destination file - -- for an empty file:// url, so pre-create - writeFile file "" curl | otherwise -> ifM (inPath "wget") (wget , curl) _ -> return False @@ -240,8 +237,12 @@ download' quiet url file uo = - the remainder to download as the whole file, - and not indicating how much percent was - downloaded before the resume. -} - curl = go "curl" $ headerparams ++ quietopt "-s" ++ - [Params "-f -L -C - -# -o"] + curl = do + -- curl does not create destination file + -- if the url happens to be empty, so pre-create. + writeFile file "" + go "curl" $ headerparams ++ quietopt "-s" ++ + [Params "-f -L -C - -# -o"] go cmd opts = boolSystem cmd $ addUserAgent uo $ reqParams uo++opts++[File file, File url] quietopt s |