diff options
| author |  Joey Hess <joeyh@joeyh.name> | 2017-08-15 13:56:12 -0400 |
|---|---|---|
| committer | Joey Hess <joeyh@joeyh.name> | 2017-08-15 13:56:12 -0400 |
| commit | 0c4c9db9e8860d05eab5b9766d8719ceec1b0b9d (patch) | |
| tree | 32d0d726c47cc3b4ca5bb3e6d7f4ae169b13bb42 /doc | |
| parent | d4b2c4b56ca97c7e51f4fd122d54e23fa7207078 (diff) | |
Disable http-client's default 30 second response timeout when HEADing an url to check if it exists. Some web servers take quite a long time to answer a HEAD request.
Diffstat (limited to 'doc')
2 files changed, 27 insertions, 0 deletions
diff --git a/doc/bugs/fails_to_verify_presence_via_http_while_wget_fetches_it_just_fine.mdwn b/doc/bugs/fails_to_verify_presence_via_http_while_wget_fetches_it_just_fine.mdwn index ccd0e8543..2dc1e641b 100644 --- a/doc/bugs/fails_to_verify_presence_via_http_while_wget_fetches_it_just_fine.mdwn +++ b/doc/bugs/fails_to_verify_presence_via_http_while_wget_fetches_it_just_fine.mdwn @@ -47,3 +47,5 @@ git-annex: drop: 1 failed [[!meta author=yoh]] + +> [[done]] --[[Joey]] diff --git a/doc/bugs/fails_to_verify_presence_via_http_while_wget_fetches_it_just_fine/comment_1_fa6649208f1882a6bb412ba40cf57fec._comment b/doc/bugs/fails_to_verify_presence_via_http_while_wget_fetches_it_just_fine/comment_1_fa6649208f1882a6bb412ba40cf57fec._comment new file mode 100644 index 000000000..ed23e8902 --- /dev/null +++ b/doc/bugs/fails_to_verify_presence_via_http_while_wget_fetches_it_just_fine/comment_1_fa6649208f1882a6bb412ba40cf57fec._comment @@ -0,0 +1,25 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 1""" + date="2017-08-15T17:28:20Z" + content=""" +The normal reason for this to happen is if the size of the file +on the website has changed. git-annex checks the reported size and if it +differs from the versioned file, it knows that the website no longer +contains the same file. + +In this case, it seems to be a cgi program generating a zip file, and the +program actually generated two different zip files when I hit it twice with +wget. (So if git-annex actually did drop the only copy of the version you +downloaded, you'd not be able to download it again. Not that git-annex can know +that; this kind of thing is why trusting the web is not a good idea..) They did +have the same size, but it looks like the web server is not sending a size +header anyway. + +The actual problem is the web server takes a long time to answer a HEAD request +for this URL. It takes 35 seconds before curl is able to HEAD it. I suspect +it's generating the 300 mb zip file before it gets around to finishing +the HEAD request. Not the greatest server behavior, all around. + +That breaks http-client due to its default 30 second timeout. So, will remove that timeout then. +"""]] |