diff options
author | 2013-03-08 19:34:33 -0400 | |
---|---|---|
committer | 2013-03-08 19:34:33 -0400 | |
commit | 51fbedc14e76d0a584cc13afea7babfc215d67c8 (patch) | |
tree | 013ddd36bde8776bb5bb3aae4e17208d618e7353 | |
parent | 88baed963571908ed6bba6aa7e740f830897038e (diff) |
blog for the day
-rw-r--r-- | doc/bugs/webapp_hang.mdwn | 136 | ||||
-rw-r--r-- | doc/design/assistant/blog/day_208__bugfixes.mdwn | 17 |
2 files changed, 153 insertions, 0 deletions
diff --git a/doc/bugs/webapp_hang.mdwn b/doc/bugs/webapp_hang.mdwn new file mode 100644 index 000000000..dde19c3e3 --- /dev/null +++ b/doc/bugs/webapp_hang.mdwn @@ -0,0 +1,136 @@ +Occasionally, clicking on a link in the webapp will hang. When this +happens, which has only been seen in Chromium so far, the tab will spin +forever, without anything loading. Other tabs can be opened with +middle-click on links in the webapp, and work fine. Stopping and reloading +in the tab tends to hang again, although eventually this will clear the +hang up. + +------- + +My best procedure to replicate this, about 25% of the time: + +* have 2 large files and a libreoffice spreadsheet +* start webapp, make repository +* open file browser using button in webapp +* move files into repository +* make media subdir; move media into it +* open spreadsheet, modify, save +* click on New Repository button for the hang + +Running recordmydesktop at the same time may also help.. Or giving a +presentation in Australia. :/ + +------- + +Hypotheses: + +* Warp's slowloris protection could be triggering. Possibly by the + repeated hits to update the alerts? I added a settingsOnException handler + that logs all exceptions, and ThreadKilled is happening several times. + The only place in Warp that kills threads is due to a timeout installed + for that. +* Something deep in git-annex, such as the inotidy code, could be + preventing a web server thread from running. But then why do other + tabs and other web browsers work while it's stuck? + It would have to affect only 1 thread. + +------- + +I captured a clean protocol dump of this happening. It includes only the +final, hanging http request and subsequent traffic, not the setup. + +We can see the web browser send a request. The server ACKs it at the TCP +level, but never sends any reply. The web browser continues sending TCP +keep-alive packets, which are acked by the server. This continued as long +as the browser tab was left open. + +Question: Did the browser send a complete & valid request? The last part +sent is a cookie and "\r\n\r\n".. So it seems complete. Unless warp is +expecting a request body? + +<pre> +17:22:30.533079 IP localhost.localdomain.53239 > localhost.localdomain.45836: Flags [P.], seq 4237976899:4237977772, ack 2608808926, win 2048, options [nop,nop,TS val 4895706 ecr 4885760], length 873 + 0x0000: 4500 039d be84 4000 4006 7ad4 7f00 0001 E.....@.@.z..... + 0x0010: 7f00 0001 cff7 b30c fc9a 6543 9b7f 43de ..........eC..C. + 0x0020: 8018 0800 0192 0000 0101 080a 004a b3da .............J.. + 0x0030: 004a 8d00 4745 5420 2f63 6f6e 6669 672f .J..GET./config/ + 0x0040: 7265 706f 7369 746f 7279 3f61 7574 683d repository?auth= + 0x0050: 6437 6266 3037 3438 6663 3863 3031 3965 d7bf0748fc8c019e + 0x0060: 6230 3966 3530 3631 6164 6663 3861 3563 b09f5061adfc8a5c + 0x0070: 3430 3437 3633 6562 3736 6630 6163 3533 404763eb76f0ac53 + 0x0080: 3663 3362 6230 3066 3835 6663 6361 3233 6c3bb00f85fcca23 + 0x0090: 6235 3639 3764 3332 3464 3737 3930 3063 b5697d324d77900c + 0x00a0: 3739 3532 6430 6165 3235 3166 6331 6337 7952d0ae251fc1c7 + 0x00b0: 3532 3632 6330 3233 6265 3936 3066 3563 5262c023be960f5c + 0x00c0: 3364 6135 6532 6262 6234 3639 3863 3035 3da5e2bbb4698c05 + 0x00d0: 2048 5454 502f 312e 310d 0a48 6f73 743a .HTTP/1.1..Host: + 0x00e0: 2031 3237 2e30 2e30 2e31 3a34 3538 3336 .127.0.0.1:45836 + 0x00f0: 0d0a 436f 6e6e 6563 7469 6f6e 3a20 6b65 ..Connection:.ke + 0x0100: 6570 2d61 6c69 7665 0d0a 4163 6365 7074 ep-alive..Accept + 0x0110: 3a20 7465 7874 2f68 746d 6c2c 6170 706c :.text/html,appl + 0x0120: 6963 6174 696f 6e2f 7868 746d 6c2b 786d ication/xhtml+xm + 0x0130: 6c2c 6170 706c 6963 6174 696f 6e2f 786d l,application/xm + 0x0140: 6c3b 713d 302e 392c 2a2f 2a3b 713d 302e l;q=0.9,*/*;q=0. + 0x0150: 380d 0a55 7365 722d 4167 656e 743a 204d 8..User-Agent:.M + 0x0160: 6f7a 696c 6c61 2f35 2e30 2028 5831 313b ozilla/5.0.(X11; + 0x0170: 204c 696e 7578 2069 3638 3629 2041 7070 .Linux.i686).App + 0x0180: 6c65 5765 624b 6974 2f35 3337 2e31 3720 leWebKit/537.17. + 0x0190: 284b 4854 4d4c 2c20 6c69 6b65 2047 6563 (KHTML,.like.Gec + 0x01a0: 6b6f 2920 4368 726f 6d65 2f32 342e 302e ko).Chrome/24.0. + 0x01b0: 3133 3132 2e36 3820 5361 6661 7269 2f35 1312.68.Safari/5 + 0x01c0: 3337 2e31 370d 0a52 6566 6572 6572 3a20 37.17..Referer:. + 0x01d0: 6874 7470 3a2f 2f31 3237 2e30 2e30 2e31 http://127.0.0.1 + 0x01e0: 3a34 3538 3336 2f3f 6175 7468 3d64 3762 :45836/?auth=d7b + 0x01f0: 6630 3734 3866 6338 6330 3139 6562 3039 f0748fc8c019eb09 + 0x0200: 6635 3036 3161 6466 6338 6135 6334 3034 f5061adfc8a5c404 + 0x0210: 3736 3365 6237 3666 3061 6335 3336 6333 763eb76f0ac536c3 + 0x0220: 6262 3030 6638 3566 6363 6132 3362 3536 bb00f85fcca23b56 + 0x0230: 3937 6433 3234 6437 3739 3030 6337 3935 97d324d77900c795 + 0x0240: 3264 3061 6532 3531 6663 3163 3735 3236 2d0ae251fc1c7526 + 0x0250: 3263 3032 3362 6539 3630 6635 6333 6461 2c023be960f5c3da + 0x0260: 3565 3262 6262 3436 3938 6330 350d 0a41 5e2bbb4698c05..A + 0x0270: 6363 6570 742d 456e 636f 6469 6e67 3a20 ccept-Encoding:. + 0x0280: 677a 6970 2c64 6566 6c61 7465 2c73 6463 gzip,deflate,sdc + 0x0290: 680d 0a41 6363 6570 742d 4c61 6e67 7561 h..Accept-Langua + 0x02a0: 6765 3a20 656e 2d55 532c 656e 3b71 3d30 ge:.en-US,en;q=0 + 0x02b0: 2e38 0d0a 4163 6365 7074 2d43 6861 7273 .8..Accept-Chars + 0x02c0: 6574 3a20 4953 4f2d 3838 3539 2d31 2c75 et:.ISO-8859-1,u + 0x02d0: 7466 2d38 3b71 3d30 2e37 2c2a 3b71 3d30 tf-8;q=0.7,*;q=0 + 0x02e0: 2e33 0d0a 436f 6f6b 6965 3a20 5f53 4553 .3..Cookie:._SES + 0x02f0: 5349 4f4e 3d45 3363 7455 496c 7341 5451 SION=E3ctUIlsATQ + 0x0300: 3631 694c 6d54 6954 7131 6f37 6465 7830 61iLmTiTq1o7dex0 + 0x0310: 3361 6f57 3049 4b63 7663 467a 5838 4344 3aoW0IKcvcFzX8CD + 0x0320: 5432 7666 4b31 6c42 416d 6279 3166 764f T2vfK1lBAmby1fvO + 0x0330: 4643 7952 7863 6f5a 6277 5633 6a4b 4971 FCyRxcoZbwV3jKIq + 0x0340: 6b35 6958 4674 7557 5261 6b48 6944 6132 k5iXFtuWRakHiDa2 + 0x0350: 7769 3075 2f53 6430 5a49 7a75 4464 7947 wi0u/Sd0ZIzuDdyG + 0x0360: 774f 6a31 7838 3130 356a 772f 5a2b 355a wOj1x8105jw/Z+5Z + 0x0370: 6f4b 6f48 396e 6779 6e4b 5366 5839 742f oKoH9ngynKSfX9t/ + 0x0380: 6862 4b79 435a 6966 7739 4148 3053 6d4b hbKyCZifw9AH0SmK + 0x0390: 436e 4c38 5358 513d 3d0d 0a0d 0a CnL8SXQ==.... +17:22:30.571152 IP localhost.localdomain.45836 > localhost.localdomain.53239: Flags [.], ack 873, win 2048, options [nop,nop,TS val 4895716 ecr 4895706], length 0 + 0x0000: 4500 0034 f15b 4000 4006 4b66 7f00 0001 E..4.[@.@.Kf.... + 0x0010: 7f00 0001 b30c cff7 9b7f 43de fc9a 68ac ..........C...h. + 0x0020: 8010 0800 fe28 0000 0101 080a 004a b3e4 .....(.......J.. + 0x0030: 004a b3da .J.. +17:22:35.803152 IP localhost.localdomain.53240 > localhost.localdomain.45836: Flags [.], ack 2157632553, win 2048, options [nop,nop,TS val 4897024 ecr 4885760], length 0 + 0x0000: 4500 0034 3a63 4000 4006 025f 7f00 0001 E..4:c@.@.._.... + 0x0010: 7f00 0001 cff8 b30c 7533 e963 809a dc29 ........u3.c...) + 0x0020: 8010 0800 fe28 0000 0101 080a 004a b900 .....(.......J.. + 0x0030: 004a 8d00 .J.. +17:22:35.803213 IP localhost.localdomain.45836 > localhost.localdomain.53240: Flags [.], ack 1, win 2048, options [nop,nop,TS val 4897024 ecr 4840696], length 0 + 0x0000: 4500 0034 10e5 4000 4006 2bdd 7f00 0001 E..4..@.@.+..... + 0x0010: 7f00 0001 b30c cff8 809a dc29 7533 e964 ...........)u3.d + 0x0020: 8010 0800 fe28 0000 0101 080a 004a b900 .....(.......J.. + 0x0030: 0049 dcf8 .I.. +17:23:15.611193 IP localhost.localdomain.53239 > localhost.localdomain.45836: Flags [.], ack 1, win 2048, options [nop,nop,TS val 4906976 ecr 4895716], length 0 + 0x0000: 4500 0034 be85 4000 4006 7e3c 7f00 0001 E..4..@.@.~<.... + 0x0010: 7f00 0001 cff7 b30c fc9a 68ab 9b7f 43de ..........h...C. + 0x0020: 8010 0800 fe28 0000 0101 080a 004a dfe0 .....(.......J.. + 0x0030: 004a b3e4 .J.. +17:23:15.611290 IP localhost.localdomain.45836 > localhost.localdomain.53239: Flags [.], ack 873, win 2048, options [nop,nop,TS val 4906976 ecr 4895706], length 0 + 0x0000: 4500 0034 f15c 4000 4006 4b65 7f00 0001 E..4.\@.@.Ke.... + 0x0010: 7f00 0001 b30c cff7 9b7f 43de fc9a 68ac ..........C...h. + 0x0020: 8010 0800 fe28 0000 0101 080a 004a dfe0 .....(.......J.. + 0x0030: 004a b3da .J.. +</pre> diff --git a/doc/design/assistant/blog/day_208__bugfixes.mdwn b/doc/design/assistant/blog/day_208__bugfixes.mdwn new file mode 100644 index 000000000..b366ac7a6 --- /dev/null +++ b/doc/design/assistant/blog/day_208__bugfixes.mdwn @@ -0,0 +1,17 @@ +Fixed the last XMPP bug I know of. Turns out it was not specific to XMPP at +all; the assistant could forget to sync with any repository on startup +under certian conditions. + +Also fixed bugs in `git annex add` and in the glob matching, and some more. + +I've been working on some screencasts. More on them later.. But while doing +them I found a perfect way to reliably reproduce the webapp hang that +I've been chasing for half a year, and last saw at my presentation in +Australia. Seems the old joke about bugs only reproducible during +presentations is literally true here! + +I have given this bug its [[own page|bugs/webapp_hang]] at last, and have a +tcpdump of it happening and everything. Am working on an hypotheses that it +might be caused by Warp's [slowloris](http://ha.ckers.org/slowloris/) +attack prevention code being falsely triggered by the repeated hits the web +browser makes as the webapp's display is updated. |