First of all, many thanks to Claes Wikstrom for isolating and finding the module in which this bug lives (and for doing it so quickly). I originally thought this was a Yaws bug, and emailed Claes with the symptoms.
It's quite repeatable (test code at end of email). He traced the bug to inet_drv.c and emailed me back with a small test program and details, and I offered to
report the bug. Then I got curious as to what it could have been, so I
did some debugging.<br>
<br>The bug is in inet_drv.c. It will cause the end of the HTTP headers to be missed if the first character following the last CRLF (the one on its "own line") is a space or tab. This causes Yaws to time out after 30 seconds waiting for more header data that will never arrive. I am sure this will be true for httpd, too.<br>
<br>The culprit (well, sort of; if you read on you will see why) is the following code on line 8324:<br><br><b style="color: rgb(204, 0, 0);"><span style="font-family: courier new,monospace;">if (SP(ptr2+1)) {</span></b><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> ptr1 = ptr2+1;</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> len = n - plen;</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;">}</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">else</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> goto done;</span><br>
<br>Consider the following HTTP POST data:<br><br><span style="font-family: courier new,monospace;">"POST /invalid/url HTTP/1.1\r\n"</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">"Connection: close\r\n"</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;">"Host: localhost:8000\r\n"</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">"User-Agent: perl post\r\n"</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;">"Content-Length: 4\r\n"</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">"Content-Type: text/xml; charset=utf-8\r\n"</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;">"\r\n"</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">" postdata..." <-- Note this data starts with a space character</span><br>
<br>When the inet_drv.c code gets to the last CRLF, it doesn't detect that it's the last CRLF (end of headers) and checks to see if the next character is a space or tab using the macro SP(ptr2+1). If there's any data following the last CRLF and it starts with a space or tab, the code as it stands now will think it's another header line and try to get more data, then time out when nothing arrives.<br>
<br>What needed to be done was to check to see if the data (in that particular state) was an LF or CRLF standing on its own, and only check for a space following that if not. This check correctly detects the CRLF that terminates the header data. I added two lines of code to inet_drv.c to fix that error, and the Erlang test code (kindly supplied by Claes, and at the end of this email) now runs without error. I can't guarantee that the fix will work under all circumstances, and it needs to be tested thoroughly, but it's a starting point and illustrates the problem. (I would say that the HTTP header parsing in inet_drv.c probably could be beefed up a little).<br>
<br>The patch to inet_drv.c is as follows (tabs may not be correct because I reformatted the code):<br><br><span style="font-family: courier new,monospace;">8318a8319,8323</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;">> </span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">> /* Test needed in case buffer is in form "\r\n\srequestdata" where \s is SP or TAB */</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;">> if (((plen == 1) && NL(ptr)) || ((plen == 2) && CRNL(ptr)))</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">> goto done;</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;">> </span><br><br clear="all">Claes's test code is here (slightly modified by me to make it easier to test the normal case and the "bug" case). You can run it as post:p(Host,Port,data) or post:p(Host,Port,bug).<br>
<br>Regards,<br>Edwin Fine<br>======================================<br>-module(post).<br>-compile(export_all).<br><br>p() -><br> p({127,0,0,1},8000,data).<br><br>p(Host, Port, What) when What =:= bug; What =:= data -><br>
{ok, S} = gen_tcp:connect(Host, Port, [{active, true}]),<br> gen_tcp:send(S, select_data(What)),<br> recloop().<br><br>recloop() -><br> receive<br> X -><br> io:format("GOT ~p~n", [X]),<br>
recloop()<br> after 4000 -><br> timeout<br> end.<br><br>select_data(What) -><br> case What of<br> bug -><br> bug();<br> data -><br> data()<br> end.<br>
<br>data() -><br> Msg = "dfoo",<br> H = "POST /invalid/url HTTP/1.1\r\n"<br> "Connection: close\r\n"<br> "Host: localhost:8000\r\n"<br> "User-Agent: perl post\r\n"<br>
"Content-Length: 4\r\n"<br> "Content-Type: text/xml; charset=utf-8\r\n"<br> "\r\n",<br> H ++ Msg.<br><br><br>%% space bug<br>bug() -><br> Msg = " foo",<br>
H = "POST /invalid/url HTTP/1.1\r\n"<br> "Connection: close\r\n"<br> "Host: localhost:8000\r\n"<br> "User-Agent: perl post\r\n"<br> "Content-Length: 4\r\n"<br>
"Content-Type: text/xml; charset=utf-8\r\n"<br> "\r\n",<br> H ++ Msg.<br><br><br><br>-- <br>The great enemy of the truth is very often not the lie -- deliberate, contrived and dishonest, but the myth, persistent, persuasive, and unrealistic. Belief in myths allows the comfort of opinion without the discomfort of thought.<br>
John F. Kennedy 35th president of US 1961-1963 (1917 - 1963)