[erlang-bugs] httpc messing with headers

[eurekafag]

Hello,

I get 404s, too, and of course it's not what we're talking about. I
tried R14B04 and R14B03 (currently I'm on it because it's in Debian
testing which I use) with no difference. Fortunately, HTTP/1.0 in
http_options "solved" this (worked around actually) and I have no
crashes using it. I tried at least two different servers and both of
them sent headers in the stream after some data related to the
previous file I guess. I mentioned before that this failure occures
only on one computer of three I have nearby which is kind of strange.
I install OTP from debian repo and now I have no 3rd-party libraries.
Maybe it's something wrong with content length because judging by the
output it looks like httpc searches for the header ahead of the real
one. I checked saved files and found that header saved in the
preceding one in the end of it. So httpc just saves too much. It's not
a problem with a non-persistent connection but if it is then httpc
just lose the stream and parses garbage (the body of the file instead
of the header) or the header but from the wrong line. This funny guy
contains the bug (in the end of the file there's a header for the next
file which hapenned to be 092.jpg in my case)
http://eureka7.ru/tmp/087.jpg Maybe this will help a little.

2011/10/28 Micael Karlberg <bmk@REDACTED>:
> Hi,
>
> Never seen this before.
> I tried your test-program, but encountered no crashes
> (we are behind a proxy, which might effect the test).
> I got some 404s, but I assume that is not what you are
> looking for.
>
> I assume you are using the latest version of OTP (R14B04)?
>
> Either the server sends us crap or we have a problem with
> our chunked decoding (that's the only case a server should
> send headers "in" a body).
>
> Try using version HTTP/1.0 instead (of 1.1). In that case
> the server MUST NOT use transfer codings.
>
> Also we did find the bug causing the unexpected crash, the
> decode problem should not cause *this* crash. Instead the
> handler process should perform a controlled stop (it was
> an un-catched throw that got all the way to the gen_server).
> Here is a diff for that problem:
>
> diff --git a/lib/inets/src/http_client/httpc_handler.erl
> b/lib/inets/src/http_client/httpc_handler.erl
> index 587e24c..bb3b5bd 100644
> --- a/lib/inets/src/http_client/httpc_handler.erl
> +++ b/lib/inets/src/http_client/httpc_handler.erl
> @@ -507,8 +507,14 @@ handle_info({Proto, _Socket, Data},
>                ClientErrMsg = httpc_response:error(Request, ClientReason),
>                NewState     = answer_request(Request, ClientErrMsg, State),
>                {stop, normal, NewState};
> -           error:_Error ->
> -               ?hcrd("data processing error", [{error, _Error}]),
> +             throw:Error ->
> +               ?hcrd("data processing error", [{throw, Error}]),
> +               ClientReason = {could_not_parse_as_http, Data},
> +               ClientErrMsg = httpc_response:error(Request, ClientReason),
> +               NewState     = answer_request(Request, ClientErrMsg, State),
> +               {stop, normal, NewState};
> +             error:Error ->
> +               ?hcrd("data processing error", [{error, Error}]),
>                ClientReason = {could_not_parse_as_http, Data},
>                ClientErrMsg = httpc_response:error(Request, ClientReason),
>                NewState     = answer_request(Request, ClientErrMsg, State),
>
>
> /Regards,
>        /BMK
>