confusing returns from httpc:request/4

[amitm]

FWIW, I had a similar problem. It went away when I disabled pipelining
completely by executing

    ok = httpc:set_options([{max_keep_alive_length, 0},
{max_pipeline_length, 0}, {max_sessions, 0} ]),

before my http calls. I used to usually see it on the 4th consecutive
call to the same host when pipelining was enabled. The URL in question
in my case was an Amazon EC2 url.

The set of URLs in my case were

"http://169.254.169.254/2009-04-04/meta-data/instance-id",
"http://169.254.169.254/2009-04-04/meta-data/local-hostname",
"http://169.254.169.254/2009-04-04/meta-data/local-ipv4",
"http://169.254.169.254/2009-04-04/meta-data/public-hostname",
"http://169.254.169.254/2009-04-04/meta-data/public-ipv4",
"http://169.254.169.254/2009-04-04/meta-data/ami-id",

While executing this set, with pipelining enabled, it used to always
bomb on the 4th URL, i.e. for 'public-hostname' in 25% of the cases

with pipelining disabled, the problem disappears.

  Amit

On Oct 13, 2:58 pm, Ingela Andin <ingela.an...@REDACTED> wrote:
> 2010/10/13 Chandru <chandrashekhar.mullapar...@REDACTED>:
>
>
>
>
>
> > On 13 October 2010 09:53, Ingela Andin <ingela.an...@REDACTED> wrote:
> >> Hi!
>
> >> The only reason that the httpc-client will return the value {error,
> >> socket_closed_remotely}
> >> is that the connection is prematurely  closed by the server.
> >> Unfortunately  that something
> >> is sent on a tcp socket does not mean it has been sent on the wire
> >> yet, and depending on
> >> the circumstances it is possible for the socket to be closed before
> >> all data has been sent
> >> so occasionally a tcp_close can come before all data is delivered.
>
> >> When you use ibrowse do you verify that you always get a correct body?
> >> (compleate body) from what I can tell from the code ibrowse will on a
> >> tcp_closed  message always return whatever happens to be in the buffer
> >> compleate or not.
>
> > ibrowse does this only in the following cases:
> >  * server returned a "Connection: Close"
> >  * Server supports HTTP/0.9 or HTTP/1.0
>
> > In all other cases, it returns an error to the caller.
>
> > regards,
> > Chandru
>
> Well I do not know if any of these things is true in Dans senario. But
> I do know that
> getting the problem with a tcp close message before all data is
> deliverd is a highly timing dependant problem
> and if it happens in the server there is nothing inets-client or
> ibrows can do about it except return an error.
>
> Regards Ingela Erlang/OTP team - Ericsson AB
>
> ________________________________________________________________
> erlang-questions (at) erlang.org mailing list.
> Seehttp://www.erlang.org/faq.html
> To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED