[erlang-questions] inets HTTP client does not persist connections when POSTing

[Edwin Fine]

Thanks for the tip.

Interestingly, the ibrowse library *does* permit the user to specify this
behavior, which works very well (so far!) for the use case in which I'm
employing it. The HTTP specification seems to say nothing forbidding
consecutive post/reply pairs on a KA connection. If the server does close
the KA connection after each post/reply, then I guess the behavior will
effectively fall back to the usual mechanism of
connect/post/reply/disconnect. Maybe it's the time of night, but I can't see
how accepting a POST followed by sending a reply to the POST would put a
server in a weird state that requires dropping the socket and a reconnect to
clear, unless the underlying server code is really poorly written. I dunno.
Maybe I'll find out the hard way...

On Fri, Aug 15, 2008 at 1:01 AM, Leif Hedstrom <
leif+lists.lang.erlang@REDACTED <leif%2Blists.lang.erlang@REDACTED>> wrote:

> Edwin Fine wrote:
>
>> I read the inets code, and if I understand it correctly, it seems that
>> when a non-idempotent operation such as POST is performed, httpc does not
>> reuse the existing connection (httpc_manager:select_session/4 returns
>> no_connection, which starts a new handler). I am not sure if this is the
>> correct thing to do, because my understanding is that the restriction on
>> non-idempotent operations applies to pipelining, not persistent connections.
>> I can see no reason why one should not be able to do a series of consecutive
>> POSTs on the same socket, as long as one waits for a reply before posting
>> the next request.
>>
>
> I don't think this is extremely unreasonable, it's basically a defense
> mechanism to avoid issues with the non-idempotent request putting the
> server/connection in an undesirable state. The reasoning generally seems to
> be that a "fresh" connection to the server is less likely to be terminated
> prematurely (or arbitrarily). Also remember a server always has the right to
> close a KA connection (particularly after a non-idempotent request).
>
> One way to improve on this could be to use the "100-continue" header
> (Expect: 100-continue) before sending a POST body on a Keep-Alive
> connection. This obviously introduces more latency, which can negate all
> benefits of Keep-Alive (or even make it worse). On high latency networks,
> with many of POSTs on a KA connection, it's probably beneficial even with
> the overhead of "100-continue" (due to 3-way handshake + TCP congestion
> control on new connections).
>
> Now, I do agree that it seems a little draconian that the library doesn't
> let the user (developer) make these decisions. I haven't looked at the code
> at all, but assuming this is what it does,  I'm guessing it was done to make
> sure people don't fall into this pit of nastiness accidentally. It's a
> reasonable default for sure, but ought to be possible to override,  by
> someone well familiar with the dangers (and how to handle them).
>
> Cheers,
>
> -- Leif
>
>
>


-- 
For every expert there is an equal and opposite expert - Arthur C. Clarke
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20080815/c5efd49f/attachment.htm>