[erlang-questions] inets HTTP client does not persist connections when POSTing

[Edwin Fine]

I am using the inets HTTP client to POST many requests to the same URL. I am
finding that even though keep-alive is set, and persistent connections are
the default for HTTP 1.1 (which is what I am using), the inets HTTP client
is opening a new connection for every http:request(post,...) I send. I
verified this by using Wireshark to monitor the TCP/IP traffic. When there
is a lot of activity involving multiple clients, this causes ephemeral port
exhaustion because of the infamous TIME_WAIT problem. I have already reduced
the TCP wait time to 30 seconds and don't want to reduce it more.

I read the inets code, and if I understand it correctly, it seems that when
a non-idempotent operation such as POST is performed, httpc does not reuse
the existing connection (httpc_manager:select_session/4 returns
no_connection, which starts a new handler). I am not sure if this is the
correct thing to do, because my understanding is that the restriction on
non-idempotent operations applies to pipelining, not persistent connections.
I can see no reason why one should not be able to do a series of consecutive
POSTs on the same socket, as long as one waits for a reply before posting
the next request.

I'm not sure if this is a bug, a feature, or my misunderstanding, so I am
posting to this mailing list in the hope that someone, whose name may or may
not be Ingela, can enlighten me and perhaps suggest how I can use inets to
do multiple sequential POSTs on a persistent connection.

I'm using OTP R12B-3 on Ubuntu Linux 8.04. The http:request call uses all
default HTTP and TCP/IP options (i.e. empty lists).

Regards,
Edwin Fine
-- 
For every expert there is an equal and opposite expert - Arthur C. Clarke
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20080801/ee2a713e/attachment.htm>