[erlang-questions] Concurrent requests with ibrowse

[Per Hedeland]

Edwin Fine <erlang-questions_efine@REDACTED> wrote:
>
>If that doesn't help, try decreasing TIME_WAIT (but first read
>http://www.erlang.org/pipermail/erlang-questions/2008-September/038154.htmland
>http://www.developerweb.net/forum/showthread.php?t=2941).
>
># Set TIME_WAIT timeout to 30 seconds instead of 120
>sudo /sbin/sysctl -w net.ipv4.tcp_fin_timeout=30

That may help, but change the TIME_WAIT time (which isn't really a
"timeout", it's not waiting for anything to "happen") it does not, as
one might guess from the name of the variable. It reduces the timeout
waiting for a close from the peer in the FIN_WAIT_2 state (see the state
diagram in RFC 793), by default 60 seconds on Linux I believe. Note that
this is generally short duration, and you shouldn't hit the timeout
unless connectivity with the peer is lost - but reducing it too
agressively might cause loss of data.

As far as I know there is no way to reduce the TIME_WAIT time on Linux
other than modifying the kernel - it's a #define (60*HZ) in a kernel
header file. There are other ways to deal with the problem of having a
lot of connections in TIME_WAIT on Linux though.

>2009/2/13 steve ellis <steve.e.123@REDACTED>
>
>> We're trying to build an app that uses ibrowse to make concurrent requests.
>> We are not able to get more than a few concurrent requests at a time to
>> return successfully. We repeatedly get "conn_failed"

If that's all you get, I'm afraid it's pretty useless. Did it time out,
get "connection refused" i.e. RST, was the connection established but
immediately closed, or did it run into the "lack of ports" problem? I
wouldn't say that unhandled "let it crash" is appropriate for problems
occurring way below the user interface of an application, but a badmatch
would at least have told us what gen_tcp said.

Anyway if you get problems with connections in the low hundreds, "lack
of ports" is really unlikely. So dig deeper instead of blindly trying to
fix a problem that you may not have - find the place(s) in the source
where 'conn_failed' is generated and make it/them report what actually
happened, and/or use tcpdump or similar to figure out what goes wrong
with the connections.

--Per