[erlang-questions] HTTP requests and the meaning of eaddrinuse

[Philip Fennell]

Thank you everyone for your input.

I must admit that a lot of that was way outside of my area but from what
I gather I'm running-out of available ports, and netstat tells me, by
its last line of output:

> TCP    PC-S051049:5000        pal.sandbox.dev.bbc.co.uk:5984
TIME_WAIT

That I have 5000 ports in the TIME_WAIT state.

I guess that the best way to work-round this issue is to use the batch
document creation mode for CouchDB and chunk the data set into thousand
or many thousand sets. That way I won't consume the ports at such a
fearsome rate.


Regards

Philip Fennell


-----Original Message-----
From: erlang-questions-bounces@REDACTED
[mailto:erlang-questions-bounces@REDACTED] On Behalf Of Per Hedeland
Sent: 01 February 2009 20:28
To: bqt@REDACTED
Cc: erlang-questions@REDACTED
Subject: Re: [erlang-questions] HTTP requests and the meaning of
eaddrinuse

Johnny Billquist <bqt@REDACTED> wrote:
>
>Per Hedeland wrote:
>> 
>> Yes, that's what you get (on connect()) on all Unices where I have 
>> seen it, and it does make some amount of sense I think - if there are

>> no local ports left, any attempt to form a connection would have to 
>> use a port that is already "in use".
>
>Well, I think that it's a flaw in the design of the interface. If you 
>use port 0 (ask the system to give you a port) you should never get 
>eaddrinuse. That is sort of guaranteed by the call (the semantic is to 
>give you a free port).

Well, we're talking about connect() here - you're not asking for a free
port, you're asking for a connection - just that the stack has to give
you a port to establish the connection (unless you have bound the
socket, which is "unusual" on connect()).

> I wonder if all systems do this, or if different systems have 
>different "solutions". If noone can tell for other systems than Linux I

>guess I'll have to go in and examine the source myself soon. :-)

Looking at the FreeBSD kernel code, it seems it will actually return
EADDRNOTAVAIL or possibly EAGAIN if it fails to find a "free" port - but
will return EADDRINUSE if the chosen port ends up conflicting with a
connection in TIME_WAIT. The latter is probably the most common failure
case.

>You have the sockopt SO_REUSEADDR which circumvents this check. Is it 
>possible to set that from Erlang?

Certainly (read the man page), but I'm not sure how or if it applies to
connect() (and can't be bothered to figure out that part of the kernel
code:-). It obviously wouldn't make sense to not try to find a "free"
port at all if it is set (but it may affect the exact meaning of
"free"), and it certainly won't allow a connection that conflicts with
an existing one.

>> Forgot one that I actually used recently on FreeBSD - it randomizes 
>> the selection of local port to make spoofing harder, and every now 
>> and then a test case that did a bunch of localhost connections would 
>> get the EADDRINUSE. Turning off the randomization (sysctl 
>> net.inet.ip.portrange.randomized 1 -> 0) made it cycle nicely through

>> the available range => problem solved. A quick check on a Linux 
>> 2.6.20 system doesn't show any such randomization nor reveal a way to

>> turn it on or off, but it may well be there somewhere...
>
>Whoa? It don't actually check that the random port is actually free 
>before giving it to you???

The check is the same whether randomizing or not, but the probability of
hitting one of your own old connections in TIME_WAIT is higher when
randomizing is used.

But anyway, I think we're getting rather off-topic here, and the
original answer of "you're running out of ports" to the original
question is probably as good as it gets - I'll shut up now.

--Per
_______________________________________________
erlang-questions mailing list
erlang-questions@REDACTED
http://www.erlang.org/mailman/listinfo/erlang-questions

http://www.bbc.co.uk/
This e-mail (and any attachments) is confidential and may contain personal views which are not the views of the BBC unless specifically stated.
If you have received it in error, please delete it from your system.
Do not use, copy or disclose the information in any way nor act in reliance on it and notify the sender immediately.
Please note that the BBC monitors e-mails sent or received.
Further communication will signify your consent to this.