[erlang-questions] HTTP requests and the meaning of eaddrinuse

[Johnny Billquist]

Per Hedeland wrote:
> Sverker Eriksson <sverker@REDACTED> wrote:
>> Johnny Billquist wrote:
>>> Sverker Eriksson wrote:
>>>> Johnny Billquist wrote:
>> It's like a quarantine to avoid late arriving packets of the old 
>> connection from being confused with a new connection using the same 
>> port. A major flaw of TCP if you ask me.
> 
> Do you have a better solution to the problem? It is a pretty important
> thing to solve, since such packets could potentially cause a reset of
> the new session or even corrupt the data stream without any error
> indication. It could possibly be argued that the default time of
> (typically) 2 minutes is pretty huge.
> 
>>> But yes, that could be it. I wonder if eaddrinuse really is returned 
>>> in that case. [...]
>> I experienced this some time ago writing a benchmark on Linux. I'm quite 
>> sure it was eaddrinuse that I got when the port numbers where exhausted.
> 
> Yes, that's what you get (on connect()) on all Unices where I have seen
> it, and it does make some amount of sense I think - if there are no
> local ports left, any attempt to form a connection would have to use a
> port that is already "in use".

Well, I think that it's a flaw in the design of the interface. If you 
use port 0 (ask the system to give you a port) you should never get 
eaddrinuse. That is sort of guaranteed by the call (the semantic is to 
give you a free port). But I can't see any good return value to give in 
case no free port was found, so I guess they might just have been 
"lazy". I wonder if all systems do this, or if different systems have 
different "solutions". If noone can tell for other systems than Linux I 
guess I'll have to go in and examine the source myself soon. :-)

> The issue is actually a bit more complex than "running out of local
> ports" though - it is only the complete 4-tuple that has to be unique,
> i.e. multiple connections from the same local address and port is OK if
> either the remote address or the remote port differs.

True.

> Checking for this uniqueness may be expensive, which could explain that
> you get this problem "unnecessarily" - the stack may optimistically use
> a local port that isn't actually free, on the assumption that you don't
> normally make all your connections to the same remote address/port, so
> it "should" work out. Except it doesn't when you are doing benchmarks
> and other things that do tons of connections to the same address/port.

Well, the problem is that when you create the local port, you still 
don't know which remote destination you will talk with, so you can't 
really tell if an existing port will be okay to reuse.

You have the sockopt SO_REUSEADDR which circumvents this check. Is it 
possible to set that from Erlang?

> There are ways to tweak things in those cases though - the port range
> has already been mentioned and is normally configurable
> (/proc/sys/net/ipv4/ip_local_port_range on Linux) so you can actually
> use the ~ 64k theoretical max, on some OSes you may be able to tinker
> with the TIME_WAIT timeout (you know that packets won't hang around for
> 2 minutes on your LAN where you are doing the benchmark), and you may be
> able to use multiple local addresses in a round-robin fashion (same port
> on different local addresses will never cause a conflict).

Good ideas. On BSD-like systems, you normally control this with sysctl.

> How any of this works on Windows I have no idea though.

Me neither. :-)

	Johnny

-- 
Johnny Billquist                  || "I'm on a bus
                                   ||  on a psychedelic trip
email: bqt@REDACTED             ||  Reading murder books
pdp is alive!                     ||  tryin' to stay hip" - B. Idol