R10B3 crashes after >1014 tcp connections + Yaws

[Serge Aleynikov]

Joel,

Since there seem to be a handful of obstacles in increasing the number 
of FDs on Mac OS, you could probably consider switching the transport 
from TCP to UDP.  If you implement basic error correction in UDP 
(retransmission timers, dropping duplicate packets, transaction id 
awareness), it could be a quite scalable solution as long as open UDP 
sockets (except for the listener) are not retained on the server for a 
long time.

I've been using this in one of the projects where I had exactly the same 
issue with TCP's TD limit, and tested the solution with over 50k 
concurrent processes handling client requests (though the size of the 
server's UDP receive buffer needs to be carefully selected).  Though I 
have seen that the response time per client increases quite a bit when 
you go over 5000 requests per second.  Yet on a good side - I haven't 
seen a degradation of server's speed measured in requests per second 
with increase of number of clients).

Serge

joel reymont wrote:
> I took a quick look at the source code and there are quite a few
> places where FD_SETSIZE is hardcoded. So the number of open files
> fixed at 1024 on my Mac OSX 10.3.9. I could certainly edit the system
> header files and increase the limit to  a more suitable number but...
> I cannot ask my customer to do this and rebuild Erlang. At least I
> can't always ask them to.
> 
> Overall, 1024 file descriptors per Erlang node is WAY too low.
> 
>     Thanks, Joel
> 
> On 6/11/05, Mickael Remond <mickael.remond@REDACTED> wrote:
> 
>>joel reymont wrote:
>>
>>>I will do this over the weekend. I compiled myself the R10B5 version
>>>just for the purpose.
>>
>>Maybe the environment variable ERL_MAX_PORT might also help in your
>>case. This use to be valid in the past but I do not know if this is
>>still the case:
>>
>>export ERL_MAX_PORTS=32000