[erlang-questions] clipping of max# of file descriptors by ertswhen kernel poll is enabled

[Rickard Green S]

Well, you are wrong. Current implementation will fallback on select() at runtime if kqueue cannot handle the filedescriptor.
 
The current behavior is a necessity not a bug. Introducing your suggested change will introduce a serious bug.
 
BR,
Rickard Green, Erlang/OTP, Ericsson AB.

________________________________

Från: Joel Reymont [mailto:joelr1@REDACTED]
Skickat: må 2008-08-25 15:50
Till: Rickard Green S
Kopia: Matthew Dempsky; Erlang Questions
Ämne: Re: SV: [erlang-questions] clipping of max# of file descriptors by ertswhen kernel poll is enabled



Rickard,

On Aug 25, 2008, at 2:14 PM, Rickard Green S wrote:

> Matthew's answer is correct. If we need to fall back on select() on 
> a filedescriptor that is larger than select() can handle, your 
> modification wont work.


The decision to use select() is made by ERTS during initialization.

If kernel poll cannot be used then the version of erl_poll.c compiled 
without the ERTS_POLL_USE_KERNEL_POLL flag will be used. This version 
will clip the file descriptors as per the code below.

#if ERTS_POLL_USE_SELECT && defined(FD_SETSIZE) && !
ERTS_POLL_USE_KERNEL_POLL
    if (max_fds > FD_SETSIZE)
        max_fds = FD_SETSIZE;
#endif

The kernel poll version of the same erl_poll.c will determine the 
maximum number of file descriptors according to the OS kernel 
configuration, a few lines above the clipping code.  This version does 
NOT need any clipping and adding && !ERTS_POLL_USE_KERNEL_POLL above 
will ensure this.

Again, there's no "fallback on select" at runtime. The decision to 
make use of select is made depending on platform capabilities at build 
time and on the +K setting.

If erl is told not to use kernel poll while the capability is there, 
then the code from erl_poll_nkp.o will be used and this code will 
always clip descriptors.

If kernel poll is available and erlang is told to use it then there 
will be NO fallback to select and NO clipping of descriptors to 
FD_SETSIZE is needed or wanted. The way to ensure this is to add && ! 
ERTS_POLL_USE_KERNEL_POLL to the clipping code above.

Erlang is supposed to scale and without a fix to the above I cannot 
scale my supposedly scalable poker server above ~300 users on a single 
Erlang VM. This plain and obviously SUCKS!

With my proposed fix I can scale to thousands of users on a single VM 
without a problem.

I insist that I'm right and the OTP team is wrong. There's a bug in 
erl_poll.c and the solution is trivial. Please fix the bug!

        Thanks, Joel

--
wagerlabs.com