[erlang-questions] clipping of max# of file descriptors by ertswhen kernel poll is enabled

[Joel Reymont]

Rickard,

On Aug 25, 2008, at 2:14 PM, Rickard Green S wrote:

> Matthew's answer is correct. If we need to fall back on select() on  
> a filedescriptor that is larger than select() can handle, your  
> modification wont work.


The decision to use select() is made by ERTS during initialization.

If kernel poll cannot be used then the version of erl_poll.c compiled  
without the ERTS_POLL_USE_KERNEL_POLL flag will be used. This version  
will clip the file descriptors as per the code below.

#if ERTS_POLL_USE_SELECT && defined(FD_SETSIZE) && !
ERTS_POLL_USE_KERNEL_POLL
    if (max_fds > FD_SETSIZE)
        max_fds = FD_SETSIZE;
#endif

The kernel poll version of the same erl_poll.c will determine the  
maximum number of file descriptors according to the OS kernel  
configuration, a few lines above the clipping code.  This version does  
NOT need any clipping and adding && !ERTS_POLL_USE_KERNEL_POLL above  
will ensure this.

Again, there's no "fallback on select" at runtime. The decision to  
make use of select is made depending on platform capabilities at build  
time and on the +K setting.

If erl is told not to use kernel poll while the capability is there,  
then the code from erl_poll_nkp.o will be used and this code will  
always clip descriptors.

If kernel poll is available and erlang is told to use it then there  
will be NO fallback to select and NO clipping of descriptors to  
FD_SETSIZE is needed or wanted. The way to ensure this is to add && !  
ERTS_POLL_USE_KERNEL_POLL to the clipping code above.

Erlang is supposed to scale and without a fix to the above I cannot  
scale my supposedly scalable poker server above ~300 users on a single  
Erlang VM. This plain and obviously SUCKS!

With my proposed fix I can scale to thousands of users on a single VM  
without a problem.

I insist that I'm right and the OTP team is wrong. There's a bug in  
erl_poll.c and the solution is trivial. Please fix the bug!

	Thanks, Joel

--
wagerlabs.com