[erlang-questions] clipping of max# of file descriptors by ertswhen kernel poll is enabled

[Joel Reymont]

Rickard,

On Aug 25, 2008, at 6:51 PM, Rickard Green S wrote:

> Well, you are wrong. Current implementation will fallback on  
> select() at runtime if kqueue cannot handle the filedescriptor.
>
> The current behavior is a necessity not a bug. Introducing your  
> suggested change will introduce a serious bug.


I took a closer look at erts_poll.c. The current code does indeed use  
select as a fallback, but it should not clip descriptors if fallback  
is not used.

Fallback to select is currently implemented thusly

#if ERTS_POLL_USE_FALLBACK
     /* We depend on the wakeup pipe being handled by kernel poll */
     if (ps->fds_status[wake_fds[0]].flags & ERTS_POLL_FD_FLG_INFLBCK)
         fatal_error("%s:%d:create_wakeup_pipe(): Internal error\n",
                     __FILE__, __LINE__);
#endif

No clipping of descriptors should be done then unless  
ERTS_POLL_USE_FALLBACK is defined and the ERTS_POLL_FD_FLG_INFLBCK is  
set in ps->fds_status[...].flags, at least when  
ERTS_POLL_USE_KERNEL_POLL is defined.

Would you agree?

You can only have 1024 descriptors with R12B3 on Mac OSX right now  
since FD_SETSIZE is 1024. ERTS clips descriptors to FD_SETSIZE  
regardless of whether it needs to fallback or not, even when kernel  
poll is present and enabled.

This is an unwarranted and crippling limitation since I can go to 12k  
file descriptors on my Mac if I remove the file descriptor clipping.

1k and 12k descriptors make a huge difference for network servers and  
scalability!

	Thanks, Joel

--
wagerlabs.com