getting to yaws, devpoll

[Per Bergqvist]

> Joe Armstrong <joe@REDACTED> writes:                                 
>                                                                     
> > If you are doing [yaws] benchmarks then the following question is 
> > very interesting.                                                 
>                                                                     
> ...                                                                 
>                                                                     
> > 3) Measure the throughput as you massivly overload the system.    
> >                                                                   
> >    (say the page rate is 1000 pages/sec)                          
> >                                                                   
> >    Subject the system to 10000 requests/sec.                      
>                                                                     
> I wonder, wouldn't this situation be helped by kernel polling?      
>                                                                     
Most certainly.                                                       
To clearify why kernel polling suites erlang extremely well one just  
have to look on the way the scheduler works.                          
On a system with load it will sit in a loop where it executes a number
of reductions and then goes away to check for i/o with zero timeout.  
If you run a unix that supports poll and have, say 15000, active      
TCP/IP connections you will have to copy 12*15000 bytes from user     
space to kernel space. The kernel then have to do a driver poll for   
each and every of those file descriptors.                             
Note that this will occur even if there is no actual i/o taking over  
the connections, like many http 1.1 sessions.                         
It is not uncommon to see 70-80% kernel load already at 2000-3000     
connections.                                                          
With kernel polling it is a totally different story.                  
Instead of having each connections fd in the pollset the sockets fds  
are bound in kernel state.                                            
Only a single fd is included in the pollset reducing the user-kernel  
space copying but more important it is not necessary for the kernel to
execute the driver poll for each fd since it keeps state.             
The system load now depends on actual traffic and not established     
connections which makes erlang + kernel an ideal platform for massive 
multiuser internet services.                                          
                                                                      
> Per Bergqvist was working on a patch, but - I don't see devpoll     
> stuff in a recent snapshot (otp_src_P9_2002-06-30).                 
>                                                                     
I haven't updated  the patch for P9. Will probably happen during the  
summer. There is still some weird behaviour with FreeBSD kqueue       
support but the Solaris and Linux+/dev/kpoll variants are flawless.   
/dev/kpoll is a derivate work from Davide Libenskis devpoll driver.   
It mimics the semantics of poll better than devpoll.                  
It is still unreleased to a wider audience. It should really have been
released a few months ago but nobody seemed interested in the kernel  
poll support.                                                         
Let me know if your interested and I'll fix it.                       
>                                                                     
http://www.erlang.org/ml-archive/erlang-questions/200203/msg00038.html
>                                                                     
=========================================================             
Per Bergqvist                                                         
Synapse Systems AB                                                    
Phone: +46 709 686 685                                                
Email: per@REDACTED