[erlang-questions] How to handle a massive amount of UDP packets?

[John-Paul Bader]

Dear list,


I'm currently writing a bittorrent tracker in Erlang. While a naive 
implementation of the protocol is quite easy, there are some performance 
related challanges where I could use some help.

In the first test run as a replacement for a very popular tracker, my 
erlang tracker got about 40k requests per second.

My initial approach was to initialize the socket in one process with 
{active, once}, handle the message in handle_info with minimal effort 
and pass the data asynchronously to a freshly spawned worker processes 
which responds to the clients. After spawning the process I'm setting 
the socket back to {active, once}.

Now when I switched the erlang tracker live the erlang vm was topping at 
100% CPU load. My guess is that the process handling the udp packets 
from the socket could not keep up. Since I'm still quite new to the 
world of erlang I'd like to know if there are some best practices / 
patterns to handle this massive amount of packets.

For example using the socket in {active, once} might be too slow? Also 
the response to the clients needs to come from the same port as the 
request was coming in. Is it a problem to use the same socket for that? 
Should I pre-spawn a couple of thousand workers and dispatch the data 
from the socket to them rather than spawning them on each packet?

It would be really great if you could give some advice or point me into 
the right directions.

~ John