[erlang-questions] How to handle a massive amount of UDP packets?

[Chandru]

Hi John,

Our RADIUS server which handles our data network is written in Erlang.
We've experimented with various values of recbuf and read_packets options
for the UDP socket. We also use {active, once}. The receiving process
receives a packet and spawns a new process to handle it. That is all it
does. The spawned process then executes the rest of the business logic.

That won't be your problem. The problem will be to make sure your system is
stable while handling all those packets. We use overload control at the
receiver. You have to pretty much look at the entire execution path for
each packet and ensure there are no bottlenecks. At that kind of load,
every little bottleneck shows up sooner or later.

cheers
Chandru

On 15 April 2012 19:08, John-Paul Bader <hukl@REDACTED> wrote:

> Dear list,
>
>
> I'm currently writing a bittorrent tracker in Erlang. While a naive
> implementation of the protocol is quite easy, there are some performance
> related challanges where I could use some help.
>
> In the first test run as a replacement for a very popular tracker, my
> erlang tracker got about 40k requests per second.
>
> My initial approach was to initialize the socket in one process with
> {active, once}, handle the message in handle_info with minimal effort and
> pass the data asynchronously to a freshly spawned worker processes which
> responds to the clients. After spawning the process I'm setting the socket
> back to {active, once}.
>
> Now when I switched the erlang tracker live the erlang vm was topping at
> 100% CPU load. My guess is that the process handling the udp packets from
> the socket could not keep up. Since I'm still quite new to the world of
> erlang I'd like to know if there are some best practices / patterns to
> handle this massive amount of packets.
>
> For example using the socket in {active, once} might be too slow? Also the
> response to the clients needs to come from the same port as the request was
> coming in. Is it a problem to use the same socket for that? Should I
> pre-spawn a couple of thousand workers and dispatch the data from the
> socket to them rather than spawning them on each packet?
>
> It would be really great if you could give some advice or point me into
> the right directions.
>
> ~ John
> ______________________________**_________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/**listinfo/erlang-questions<http://erlang.org/mailman/listinfo/erlang-questions>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20120416/8955a25d/attachment.htm>