[erlang-questions] Heavy duty UDP server performance

[Sergej Jurečko]

UDP performance in erlang is not that good. If I were you I would write a
NIF. UDP is relatively simple to work with in C/C++.

Sergej

On Mon, Feb 1, 2016 at 8:40 PM, Ameretat Reith <ameretat.reith@REDACTED>
wrote:

> I'm playing with Erlang to make an experimental protocol.  I'm trying
> to make it use full of 1Gbit link but It won't scale that much and I'm
> failing to found a bottleneck in my code or even anything I could call
> it bottleneck.
>
> My software is very like a messaging server software in behavior, with
> bigger packets, many clients (more than 4k) and uses more complex
> sub-components, like a distributed database but those components are
> not blocking other portions of system;  It's just the client-server
> channel that is heavy IO and involve some encryption and decryption.
>
> I made a gen_server process for each UDP socket to clients.  There is a
> central process registry but It being called just for new clients and
> Its message queue is often empty.
>
> I found there was a bottleneck in `scheduler_wait` when I had few
> clients (around 400) and It consumed around 50% of total CPU usage.  I
> found an old patch by Wei Cao [1] which seemed to target same issue.
> But on a modern version of Erlang (18.0) blockage in `scheduler_wait`
> dropped well in more congested network, specifically to around 10%
> when my software reached Its apparent limit, around 600Mbit/s read and
> write to network. At this point my incoming UDP packet rate is around
> 24K/s. Maybe an experienced Erlang developer here can remember that
> problem and can tell whether Erlang is now optimized to poll for
> network packets more often or not..
>
> I also concerned async pool since there was fairly high work in Erlang
> work with pthread but found those threads just used for file IO
> operations.  I didn't found any assuring documentation about this, just
> saw the only user of this dirty IO thing is `io.c` in otp source code.
> I'm very grateful if anyone clear the usage and effect of this pool.
>
>
> I made flame graphs of function calls both inside VM (using eflame2
> [2]) which is very even and cannot find any outstanding usage [3]. And
> made another flamegraph of perf report outside of VM which cannot find
> some symbols [4].  I doubt whether process_main shoud take that much
> work itself or not.  Apparently encryption and decryption (enacl_nif
> calls) didn't take much time too.
>
> Do you have any suggestion for me to analyze better my software and
> understand VM working?  Is It those limits I should expect and there is
> not more room for optimizations?
>
> Thanks in advance
>
> 1: http://erlang.org/pipermail/erlang-questions/2012-July/067868.html
> 2: https://github.com/slfritchie/eflame
> 3: http://file.reith.ir/in-erl-3k.gif
> 4: http://file.reith.ir/out-erl-perf.svg (interactive, use web browser)
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20160203/88a0d2ab/attachment.htm>