[erlang-questions] 300k HTTP GET RPS on Erlang 21.2, benchmarking per scheduler polling

[Lukas Larsson]

Hello,

On Tue, Dec 18, 2018 at 2:36 AM Vans S <vans_163@REDACTED> wrote:

> If anyone is interested here is the writeup
> https://elixirforum.com/t/300k-requests-per-second-webserver-in-elixir-otp21-2-10-cores/18823
> .
>
> tl;dr; About 22% of time the scheduler spent in poll. to serve 30k~ HTTP
> Get requests. I think its a little much still?
>

Firstly, it is not poll that you spend 22% in, it is PORT, i.e. the work
done by gen_tcp to call writev/read. Polling shows up in the state
CHECK_IO. The optimizations introduced in 21.2 were mainly done to reduce
the time spent doing polling.

Secondly, I'd say it is too little. As you saw in the edit that you made,
if your remove/optimize the Erlang parts you will get higher a throughput
rate as the system can spend more time doing port work. What you are seeing
as OTHER is most likely the system spinning looking for work to do. You can
get more states if you are interested in digging deeper by passing
--with-microstate-accounting=extra to configure.

The inet_driver (the port driver that is used for TCP/UDP/SCTP) is not
perfect, but almost 2 decades have been spent improving it, so there are
very few low hanging fruits left.

Lukas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20181218/dd8ee2ea/attachment.htm>