[erlang-patches] Pollset per scheduler and bind port to scheduler

[Wei Cao]

In our proxy project which has massive network IO, we found
erts_check_io()  become system bottleneck,
a call to erts_check_io() would take 500 microseconds to complete, and
invoked about 3.5k times per second
under heavy preesure, which resulted in the average response time of
user request longer than 2 milliseconds,
so that limited system throughput down, while cpus were idle(in
starvation) at the same time.

This patch allocates one pollset for each scheduler, so that each
scheduler could invoke erts_check_io()
on its own pollset concurrently. To be incorporate well with port
migration, all used fd in each port're recorded,
removed from old scheduler's pollset and added to new scheduler's when
port is migrating between schedulers.

After applying this patch, together with binding process&port with
schedulers, erts_check_io() is invoked about 230k
times per second (dozens of times more than before), and throughtput
increases from 45k to 105k, so it works.

This patch SHOULD be compiled with ./configure
CFLAGS=-DERTS_POLLSET_PER_SCHEDULER

Pollset per scheduler:

git fetch git://github.com/weicao/otp.git pollset_per_scheduler

https://github.com/weicao/otp/compare/weicao:master...weicao:pollset_per_scheduler
https://github.com/weicao/otp/compare/weicao:master...weicao:pollset_per_scheduler.patch


Bind port to scheduler:

git fetch git://github.com/weicao/otp.git bind_port_to_scheduler

https://github.com/weicao/otp/compare/weicao:pollset_per_scheduler...weicao:bind_port_to_scheduler
https://github.com/weicao/otp/compare/weicao:pollset_per_scheduler...weicao:bind_port_to_scheduler.patch



--

Best,

Wei Cao