[erlang-patches] Pollset per scheduler and bind port to scheduler

[Lukas Larsson]

Hi,

Thanks a lot for these patches!

Multiple poll-sets is something which we have been thinking about doing 
for a while now, but haven't as yet gotten around to do. So it is great 
that you have taken the time to implement it!

What is the reason for not enabling the multiple poll-set functionality 
by default? Have you seen any scenarios where it is unwanted? Or 
platforms which the approach does not work?

When compiling the multiple poll-set patch on Windows we got an error[1] 
because the compiler we use there does not allow empty structs. Do you 
have a windows environment available to test your patch on? For 
instructions on how to set up a test environment on windows see[2]. We 
really want this patch, but have to make sure that it works properly on 
all platforms as it is a core change to how erts handler fd polling.

When the Windows compile issue is fixed, we will include this patch in 
our daily builds and let you know of any test failures which we find.

Regarding binding a port to a scheduler, we are unsure about building 
more functionality on top of the internal `scheduler` option as it 
effects the soft-realtime properties of the system. Generally we do not 
want to add functionality which exposes the internal scheduler semantics 
to the programmer, but rather make the scheduler more intelligent about 
where they schedule tasks. We'll have to discuss internally what we want 
to do with this.

Thanks!
Lukas

[1]: Windows 7, mingw, vs 10 compiler

cc.sh   -O3 -I/c/jenkins/workspace/compile-master-opu-win32/erts/win32   -D_WIN32_WINNT=0x0500 -DWINVER=0x0500 -DERTS_MIXED_MSYS_VC -DSTATIC_ERLANG_DRIVER  -DHAVE_CONFIG_H  -DUSE_THREADS -DWIN32_THREADS  -Iwin32/opt/plain -Ibeam -Isys/win32 -Isys/common -Iwin32 -Izlib  -Ipcre -Ihipe -I../include -I../include/win32 -I../include/internal -I../include/internal/win32 -c sys/common/erl_check_io.c -o obj/win32/opt/plain/erl_check_io.o
sys/common/erl_check_io.c(99) : error C2016: C requires that a struct or union has at least one member

[2]: http://www.erlang.org/doc/installation_guide/INSTALL-WIN32.html

On 09/07/12 08:47, Wei Cao wrote:
> In our proxy project which has massive network IO, we found
> erts_check_io()  become system bottleneck,
> a call to erts_check_io() would take 500 microseconds to complete, and
> invoked about 3.5k times per second
> under heavy preesure, which resulted in the average response time of
> user request longer than 2 milliseconds,
> so that limited system throughput down, while cpus were idle(in
> starvation) at the same time.
>
> This patch allocates one pollset for each scheduler, so that each
> scheduler could invoke erts_check_io()
> on its own pollset concurrently. To be incorporate well with port
> migration, all used fd in each port're recorded,
> removed from old scheduler's pollset and added to new scheduler's when
> port is migrating between schedulers.
>
> After applying this patch, together with binding process&port with
> schedulers, erts_check_io() is invoked about 230k
> times per second (dozens of times more than before), and throughtput
> increases from 45k to 105k, so it works.
>
> This patch SHOULD be compiled with ./configure
> CFLAGS=-DERTS_POLLSET_PER_SCHEDULER
>
> Pollset per scheduler:
>
> git fetch git://github.com/weicao/otp.git pollset_per_scheduler
>
> https://github.com/weicao/otp/compare/weicao:master...weicao:pollset_per_scheduler
> https://github.com/weicao/otp/compare/weicao:master...weicao:pollset_per_scheduler.patch
>
>
> Bind port to scheduler:
>
> git fetch git://github.com/weicao/otp.git bind_port_to_scheduler
>
> https://github.com/weicao/otp/compare/weicao:pollset_per_scheduler...weicao:bind_port_to_scheduler
> https://github.com/weicao/otp/compare/weicao:pollset_per_scheduler...weicao:bind_port_to_scheduler.patch
>
>
>
> --
>
> Best,
>
> Wei Cao
> _______________________________________________
> erlang-patches mailing list
> erlang-patches@REDACTED
> http://erlang.org/mailman/listinfo/erlang-patches
>