[erlang-patches] Pollset per scheduler and bind port to scheduler
Mon Jul 9 15:05:02 CEST 2012
Thanks a lot for these patches!
Multiple poll-sets is something which we have been thinking about doing
for a while now, but haven't as yet gotten around to do. So it is great
that you have taken the time to implement it!
What is the reason for not enabling the multiple poll-set functionality
by default? Have you seen any scenarios where it is unwanted? Or
platforms which the approach does not work?
When compiling the multiple poll-set patch on Windows we got an error
because the compiler we use there does not allow empty structs. Do you
have a windows environment available to test your patch on? For
instructions on how to set up a test environment on windows see. We
really want this patch, but have to make sure that it works properly on
all platforms as it is a core change to how erts handler fd polling.
When the Windows compile issue is fixed, we will include this patch in
our daily builds and let you know of any test failures which we find.
Regarding binding a port to a scheduler, we are unsure about building
more functionality on top of the internal `scheduler` option as it
effects the soft-realtime properties of the system. Generally we do not
want to add functionality which exposes the internal scheduler semantics
to the programmer, but rather make the scheduler more intelligent about
where they schedule tasks. We'll have to discuss internally what we want
to do with this.
: Windows 7, mingw, vs 10 compiler
cc.sh -O3 -I/c/jenkins/workspace/compile-master-opu-win32/erts/win32 -D_WIN32_WINNT=0x0500 -DWINVER=0x0500 -DERTS_MIXED_MSYS_VC -DSTATIC_ERLANG_DRIVER -DHAVE_CONFIG_H -DUSE_THREADS -DWIN32_THREADS -Iwin32/opt/plain -Ibeam -Isys/win32 -Isys/common -Iwin32 -Izlib -Ipcre -Ihipe -I../include -I../include/win32 -I../include/internal -I../include/internal/win32 -c sys/common/erl_check_io.c -o obj/win32/opt/plain/erl_check_io.o
sys/common/erl_check_io.c(99) : error C2016: C requires that a struct or union has at least one member
On 09/07/12 08:47, Wei Cao wrote:
> In our proxy project which has massive network IO, we found
> erts_check_io() become system bottleneck,
> a call to erts_check_io() would take 500 microseconds to complete, and
> invoked about 3.5k times per second
> under heavy preesure, which resulted in the average response time of
> user request longer than 2 milliseconds,
> so that limited system throughput down, while cpus were idle(in
> starvation) at the same time.
> This patch allocates one pollset for each scheduler, so that each
> scheduler could invoke erts_check_io()
> on its own pollset concurrently. To be incorporate well with port
> migration, all used fd in each port're recorded,
> removed from old scheduler's pollset and added to new scheduler's when
> port is migrating between schedulers.
> After applying this patch, together with binding process&port with
> schedulers, erts_check_io() is invoked about 230k
> times per second (dozens of times more than before), and throughtput
> increases from 45k to 105k, so it works.
> This patch SHOULD be compiled with ./configure
> Pollset per scheduler:
> git fetch git://github.com/weicao/otp.git pollset_per_scheduler
> Bind port to scheduler:
> git fetch git://github.com/weicao/otp.git bind_port_to_scheduler
> Wei Cao
> erlang-patches mailing list
More information about the erlang-patches