[erlang-patches] Pollset per scheduler and bind port to scheduler

[Wei Cao]

2012/7/17 Lukas Larsson <lukas@REDACTED>:
> Hi!
>
> After fixing the patch to compile on windows[1] I put it in our daily builds

Glad to hear it can compile on windows now, thanks a lot~ I tried to set up
mingw etc on a windows virtual machine yesterday, but it runned so slow, and
met a lot of mistakes during configuration.

> and a couple of issues came up. I did not include 'move erts_deliver_time
> out of erl_check_io' commit as it deadlocks the non-smp emulator.

Yeah, I can confirm this problem, there is only one OS thread
innon-smp emulator,
if check_io starts waiting, other threads will have no opportunity to
run, including the
thread I added to call erts_deliver_time() periodically, so check_io
will have no timer
to wake it up, and would wait endlessly.

I'll consider to replace a better solution.

>
> When running a debug test build on Linux we got the following assertion:
> Assertion failed: 0 <= (ix) && (ix) < erts_no_pollsets in
> sys/common/erl_check_io.c, line 1909

hm... this is a very silly mistake, the variable erts_no_pollsets is
set after the assertion,
after fixing it(post to pollset_per_scheduler branch), I met another
assertion error,

erlc -W  +debug_info +inline -o../ebin hipe_icode2rtl.erl
Assertion failed: VALID_INSTR(* (Eterm *)(c_p->i)) in beam/beam_emu.c, line 1241

>
> On OS X Lion we got the following when compiling gs:
> erl -pa ../ebin -s gs_make -s erlang halt -noshell
> ../include/internal/ethr_mutex.h:655: Fatal error in ethr_mutex_lock():
> Invalid argument (22)
> make[3]: *** [gstk_generic.hrl] Abort trap: 6
> make[2]: *** [opt] Error 2
> make[1]: *** [opt] Error 2
> make: *** [libs] Error 2

I didn't have a mac machine to test it.

>
> I'll remove the branch and see if the same problems appear again tomorrow
> (unfortunately we do not have enough machines to let your branch run alone,
> so it might be some other branch causing this). Let me know if you need any
> help tracking down these issues.
>
> Lukas
>
> [1]: https://github.com/garazdawi/otp/tree/wc/pollset_per_scheduler
>
>
> On 11/07/12 17:34, Wei Cao wrote:
>
> In non keep-alive cases, all new connections 're accepted by the
> Erlang port which listens on the TCP port, and how frequently/fast the
> port be scheduled to run limits the QPS. (requests per second), so
> this port can be regarded as bottleneck of non keep-alive
> applications.
>
> So I guess performance degradation observed is caused by the listener
> port not be scheduled frequent or fast enough, I'll look into this
> problem tomorrow, now is at night in China, :-)
>
> BTW, I found this patch should be compiled like this today,
> ./configure CFLAGS="-DERTS_POLLSET_PER_SCHEDULER -g -O3
> -fomit-frame-pointer"
> otherwise compiler optimization is disabled.
>
> Regarding binding processes/ports to scheduler, I admit it's really a
> temporary solution to bind port to the same scheduler as its owner
> process like the pb patch did, and I suggest it's better to add a
> additional BIF like erlang:process_flag, to allow user explicitly bind
> port to a given scheduler, if it benefits.
>
>
> 2012/7/11 Lukas Larsson <lukas@REDACTED>:
>
> Hi,
>
> The reason I'm skeptical about anything which binds processes/ports to
> scheduler is that it feels like a temporary solution and would much rather
> do a proper solution where the scheduler takes care of these things for you.
> But as I said, internally we need to talk this over when it is not in the
> middle of summer vacation.
>
> I did some benchmarking using ab and found basically the same figures as
> you. The below is with keep-alive and the values are requests per second:
>
>                    not-bound        bound
>
> R15B01                 44k        37k
>
> master                  44k        35k
>
> master+mp          48k        49k
>
> master+mp+pb    49k        55k
>
> [mp]: multi-poll patch
> [pb]: port bind patch
> [bound]: Used {scheduler,I} to spread load
>
> Unfortunately I also found that when doing the non-keep alive benchmark the
> performance is seriously degraded.
>
> R15B01 not-bound                  8255
> master+mp+pb not-bound    7668
> master+mp+pb bound           5765
>
> I did some gprof runs but could not find anything obvious that is going
> wrong.
>
> Lukas
>
>
> On 11/07/12 04:21, Wei Cao wrote:
>
> I added a macro to conditional compile the patch because I think it
> can be more selectable, I can remove the macro, fix the compilation
> error and test on mingw platform in later version.
>
> how about provide another BIF named port_flag (like process_flag) to
> let user bind port to a given scheduler?
>
>
>
>
>
>



-- 

Best,

Wei Cao