[erlang-patches] Pollset per scheduler and bind port to scheduler

[Lukas Larsson]

fyi the patch has introduced three new gcc warnings:

erl_port_task.c:230, gcc4,

control reaches end of non-void function [-Wreturn-type]

erl_check_io.c:992, gcc4,

‘inps’ may be used uninitialized in this function [-Wuninitialized]

erl_check_io.c:998, gcc4,

‘outps’ may be used uninitialized in this function [-Wuninitialized]

Lukas

On 18/07/12 11:28, Wei Cao wrote:
> 2012/7/17 Lukas Larsson <lukas@REDACTED>:
>> Hi!
>>
>> After fixing the patch to compile on windows[1] I put it in our daily builds
>> and a couple of issues came up. I did not include 'move erts_deliver_time
>> out of erl_check_io' commit as it deadlocks the non-smp emulator.
> This problem is also fixed by testing macro ERTS_SMP to tell whether
> it's a non-smp emulator, now both smp and non-smp emulators work.
>
>> When running a debug test build on Linux we got the following assertion:
>> Assertion failed: 0 <= (ix) && (ix) < erts_no_pollsets in
>> sys/common/erl_check_io.c, line 1909
>>
>> On OS X Lion we got the following when compiling gs:
>> erl -pa ../ebin -s gs_make -s erlang halt -noshell
>> ../include/internal/ethr_mutex.h:655: Fatal error in ethr_mutex_lock():
>> Invalid argument (22)
>> make[3]: *** [gstk_generic.hrl] Abort trap: 6
>> make[2]: *** [opt] Error 2
>> make[1]: *** [opt] Error 2
>> make: *** [libs] Error 2
>>
>> I'll remove the branch and see if the same problems appear again tomorrow
>> (unfortunately we do not have enough machines to let your branch run alone,
>> so it might be some other branch causing this). Let me know if you need any
>> help tracking down these issues.
>>
>> Lukas
>>
>> [1]: https://github.com/garazdawi/otp/tree/wc/pollset_per_scheduler
>>
>>
>> On 11/07/12 17:34, Wei Cao wrote:
>>
>> In non keep-alive cases, all new connections 're accepted by the
>> Erlang port which listens on the TCP port, and how frequently/fast the
>> port be scheduled to run limits the QPS. (requests per second), so
>> this port can be regarded as bottleneck of non keep-alive
>> applications.
>>
>> So I guess performance degradation observed is caused by the listener
>> port not be scheduled frequent or fast enough, I'll look into this
>> problem tomorrow, now is at night in China, :-)
>>
>> BTW, I found this patch should be compiled like this today,
>> ./configure CFLAGS="-DERTS_POLLSET_PER_SCHEDULER -g -O3
>> -fomit-frame-pointer"
>> otherwise compiler optimization is disabled.
>>
>> Regarding binding processes/ports to scheduler, I admit it's really a
>> temporary solution to bind port to the same scheduler as its owner
>> process like the pb patch did, and I suggest it's better to add a
>> additional BIF like erlang:process_flag, to allow user explicitly bind
>> port to a given scheduler, if it benefits.
>>
>>
>> 2012/7/11 Lukas Larsson <lukas@REDACTED>:
>>
>> Hi,
>>
>> The reason I'm skeptical about anything which binds processes/ports to
>> scheduler is that it feels like a temporary solution and would much rather
>> do a proper solution where the scheduler takes care of these things for you.
>> But as I said, internally we need to talk this over when it is not in the
>> middle of summer vacation.
>>
>> I did some benchmarking using ab and found basically the same figures as
>> you. The below is with keep-alive and the values are requests per second:
>>
>>                     not-bound        bound
>>
>> R15B01                 44k        37k
>>
>> master                  44k        35k
>>
>> master+mp          48k        49k
>>
>> master+mp+pb    49k        55k
>>
>> [mp]: multi-poll patch
>> [pb]: port bind patch
>> [bound]: Used {scheduler,I} to spread load
>>
>> Unfortunately I also found that when doing the non-keep alive benchmark the
>> performance is seriously degraded.
>>
>> R15B01 not-bound                  8255
>> master+mp+pb not-bound    7668
>> master+mp+pb bound           5765
>>
>> I did some gprof runs but could not find anything obvious that is going
>> wrong.
>>
>> Lukas
>>
>>
>> On 11/07/12 04:21, Wei Cao wrote:
>>
>> I added a macro to conditional compile the patch because I think it
>> can be more selectable, I can remove the macro, fix the compilation
>> error and test on mingw platform in later version.
>>
>> how about provide another BIF named port_flag (like process_flag) to
>> let user bind port to a given scheduler?
>>
>>
>>
>>
>>
>>
>
>