[erlang-patches] Pollset per scheduler and bind port to scheduler

[Lukas Larsson]

Hi!

After fixing the patch to compile on windows[1] I put it in our daily 
builds and a couple of issues came up. I did not include 'move 
erts_deliver_time out of erl_check_io 
<https://github.com/weicao/otp/commit/7577507ae849aa94a5cff147d8d13e7c1d327a2f>' 
commit as it deadlocks the non-smp emulator.

When running a debug test build on Linux we got the following assertion:
Assertion failed: 0 <= (ix) && (ix) < erts_no_pollsets in 
sys/common/erl_check_io.c, line 1909

On OS X Lion we got the following when compiling gs:
erl -pa ../ebin -s gs_make -s erlang halt -noshell
../include/internal/ethr_mutex.h:655: Fatal error in ethr_mutex_lock(): 
Invalid argument (22)
make[3]: *** [gstk_generic.hrl] Abort trap: 6
make[2]: *** [opt] Error 2
make[1]: *** [opt] Error 2
make: *** [libs] Error 2

I'll remove the branch and see if the same problems appear again 
tomorrow (unfortunately we do not have enough machines to let your 
branch run alone, so it might be some other branch causing this). Let me 
know if you need any help tracking down these issues.

Lukas

[1]: https://github.com/garazdawi/otp/tree/wc/pollset_per_scheduler

On 11/07/12 17:34, Wei Cao wrote:
> In non keep-alive cases, all new connections 're accepted by the
> Erlang port which listens on the TCP port, and how frequently/fast the
> port be scheduled to run limits the QPS. (requests per second), so
> this port can be regarded as bottleneck of non keep-alive
> applications.
>
> So I guess performance degradation observed is caused by the listener
> port not be scheduled frequent or fast enough, I'll look into this
> problem tomorrow, now is at night in China, :-)
>
> BTW, I found this patch should be compiled like this today,
> ./configure CFLAGS="-DERTS_POLLSET_PER_SCHEDULER -g -O3 -fomit-frame-pointer"
> otherwise compiler optimization is disabled.
>
> Regarding binding processes/ports to scheduler, I admit it's really a
> temporary solution to bind port to the same scheduler as its owner
> process like the pb patch did, and I suggest it's better to add a
> additional BIF like erlang:process_flag, to allow user explicitly bind
> port to a given scheduler, if it benefits.
>
>
> 2012/7/11 Lukas Larsson <lukas@REDACTED>:
>> Hi,
>>
>> The reason I'm skeptical about anything which binds processes/ports to
>> scheduler is that it feels like a temporary solution and would much rather
>> do a proper solution where the scheduler takes care of these things for you.
>> But as I said, internally we need to talk this over when it is not in the
>> middle of summer vacation.
>>
>> I did some benchmarking using ab and found basically the same figures as
>> you. The below is with keep-alive and the values are requests per second:
>>
>>                     not-bound        bound
>>
>> R15B01                 44k        37k
>>
>> master                  44k        35k
>>
>> master+mp          48k        49k
>>
>> master+mp+pb    49k        55k
>>
>> [mp]: multi-poll patch
>> [pb]: port bind patch
>> [bound]: Used {scheduler,I} to spread load
>>
>> Unfortunately I also found that when doing the non-keep alive benchmark the
>> performance is seriously degraded.
>>
>> R15B01 not-bound                  8255
>> master+mp+pb not-bound    7668
>> master+mp+pb bound           5765
>>
>> I did some gprof runs but could not find anything obvious that is going
>> wrong.
>>
>> Lukas
>>
>>
>> On 11/07/12 04:21, Wei Cao wrote:
>>> I added a macro to conditional compile the patch because I think it
>>> can be more selectable, I can remove the macro, fix the compilation
>>> error and test on mingw platform in later version.
>>>
>>> how about provide another BIF named port_flag (like process_flag) to
>>> let user bind port to a given scheduler?
>>>
>>>
>
>


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-patches/attachments/20120717/366446dd/attachment.htm>