[erlang-questions] erlang smp on aix

[Jon Watte]

I haven't kept up on the latest in AIX kernel development, but it seems that
IBM on POWER worries more about virtualizing processors across multiple
partitioned workloads, rather than the best possible API performance for a
unified machine view. It's possible that the AIX kernel does not have the
kind of primitives that Erlang needs to run well in highly parallel
situations.
When I worked with them, IBM technical support and ISV support was fairly
well educated, and you could possible send an strace or a copy of your
program to them to ask them what they think and get a real answer (as
opposed to some other IHVs out there...)

Sincerely,

jw

--
Americans might object: there is no way we would sacrifice our living
standards for the benefit of people in the rest of the world. Nevertheless,
whether we get there willingly or not, we shall soon have lower consumption
rates, because our present rates are unsustainable.



2011/9/1 caox <caox@REDACTED>

> No, the kernel poll is not supported on AIX.
>
> AIX has pollset interface like epoll but seems can not be used by erts. Is
> this the reason of thread_waitlock?
>
> Our test scenario is quite simple. Create 1000 erlang processes or more to
> execute our main function repeatedly. This function do nothing special but
> call a linkded-in driver with port_control/3 and do some template rendering
> then return. The whole function does not have any IO operations.
>
> During the test on a 2 cores AIX, we found no thread_waitlock() call  in
> the strace log, which was full of poll() call.
>
> With same test scenario on a 4 cores Centos with kernel-poll off, there is
> no sys call in the strace log(even without poll() call). Cpu will be
> consumed perfectly and we can gain quite high system throughput.
>
> But when migrating to 4cores or 8cores AIX, thread_watilock() appears and
> there is lots of context switches according to vmstat. As a result, the
> throughput drop dramatically compared with Centos.
>
> So, what is role of poll() in the scenario, or in the erlang processes
> scheduling?
>
> BR
>
>
> 在 2011-9-2，上午12:51， Jon Watte 写道：
>
> Did you enable kernel_poll? And does AIX actually support a suitable kernel
> polling mechanism (like epoll, etc)?
>
> Sincerely,
>
> jw
>
>
> --
> Americans might object: there is no way we would sacrifice our living
> standards for the benefit of people in the rest of the world. Nevertheless,
> whether we get there willingly or not, we shall soon have lower consumption
> rates, because our present rates are unsustainable.
>
>
>
> On Thu, Sep 1, 2011 at 3:01 AM, caox <caox@REDACTED> wrote:
>
>> hi
>>
>> we have a application used to run on centos but now we want to migrate it
>> to AIX. During the test on Aix, we found the cpu scaling of the system is
>> quite disappointing. With 8 cores and erlang smp auto, there were very high
>> system cpu consumption and lots of thread_wait_lock call in the
>> truss(strace) log, which would not appear on centos under the same test
>> scenario. And the sys call will disappear if we turn smp off.
>>   So we guess there is something wrong with smp on Aix, perhaps caused by
>> the schedulers switching. But we don't know where the thread_wait_call is
>> from and what it is for.
>>  By the way, it is quite strange that the smp description of erl shell
>> would be smp32:8 with smp auto on the 8 cores aix.
>>  Is there any hint to fix the problem? Appreciate your help.
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110902/c406cb27/attachment.htm>