[erlang-questions] VM using only 2 out of 8 schedulers available

[Michal Ptaszek]

Hi Knut, 

have you tried to play with scheduler wakeup thresholds?
I would probably try running beam with '+swt very_low'
(see http://erlang.org/doc/man/erl.html for details). 

Best regards,
Michal Ptaszek


On Feb 16, 2012, at 5:00 PM, Knut Nesheim wrote:

> Hi,
> 
> We just tried moving more load to the node using only two schedulers.
> The VM is using 100% cpu on the two cores, with a run queue hovering
> between 50 and 100 processes. There is no change in active schedulers.
> 
> As far as I can see, the load is now at a point where the schedulers
> should become active. Is there anything we can do to understand more
> about why this is not happening?
> 
> Knut
> 
> On Thu, Feb 16, 2012 at 10:19 AM, Knut Nesheim <knutin@REDACTED> wrote:
>> Thanks for a quick reply.
>> 
>> During the GMT night which is our low point, we see quite low traffic
>> on each machine with maybe 5% cpu utilization of each of the 8 cores.
>> This could explain why some schedulers are put as idle. When traffic
>> picks up, we're using around 25% of each core.
>> 
>> The problem for us is that when traffic picks up again, the VM did not
>> start using the idle schedulers again. erlang:statistics(run_queue)
>> reports that there almost always are processes waiting, with peaks of
>> around 20 processes in our case. This causes much higher latency than
>> when using all cores.
>> 
>> We will try pushing more load to this node to see if it starts using
>> the idle schedulers.
>> 
>> Thanks
>> Knut
>> 
>> On Thu, Feb 16, 2012 at 8:01 AM, Kenneth Lundin
>> <kenneth.lundin@REDACTED> wrote:
>>> What is your real problem except that there is no reported cpu load on some
>>> of the cores?
>>> 
>>> When there is low load on the system i.e. when there are too few runnable
>>> Erlang processes some of the schedulers will automatically be put as idle
>>> letting the remaining schedulers run all processes. When the load increases
>>> the idle schedulers will be taken into service again.
>>> 
>>> Each scheduler normally runs on one processor core.
>>> 
>>> Can this explain what you have observed?
>>> 
>>> /Kenneth Erlang/OTP Ericsson
>>> 
>>> Den 15 feb 2012 17:53 skrev "Knut Nesheim" <knutin@REDACTED>:
>>>> 
>>>> Dear list,
>>>> 
>>>> From time to time we see the VM using only some of the available
>>>> cores. Typically 2 or 4 out of 8 available cores will be busy, while
>>>> the others are not used at all. From 'top' and 'htop' we see only some
>>>> cores being used.
>>>> 
>>>> We are running the halfword emulator on 8 core machines and the VM is
>>>> correctly reporting the number of schedulers and the underlying
>>>> topology. The nodes are utilizing roughly 25% of each of the 8 cores
>>>> during normal operation.
>>>> 
>>>> This problem appeared again today and we still have the process
>>>> running if anyone has any ideas for how to debug the issue.
>>>> 
>>>> We are on Ubuntu 10.04 LTS, 2.6.32-342-ec2, 64bit, R14B04 with the
>>>> halfword emulator. The only NIF library we are using is jiffy.
>>>> 
>>>> Regards
>>>> Knut
>>>> _______________________________________________
>>>> erlang-questions mailing list
>>>> erlang-questions@REDACTED
>>>> http://erlang.org/mailman/listinfo/erlang-questions
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20120217/e562f316/attachment.htm>