[erlang-questions] Strange multi-core result on windows.

[Jesper Louis Andersen]

On 2011-11-28 17:40, Witold Baryluk wrote:
> On 11-27 13:30, Tony Rogvall wrote:
>> Interesting reading, please post a link to where to find the code and data examples to backup this findings, and
>> allow for others to repeat your experiments.
>> /Tony
>>
>> On 27 nov 2011, at 00:41, Ian wrote:
>>
>>> HI all,
>>>
>>> I have written a program in Erlang, that has 4 async processes in "string of pearls" formation. The results were surprisingly quick, so I threw a huge input file at it to see what would happen.
>>>
>>> Expected one process (the 2nd) to require more cpu than the others. As the limiting factor it would max out one core to 100%. The rest would be  less, constant and approx equal.  But this isn't what happened.
>>>
>>> Three cores were loaded about 40-50%. The 4th core was left at virtually idle (2%).  Job ran in 25 seconds, average load about 32%.
>>>
>>> So I moved the input and output files to a ram disk. And got much the same result. It ran in 22 seconds, and loaded 3 cores, one 50% and varying, the others 30% or so, Overall 30%.
>>>
>>> Then I tried with +A 5 and the RAM drive. Now it loaded one core 60 to 90%, and a a second about 40-60%. The other two cores flickered a little and settled back to background! Job ran in 23 seconds, average load about 30%.
>>>
>>> So I guess that disk was not limiting the speed. But what was? I never got the load over 35%.
>>>
>>> Can some kind person please explain the scheduling algorithm in the VM, and explain these results. They seem very odd to me.
>>>
>>> Thanks
>>>
>>> Ian
>>>
> Hi.
>
> I would add that Windows scheduler before Windows 7 performs
> unoptimally on multi-core machines. Explicit pinning (setting affinity)
> is needed to exploit full multi core performance.
> You can try setting affinity for Erlang schedulers (running as threads)
> as well set affinity of  Erlang processes to certain Erlang
> schedulers to improve schduling.
>
> However, it looks it may not be good explanation because
> you say 4th core was idle (but this may be due other aspects
> like hyperthreading, and other scheduler parameters).
>
> Exact Windows version and data about CPUs loads when single Erlang
> process is doing computations would be helpfull.
>

Also, take a look at the "percept" tool in Erlang. It is a concurrency 
profiler and it will give you some idea about which processes are 
runnable from the perspective of the Erlang VM. If it has large gaps 
blocking you know you may have a problem.


-- 
Jesper Louis Andersen
   Erlang Solutions, Copenhagen, DK