[erlang-questions] "processor affinity" for processes

[Zvi]

Kenneth Lundin wrote:
> 
> See questions below.
> 
> On Jan 10, 2008 3:24 PM, Zvi <exta7@REDACTED> wrote:
>>
>> Another usefull feature will be specifying CPU Affinity for ports and
>> external processes launched via os:cmd/1 .
> 
> This might be a potentially interesting feature, but what knowledge,
> research or measurements do
> you have which says that you get better performance with that feature?
> 

Kenneth,

I'll post here some snippets from my presentation "Migration to Multi-Core"
(I prepared it for one-day seminar, maybe I'll clean it up and put on the
web some day):

----------------------------------------
Why mess with CPU Affinity?
- Legacy Code Migration

Usually OS handles CPU Affinity automatically very good.
So why mess with it?
* When migrating old applications to multi-core CPU, their performance may
start degrading. This happens because OS moving application’s threads and
interrupt routines between cores.
* By fixing application’s process and threads to CPU #0, application will
start behave the same way, as on uniprocessor machine.

----------------------------------------
Why mess with CPU Affinity?
- Real-Time and Determinism

* By setting CPU affinity for our application’s thread - we get
deterministic execution times, needed for Real-Time.
* OS will no more move our threads between cores.
* For example, let’s say we developing video surveillance system using
dual-core CPU.
We have two threads:
#1 – receiving video frames from the Matrox Frame Grabber
#2 – transmitting the received frames via TCP/IP to the Server
By setting affinity of the receiving thread on CPU #0 and of the
transmitting thread on CPU #1 – we getting deterministic execution times.

------------------------------------------
Why mess with CPU Affinity? 
- Memory Affinity

Cache locality

* There are some multi-core CPUs with separate L1 caches (for example:
Pentium-D, some Intel’s quad-core CPUs).
* If OS moving thread from one core to another, the data is missing on the
2nd core, so it must fetch data from RAM, thus we get performance
degradation.

NUMA

* On the NUMA, each memory bank connect directly to single CPU. When
accessing RAM from another CPU it will be slower.
* By setting thread’s CPU affinity and allocating memory on the faster RAM
bank, we getting performance improvement.




Kenneth Lundin wrote:
> 
>> My current workarround is to launch script whcih temporary changing CPU
>> affinity of the executable.
> 
> Can you explain more about this?
> A workaround for what?
> What do you want to achieve with explicit spec of CPU affinity?
> 
> Suppose you already use all processor cores or processors inside the
> Erlang VM, how do
> you then know if it pays off to specify a specific CPU for the external
> program?
> 

There are also need to control number of CPUs/cores Erlang VM using, then
the rest we can use for ports or another non-Erlang external processes.

for example, I have 8 cores server, when I running Erlang VM it shows smp:8.
Now let's say I want Erlang VM only use 1 core and control long-running
computations/encoding on the other cores.
In Windows I can do it using this command:

%% running mp3 encoding on CPU #3
Cmd = "ffmpeg -y -i input.wav output.mp3".
AffinityMask = "100".%% this means CPU #3
os:cmd("process.exe -a \""++Cmd++"\" "++AffinityMask).

I hope this clear the things.
Best Regards,

Zvi

-- 
View this message in context: http://www.nabble.com/%22processor-affinity%22-for-processes-tp9956818p14812184.html
Sent from the Erlang Questions mailing list archive at Nabble.com.