[erlang-questions] Concurrent processes on multi-core platforms with lots of chatter

[Evans, Matthew]

Hi,

I actually agree with both you and Zoltan. I absolutely don't want the developer to have any understanding of CPU architecture. But, at the same time I'm not 100% convinced that the runtime should know what an application is doing or needs to do (other than in the way it does today).

This is one reason why I thought of something along the lines of a process group (Zoltan's idea has merits too). By that I mean the developer knows that process A, B, C and D are going to be involved in a large amount of IPC for the next few seconds (or longer, or shorter). The developer shouldn't care if it is single core or multi-core. All they are saying to the runtime is "please ensure that these processes are on the same scheduler for the next X micro seconds since they are going to be communicating a lot". Now the runtime could ignore that request if it thought things could run faster if they were on different cores.

I think my point is that you are correct in that developers should not care what the CPU topology is, but at the same time you don't want the VM making bad choices too.

Interesting discussion....

________________________________
From: Wallentin Dahlberg [mailto:wallentin.dahlberg@REDACTED]
Sent: Monday, November 30, 2009 2:46 PM
To: Evans, Matthew
Cc: erlang-questions@REDACTED
Subject: Re: [erlang-questions] Concurrent processes on multi-core platforms with lots of chatter

2009/11/30 Evans, Matthew <mevans@REDACTED<mailto:mevans@REDACTED>>
First the good news: When running tests that do more than just message passing the SMP features of R13B02 are leaps and bounds over R12B05 that I was running previously. What I have however noticed is that in a pure messaging test (lots of messages, in a tight loop) we appear to run into caching issues where messages are sent between processes that happen to be scheduled on different cores. This got me into thinking about a future enhancement to the Erlang VM: Process affinity.

In this mode two or more processes that have a lot of IPC chatter would be associated into a group and executed on the same core. If the scheduler needed to move one process to another core - they would all be relocated.

Although this grouping of processes could be done automatically by the VM I believe the decision making overhead would be too great, and it would likely make some poor choices as to what processes should be grouped together. Rather I would leave it to the developer to make these decisions, perhaps with a library similar to pg2.

Process/scheduler affinity has been discussed at length before and there are several schools of thought on the matter.

It is true that accounting and statistics gathering in the runtime has some overhead. However, this should be handled by the scheduler. The user (the erlang developer) might not have all the information at hand and if he doesn't he has to collect them which also costs. It would be easier for the system to schedule the process right. The scheduler still has collect other information and make decisions on memory models, memory distance, number of processing units, process to process message affinity, load balancing and other characteristics. It would be reasonable easy to model a scheduler algorithm after these charateristics and we can make it much more dynamic in the runtime system.

Ideally the user shouldn't be concerned about the hardware he is running on.   Thats my take on the situation anyway.

Regards,
Björn-Egil