[erlang-questions] Concurrent processes on multi-core platforms with lots of chatter

[Robert Virding]

Another solution would be to use the existing process groups as these are
not really used very much today. A process group is defined as all the
processes which have the same group leader. It is possible to change group
leader. Maybe the VM could try to migrate processes to the same core as
their group leader.

One problem today is that afaik the VM does not keep track of groups as
such, it would have to do this to be able to load balance efficiently.

Robert

2009/11/30 Evans, Matthew <mevans@REDACTED>

> Hi,
>
> I've been running messaging tests on R13B02, using both 8 core Intel and 8
> core CAVIUM processors. The tests involve two or more processes that do
> nothing more than sit in a loop exchanging messages as fast as they can.
> These tests are, of course, not realistic (as in real applications do more
> than sit in a tight loop sending messages), so my findings will likely not
> apply to a real deployment.
>
> First the good news: When running tests that do more than just message
> passing the SMP features of R13B02 are leaps and bounds over R12B05 that I
> was running previously. What I have however noticed is that in a pure
> messaging test (lots of messages, in a tight loop) we appear to run into
> caching issues where messages are sent between processes that happen to be
> scheduled on different cores. This got me into thinking about a future
> enhancement to the Erlang VM: Process affinity.
>
> In this mode two or more processes that have a lot of IPC chatter would be
> associated into a group and executed on the same core. If the scheduler
> needed to move one process to another core - they would all be relocated.
>
> Although this grouping of processes could be done automatically by the VM I
> believe the decision making overhead would be too great, and it would likely
> make some poor choices as to what processes should be grouped together.
> Rather I would leave it to the developer to make these decisions, perhaps
> with a library similar to pg2.
>
> For example, library process affinity (paf) could have the functions:
>
> paf:create(Name,[Opts]) -> ok, {error, Reason}
> paf:join(Name,Pid,[Opts]) -> ok, {error, Reason}
> paf:leave(Name,Pid) -> ok
> paf:members(Name) -> MemberList
>
> An affinity group would be created with options for specifying the maximum
> size of the group (to ensure we don't have all processes on one core), a
> default membership time within a group (to ensure we don't unnecessarily
> keep a process in the group when there is no longer a need) and maybe an
> option to allow the group to be split over different cores if the group size
> reaches a certain threshold.
>
> A process would join the group with paf:join/3, and would be a member for
> the default duration (with options here to override the settings specified
> in paf:create). If the group is full the request is rejected (or maybe
> queued). After a period of time the process is removed from the group and a
> message {paf_leave, Pid} is sent to the process that issued the paf:join
> command. If needed the process could be re-joined at that time with another
> paf:join call.
>
> Any takers? R14B01 perhaps ;-)
>
> Thanks
>
> Matt
>