shared memory in Erlang

[Pekka Hedqvist]

>-----Original Message-----
>From: owner-erlang-questions@REDACTED
>[mailto:owner-erlang-questions@REDACTED]On Behalf Of David Gould

>
>Message passing?

In a way yes, msg-passing + context switch between two Erlang processes
running on different nodes with shared mem (same physical machine) is way
slower than if they ran in the same process space with either native threads
or (preferably) worker threads scheduling Erlang processes.

>
>More control of scheduling?

Yes, with the node modell one has to ensure that not all heavy Erlang
processes ends on one node and the the lighter ones ends up on the other
etc. With all processes scheduled within one process, either on native
threads or with worker-model this is taken care automagically.

>
>Something else?

>But any kind of sharing on an MP is going to involve some amount of
>overhead for spinlocks and other sychronization. This can eat most of the
>gains of an MP pretty fast. And MP friendly programs are much harder to
>debug for some kinds of bug.

The cost of locking really depends on the design of the runtime. The one
Tony and I did many years back basically kept most of the original JAM/BEAM
Erlang (multiple heaps, per-heap-gc, copying msg-passing, copying ets tables
etc) characteristics, with a few tweaks though. There locking-bottlenecks
would be hard to get even with many cpu:s. With a unified heap (VEE, or like
some parallel prologs has and most Java runtimes has) things becomes more
difficult and unpredictable. But the one we did mainly kept most of the good
(and bad) things from the separate heaps (which todays Erlang still use)
implementation but also took advantage of MP machines. I belive that a
productified version of that one would given users few backlash-surprises,
just run better on multiple CPU machines than any multi-Erlang-node aproach
could ever do.

Maybe obvious, but with an MP-Erlang the MP part would not be visible. Just
plain ol'Erlang.

>Erlang, not being coupled at all, avoids all this, but presumably can
>take advantage of the low latency message that could be built for nodes
>on a NUMA.

BTW, improving Erlang-communication between nodes on the same machine and on
separate machines can be done with new aproaches like VIA (www.viaarch.org).
And new IO devices (NGIO etc). Erlang node communication on LAN/SAN:s with
VIA could be really fast with via (avoiding tcp) etc and still portable.
There is a free VIA around for Linux called M-VIA
(http://www.nersc.gov/research/FTG/via/faq.html), but development around VIA
does not happen as fast as I thought a few years back, but the basic ideas
are just right (superfast direct userspace network IO, smart IO cards,
portability etc)

/pekka