[erlang-questions] Shared memory and message passing

[Richard Carlsson]

Joel Reymont wrote:
> I thought Erlang as it is right now _does_ copy data. Am I mistaken?
> 
> Why the talk about about copying binaries, etc. otherwise?

The _behaviour_ (the semantics) of message passing must always
be as if each process has a separate copy (so that there is no
observable difference in behaviour if the communicating processes
are on the same node or on separate nodes). But if they _are_ on
the same node, the message passing can be optimized by letting
the processes share the data "under the hood", and just send a
pointer instead of making a complete copy. As long as there are
no destructive writes to data (and there aren't any in Erlang),
it does not matter, because the difference cannot be observed
(except in the amount of time it takes to send large messages).

The default Erlang/OTP runtime system does _not_ currently do
this sort of optimization, however. (It can be turned on by using
an alternative build of the runtime system.) In the default
system, each process has a separate memory area, and message
passing will copy data from the area of the sender to that of the
receiver. (The exception is large binary objects, which are handled
specially by placing them outside the normal data areas and using
reference counting to keep track of them. They are not copied when
sent as messages between processes on the same node.)

However, there has been an experimental version where all
processes shared a single area (giving fast message passing,
but causing problems with long garbage collection times). This
"shared" runtime system is now discontinued, in favour of a
"hybrid" version which tries to keep messages in a shared area
and keep process-local data in per-process areas. It is still
not the default, though: the old tried and true version with
separate areas only is what you get out of the box, since it
has very well known time/space properties. We'll see if the
hybrid system will replace it in some possible future.

	/Richard

PS. Shameless plug: see the July issue of TOPLAS (vol 28, issue 4)
for all the gruelling details on the hybrid architecture.