shared memory in Erlang

[Jim Larson]

In message <3A77203C.DAE68BEB@REDACTED> Karel Van Oudheusden writes:
>Are there Erlang compilers that support shared memory between processes?

I can interpret your question in one of three ways:

1) "Are there Erlang implementations that allow you to send some data
between Erlang processes on the same Erlang node without copying?"

	Yes - in Open-Source Erlang, binary-type data is passed by
	reference in messages.

2) "Are there Erlang implementations that pass *all* data between
Erlang processes without copying?"

	Not to my knowledge.  Although the functional properties
	of Erlang should allow such an implementation, I don't
	think it's desirable for two reasons:

		- it would imply a single global heap, or massive
		  synchronization between per-process heaps, which
		  would prevent independent per-process GC, which
		  would hurt Erlang's soft real-time properties;

		- I recall hearing that the overhead of message
		  passing within an Erlang node is high enough that
		  the cost of copying message data isn't overwhelming,
		  for typically-sized messages.

3) "Are there Erlang implementations that allow shared memory IPC
between and Erlang node and another OS process?"

	Depending on your needs, you might be able to create a
	special-purpose linked-in driver that can transform messages
	into shared-memory updates, and vice-versa.  Using this
	mechanism, you incur an overhead in copying data, even from
	binaries, into the shared memory, and vice-versa.

	Scott Lystig Fritchie implemented hacks to R6B that put
	all of Erlang's binaries in a shared-memory space, so that
	bulk data received from the network could be processed by
	Erlang code and sent to another C application, with zero-copy
	overhead.  Unfortunately, the performance wasn't as great
	as we'd hoped, possibly due to the locking overheads of
	the shared-memory malloc() library we needed to use.

>I do not really understand how database applications or network
>applications can be implemented in Erlang efficiently without some kind
>of form of shared memory.  Only providing message passing does not seem
>to be a good choice for these kind of applications.

I don't understand your concern here.  Shared memory is only useful
for communicating with other OS processes on the same host, so
you're either having to integrate with applications written in
other languages, or perhaps take advantage of a multiprocessor
through running several Erlang nodes on the same host.  Other than
that, network and database applications seem to work fine in Erlang.

Jim