shared memory in Erlang
Wed Jan 31 01:50:32 CET 2001
In message <3A77203C.DAE68BEB@REDACTED> Karel Van Oudheusden writes:
>Are there Erlang compilers that support shared memory between processes?
I can interpret your question in one of three ways:
1) "Are there Erlang implementations that allow you to send some data
between Erlang processes on the same Erlang node without copying?"
Yes - in Open-Source Erlang, binary-type data is passed by
reference in messages.
2) "Are there Erlang implementations that pass *all* data between
Erlang processes without copying?"
Not to my knowledge. Although the functional properties
of Erlang should allow such an implementation, I don't
think it's desirable for two reasons:
- it would imply a single global heap, or massive
synchronization between per-process heaps, which
would prevent independent per-process GC, which
would hurt Erlang's soft real-time properties;
- I recall hearing that the overhead of message
passing within an Erlang node is high enough that
the cost of copying message data isn't overwhelming,
for typically-sized messages.
3) "Are there Erlang implementations that allow shared memory IPC
between and Erlang node and another OS process?"
Depending on your needs, you might be able to create a
special-purpose linked-in driver that can transform messages
into shared-memory updates, and vice-versa. Using this
mechanism, you incur an overhead in copying data, even from
binaries, into the shared memory, and vice-versa.
Scott Lystig Fritchie implemented hacks to R6B that put
all of Erlang's binaries in a shared-memory space, so that
bulk data received from the network could be processed by
Erlang code and sent to another C application, with zero-copy
overhead. Unfortunately, the performance wasn't as great
as we'd hoped, possibly due to the locking overheads of
the shared-memory malloc() library we needed to use.
>I do not really understand how database applications or network
>applications can be implemented in Erlang efficiently without some kind
>of form of shared memory. Only providing message passing does not seem
>to be a good choice for these kind of applications.
I don't understand your concern here. Shared memory is only useful
for communicating with other OS processes on the same host, so
you're either having to integrate with applications written in
other languages, or perhaps take advantage of a multiprocessor
through running several Erlang nodes on the same host. Other than
that, network and database applications seem to work fine in Erlang.
More information about the erlang-questions