shared memory in Erlang
Wed Jan 31 10:36:02 CET 2001
On Tue, 30 Jan 2001, David Gould wrote:
>On Tue, Jan 30, 2001 at 04:50:32PM -0800, Jim Larson wrote:
>> >I do not really understand how database applications or network
>> >applications can be implemented in Erlang efficiently without some kind
>> >of form of shared memory. Only providing message passing does not seem
>> >to be a good choice for these kind of applications.
>> I don't understand your concern here. Shared memory is only useful
>> for communicating with other OS processes on the same host, so
>> you're either having to integrate with applications written in
>> other languages, or perhaps take advantage of a multiprocessor
>> through running several Erlang nodes on the same host. Other than
>> that, network and database applications seem to work fine in Erlang.
>As a database engine guy in a former life, I think I do understand.
>Basically database engines run as a collection of processes (one per
>physical CPU) sharing a common set of datastructures:
> the database page cache
> database metadata (table names, schema definitions, procedure defs etc)
> lock manager tables
> transaction / logging information
> scheduler information
> thread data including execution stacks
A few comments from my end:
- There was an Erlang implementation with a global heap, called VEE,
developed by Robert Virding. It used incremental GC, and did not
copy messages. It didn't catch on though. Implementing message
passing can of course be done transparently to erlang. One should
also consider latency when dealing with message passing programs.
- Implementing ets:insert and ets:lookup without copying could
probably be done, but it's not obvious how it should be done.
Apparently, experiments have been made with ets tables on the
process heap, but this didn't turn out as well as expected
(mainly due to GC.) The obvious problem with non-copying ets
shows up when multiple processes update the same object;
you would then need to keep and garbage collect the old object,
if it is referenced by some process(es). Private ets tables
could be implemented without copying, but for various reasons
(mainly debugging), private ets tables are not used much.
- The erlang processor, presented at EUC 2000, does not copy messages.
The main thing wrong with the erlang processor at this time is that
it is not ready yet.
- There was also a multipro erlang implementation
Good reading for those who are interested. Demand for MP erlang
was not high enough to warrant the extra effort needed to make
it a product... at the time.
- We (AXD 301) are discussing the idea of having a shared memory
database that has erlang and C applications as clients.
- We're also starting to look into a port of erlang to OSE/Delta
(http://www.enea.com/PDF/rtk.pdf). We will at least study the
possibility to make a native port, which maps erlang processes
onto OSE threads. There are several good reasons to look into
- I've also played with the idea of using something like Encore's
Reflective Memory for really fast replication of process state
between processors - distributed shared memory if you will.
It's not exactly clear to me if it could be reasonably well
integrated into erlang, semantically, or if one would simply
write a driver that interfaces to it. Our main problem with
Encore's solution is that the memory size is too small for us.
Ulf Wiger tfn: +46 8 719 81 95
Senior System Architect mob: +46 70 519 81 95
Strategic Product & System Management ATM Multiservice Networks
Data Backbone & Optical Services Division Ericsson Telecom AB
More information about the erlang-questions