shared memory in Erlang

[Ulf Wiger]

On Tue, 30 Jan 2001, David Gould wrote:

>On Tue, Jan 30, 2001 at 04:50:32PM -0800, Jim Larson wrote:
> 
>> >I do not really understand how database applications or network
>> >applications can be implemented in Erlang efficiently without some kind
>> >of form of shared memory.  Only providing message passing does not seem
>> >to be a good choice for these kind of applications.
>> 
>> I don't understand your concern here.  Shared memory is only useful
>> for communicating with other OS processes on the same host, so
>> you're either having to integrate with applications written in
>> other languages, or perhaps take advantage of a multiprocessor
>> through running several Erlang nodes on the same host.  Other than
>> that, network and database applications seem to work fine in Erlang.
>
>As a database engine guy in a former life, I think I do understand.
>Basically database engines run as a collection of processes (one per
>physical CPU) sharing a common set of datastructures:
>
>   the database page cache
>   database metadata (table names, schema definitions, procedure defs etc)
>   lock manager tables
>   transaction / logging information
>   scheduler information
>   thread data including execution stacks

A few comments from my end:

- There was an Erlang implementation with a global heap, called VEE,
  developed by Robert Virding. It used incremental GC, and did not
  copy messages. It didn't catch on though. Implementing message
  passing can of course be done transparently to erlang. One should
  also consider latency when dealing with message passing programs.

- Implementing ets:insert and ets:lookup without copying could
  probably be done, but it's not obvious how it should be done.
  Apparently, experiments have been made with ets tables on the
  process heap, but this didn't turn out as well as expected
  (mainly due to GC.) The obvious problem with non-copying ets
  shows up when multiple processes update the same object; 
  you would then need to keep and garbage collect the old object,
  if it is referenced by some process(es). Private ets tables
  could be implemented without copying, but for various reasons
  (mainly debugging), private ets tables are not used much.

- The erlang processor, presented at EUC 2000, does not copy messages.
  The main thing wrong with the erlang processor at this time is that 
  it is not ready yet.

- There was also a multipro erlang implementation
  http://www.erlang.se/publications/xjobb/0089-hedqvist.pdf
  Good reading for those who are interested. Demand for MP erlang
  was not high enough to warrant the extra effort needed to make
  it a product... at the time.

- We (AXD 301) are discussing the idea of having a shared memory
  database that has erlang and C applications as clients.

- We're also starting to look into a port of erlang to OSE/Delta
  (http://www.enea.com/PDF/rtk.pdf). We will at least study the 
  possibility to make a native port, which maps erlang processes
  onto OSE threads. There are several good reasons to look into 
  this.

- I've also played with the idea of using something like Encore's
  Reflective Memory for really fast replication of process state
  between processors - distributed shared memory if you will.
(http://www.encore.com/products/hardware/reflective/980012.shtml)
  It's not exactly clear to me if it could be reasonably well
  integrated into erlang, semantically, or if one would simply 
  write a driver that interfaces to it. Our main problem with 
  Encore's solution is that the memory size is too small for us.

/Uffe
-- 
Ulf Wiger                                    tfn: +46  8 719 81 95
Senior System Architect                      mob: +46 70 519 81 95
Strategic Product & System Management    ATM Multiservice Networks
Data Backbone & Optical Services Division      Ericsson Telecom AB