shared memory in Erlang

[David Gould]

On Tue, Jan 30, 2001 at 04:50:32PM -0800, Jim Larson wrote:
 
> >I do not really understand how database applications or network
> >applications can be implemented in Erlang efficiently without some kind
> >of form of shared memory.  Only providing message passing does not seem
> >to be a good choice for these kind of applications.
> 
> I don't understand your concern here.  Shared memory is only useful
> for communicating with other OS processes on the same host, so
> you're either having to integrate with applications written in
> other languages, or perhaps take advantage of a multiprocessor
> through running several Erlang nodes on the same host.  Other than
> that, network and database applications seem to work fine in Erlang.

As a database engine guy in a former life, I think I do understand. Basically
database engines run as a collection of processes (one per physical CPU)
sharing a common set of datastructures:

   the database page cache
   database metadata (table names, schema definitions, procedure defs etc)
   lock manager tables
   transaction / logging information
   scheduler information
   thread data including execution stacks

The database engines typically implement their own lightweight threading
(somewhat akin to Erlang processes), and are able to schedule threads
across multiple physical CPUs. That is, threads can migrate. So there is
not any local data, everything is in the shared memory, even if it is
private to a thread because that thread may next be scheduled onto a
different process/cpu.

It does not really look easy to build such a thing in Erlang. Possibly
a linked in driver could provide a suffcient interface to the critical
parts, eg lock manager, page cache, page qualification (get rows on this
page that match a predicate), page update etc.

Indeed, such a beast might be much cleaner and more stable than existing
db systems because of the enforced strict isolation of the shared memory
behind a defined interface (ie, no random pointer poking). Interesting
project for someone with a big need for a high performance Erlang dbms.

-dg

-- 
David Gould                                                 dg@REDACTED
SuSE, Inc.,  580 2cd St. #210,  Oakland, CA 94607          510.628.3380
You left them alone in a room with a penguin?!  Mr Gates, your men are
already dead.