Atomic ets

[Robert Raschke]

Ulf Wiger wrote:
> Robert Raschke wrote:
> 
>> Where does this desire for programming with locks
>> come from? Erlang has a much simpler way of
>> dealing with parallelism.
> 
> But there are issues with using processes too.
> I have suggested serializing all writes through
> a gen_server, but still using ets. This has
> some performance consequences, as Thomas Lindgren
> has pointed out. They get further compounded if
> you also pipe the reads through the same process,
> (also because you get two term copies per read
> instead of one) and if you don't, you may run into
> consistency issues if one process writes, while
> others read.
> 
> Storing the data in a functional data structure
> on the process heap solves these issues for the 
> most part, but then, if you have a very large 
> data set, you will be at odds with the garbage
> collector.

I can understand the problems you get once you decide to go down the
route of shared storage between parallel tasks.  I guess I myself
would find it way too complicated.  My approach to dealing with shared
storage is to use one of the two technologies that have solved this
problem: databases or file systems.  I am lazy.

In my mind, if you want to have a parallel application, then any kind
of shared storage creates a bottleneck.  You can whittle away all you
like it will remain a limiting factor.

Granted, the opposite approach of message passing and local storage
has the overhead of copying lots of data.

So, you have the choice of being limited by access to storage or
duplication of storage.  I'm guessing that quantum computing will try
to address this particular problem, amongst others.

But back to atomic ets.  Implementing such a thing would bring you
halfway toward mnesia, wouldn't it?  Maybe it would be a nice tutorial
on how start writing a database system.

Robby