[erlang-questions] Re: how to scale into the cloud using process? example computing simple average
Sat May 30 14:44:32 CEST 2009
----- "Joel Reymont" <> wrote:
> The intractable (I think) problem is using Mnesia as a backend for
> internet services. I don't think it can be used since there's now way
> to automatically sync up the various database instances after a
> network split.
I think it ought to be fairly straightforward to design
an arbitrator similar to the one used by MySQL Cluster.
The tools needed to detect partitioned network and to
resolve it are there - the thing that mnesia doesn't
provide is a default resolution algorithm. If the
algorithm used by MySQL Cluster is acceptable, I see
no reason why it couldn't be applied to Mnesia as well.
(For those who don't want to go chasing for it,
the MySQL Arbitrator basically selects the biggest
group of connected nodes, or the group that reported
to the Arbitrator last. This is from memory. The details
may be wrong.
FWIW, the AXD 301 used a somewhat similar arbitration
mechanism: each node tried to ping itself through the
switch core. If it couldn't, it had no means to control
the HW resources and had to reboot; otherwise, the
node that had a stable copy of the traffic handling
application became the master; if both nodes were
running traffic handling (which could happen if
the outage lasted a few seconds), I believe the one
hard-wired as Control Processor 1 was designated
Bottom line: if you can pick an arbitration algorithm
that offers suitable recovery behaviour, I believe you
can implement it on top of what mnesia provides.
One could imagine a peer-to-peer resolution protocol
(difficult to make generic) or a 'central observer',
which is the MySQL strategy.
Or are you referring to the lack of a 'merge' of
two table copies? This is a more difficult thing to
add. Right now, that would have to be done by
extracting records from the non-master, have it
restart and load tables from the master, then
re-inserting the records. Since this cannot be done
within a larger transaction, it might be necessary to
delay requests from the application layer during this
Which DBMSes have a good solution to this problem?
How do they do it? And what prevents using the same
strategy with mnesia?
(Mnesia's supposed lack of support for handling
partitioned networks has more or less become an
urban legend. Everyone assumes that this is a
major weakness in mnesia, but there is very little
detail about how the competing alternatives
supposedly handle this in a much better way.
I would love to be enlightened about this.)
CTO, Erlang Training & Consulting Ltd.
More information about the erlang-questions