[erlang-questions] Mnesia

Richard Carlsson carlsson.richard@REDACTED
Wed Oct 14 10:46:33 CEST 2015


The simple view of Mnesia is that it's a transaction layer on top of ETS
tables, with some varying forms of backing storage. Transactions are
pessimistic, based on locking. Some small optimizations have been done to
the locking mechanism in recent years, but it's maybe more could be done in
that area. It might also be possible to add built-in support for optimistic
transactions based on timestamps or compare-and-swap, for certain usage
patterns. The semantics of dirty reads/writes need to be better documented,
and if possible cleaned up a bit, because the behaviour can depend on table
type and whether or not the tables are local or remote. The table size
problems can probably be considered to be solved by mnesia_ext with leveldb
or other backends. None of this will make it less of a 90's database
though. Adding eventual consistency (e.g. based on vector clocks) as
alternative to transactions would make it more modern.

The big thing that would help, as Gordon mentioned, is a new
distribution/replication layer. The existing one basically assumes that
tables are not huge and the network between nodes is fast and reliable with
netsplits being rare, like in a small cluster in a telecom base station. We
use Mnesia, but not in distributed mode - we have a custom distribution
layer (stable, but very limited) on top of local Mnesia instances that are
not directly aware of each other.

    /Richard



        /Richard

On Wed, Oct 14, 2015 at 9:25 AM, Gordon Guthrie <gguthrie@REDACTED> wrote:

> Mnesia is also a pain when it partitions - you have to write your own
> reconciliation programmes.
>
> There has been a lot of work done on eventual consistency in *the other*
> Erlang database - Riak (disclaimer I am working at Basho now)
>
> Riak implements eventual consistency - and post-partition self-healing
> using Consistent Replicated Data Types (or CRDTs) and the canonical set of
> standalone CRDT libraries is written in Erlang:
> https://github.com/basho/riak_dt
>
> There is a comprehensive reading list here:
> https://christophermeiklejohn.com/crdt/2014/07/22/readings-in-crdts.html
>
> The combination of using Klarna’s (forthcoming) leveldb backend and a CRDT
> eventual consistency layer on top would be an interesting start offering a
> distributed transactional database with eventual consistency
>
> Gordon
>
> On 14 Oct 2015, at 02:16, Richard A. O'Keefe <ok@REDACTED> wrote:
>
>
> On 14/10/2015, at 6:46 am, <lloyd@REDACTED> <lloyd@REDACTED>
> wrote:
>
>
> I asked Fred what it would it take to upgrade Mnesia for the 21st century
> (or, at least, for the next decade). He didn't know.
>
>
> There's one thing that strikes me.
>
> I could go to a shop today and buy a 1 TB external drive for
> NZD 75, including Goods and Services Tax of 15%.  (At least
> that's what the ad I saw a couple of days ago said.)
> That's almost exactly USD 50.  This is a drive that fits in
> a shirt pocket, with room left over for all sorts of junk.
>
> To make Mnesia a data base for the 2010s, it has to be able to
> handle at least 1TB of data.  Heck, I've got enough goodies-for-
> research money left that I could get the department to buy me
> ten of these gadgets, so let's say Mnesia
> - should be able to handle a single table in the low TB
> - should be able to handle a collection of tables in the
>   tens of TB
> - where "handle" includes creating, populating, checking,
>   recovering, and accessing in "a reasonable time".
>
> That's a "single machine data base for the 2010s".
> Of course there are multicore, cluster, and network issues as
> well.
>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20151014/a6cc76e2/attachment.htm>


More information about the erlang-questions mailing list