[erlang-questions] Mnesia

[lloyd@REDACTED]

Hello,

During a break at the Chicago Erlang Workshop Fred Hebert mentioned that Mnesia is a fine database for the 1990s.

Mnesia was a major draw when I was first attracted to Erlang in part because I hate transiting syntactic boundaries when I'm programming.

I asked Fred what it would it take to upgrade Mnesia for the 21st century (or, at least, for the next decade). He didn't know.

Martin's list of negatives may be a good place to start. 

So, just how much effort and knowledge would it take to overcome or ameliorate these and other unspecified negatives of Mnesia to produce Mnesia2?

Best,

LRP


-----Original Message-----
From: "Martin Karlsson" <martink@REDACTED>
Sent: Monday, October 12, 2015 9:04pm
To: "Steve Davis" <steven.charles.davis@REDACTED>
Cc: "Erlang Questions" <erlang-questions@REDACTED>
Subject: Re: [erlang-questions] Mnesia

Hi Steve,

I'm with you:)

The negatives are being mentioned a lot and unfortunately put people
off instead of just being used as good things to know about when you
are picking your technology.

If you know about the quirks you can usually come up with a workaround
that matches your use-case.

It is about weighing pros and cons. Why not list a few of the
negatives and what you can do to work around them.

PROBLEM: Net splits must be manually handled.
WORKAROUND:
- https://github.com/uwiger/unsplit for automatic healing
- Use the majority options (when you need to heal you choose the
majority partition when you pick master nodes). This should reduce
(eliminate?) risk of data loss.

PROBLEM: disc_only_copies uses dets and is limited to 2GB
WORKAROUND:
- Partition the tables
- Use the mnesia_ex patch with leveldb backend (to be released soon I
think, it was mentioned in another email just recently)
- Perhaps you can use disc_copies and hold your data-set in RAM

PROBLEM: Prone to overloading
WORKAROUND: Load regulate. Reduce concurrency. I think I read
somewhere that mnesia and ets works best with a max concurrency of the
number of scheduler threads and degrades from there on.

PROBLEM: Slow startup
WORKAROUND: Increase the number of table loaders.

PROBLEM: Upgrading table definition. transform_table can have problems
with big and distributed tables.
WORKAROUND: Treat large tables like a key value store to reduce risk
of having to modify the record

I'm sure there are more problems and workarounds especially if you
start pushing it (for example like whatsapp did).

Also I've heard dirty_write is not safe with replicated tables. I
don't know if this is true but is holding me back from using dirty
transactions rather than transactions (which I don't need apart from
safety on replicated tables). If anyone knows about this one please
let me know.

Feel free to add/remove/correct items:)

Cheers,
Martin


On 13 October 2015 at 13:08, Steve Davis <steven.charles.davis@REDACTED> wrote:
> Hi,
>
> I’m almost ashamed to say that it’s taken me over 5 years to come around to understanding the value of mnesia.
>
> I bought the “well-known” negatives too fast. I have explored relational connectors, DHT solutions and a number of other approaches...
>
> It’s dawned on me *finally* that 90% of the time, a well implemented mnesia solution would have been better, faster, and cheaper.
>
> Did I mention "better"?
>
> Has anyone else had this experience?
>
> /s
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
_______________________________________________
erlang-questions mailing list
erlang-questions@REDACTED
http://erlang.org/mailman/listinfo/erlang-questions