Mnesia dB size limits
Thu Aug 31 12:37:14 CEST 2006
> Valentin - maybe you can share some of your experience with large Mnesia
> databases with us? What are the things we need to be careful of?
The point I've been trying to make is that one should encourage usage, and
thus level of commitment to mnesia. By amplifying differences and/or
problems this is not going to happen. Urban legned has it: there was a
French mathematician that prove that it is impossible to fly an object
heavier than air. Thanks goodness that some people never got around to read
his thesis. Nobody remembers the mathematician anymore.
I had a number of reasons for not sharing my experiences, primarely because
I do not consider one-fits-it-all approach is the way to go. I strongly
beleive that Erlang and Mnesia are at their best when viewed in a context of
a given problem. However, since you've asked, I would like to share few
1) Do your own replication. We've created a database journalling system that
is appropriate for our data model, and it serves dual role -- replication,
and roll-forward functions. IMHO, it would be great if we can take this
2) Always create specialized API to access your data -- it will help you
repacle mnesia if you have to do that in a future.
3) Always use mnesia:activity, instead of "higher level" functions. It
might be a bit "low-level" initially, but will pay a significant dividend in
4) With a risk of going to extreeme, always had table fragmentation in mind,
unless you're talking small databases, say up to 150000 records. If the
database gets bigger, you would be better off with fragmentation.
5) Fragmentation also helps with recovery -- under extreme conditions it is
much easier to find extra 204MB on disk, and do it ten times over, than 2GB
all at once.
6) If in doubt -- try it out first, and ask questions later. Help others
help you. Erlang is extremely friendly to prototyping.
And BTW: There was a bug in dets, picked up & fixed by Hans Bolinder (THANX
HANS!!!). It is my understanding that R11B-1 shall include this bug-fix.
More information about the erlang-questions