[erlang-questions] clarify: why is Mnesia not fit for large databases
Juan Jose Comellas
Tue Dec 11 23:59:36 CET 2007
This may not be the response you're looking for, but have you tried
Postgres? We're using it for a database that's over 100 GB with very good
On 12/11/07, Dominic Williams <erlang@REDACTED> wrote:
> Ulf Wiger wrote :
> > Mnesia doesn't, but dets files are currently limited to 2
> > GB per file, and mnesia uses dets files for disc_only
> > tables. In a fragmented disc_only table, that would
> > amount to max 2 GB per fragment (which is not something
> > that mnesia will check or enforce, so living close to
> > that limit is inadvisable to say the least).
> Thanks, that confirms my understanding that the 2GB limit
> can be overcome with fragmented tables.
> Would monitoring the size of each fragment and raising an
> alarm when it approaches the 2GB limit be a simple way to
> stay out of trouble? A scheme in which additional fragments
> get automatically created does not seem all that hard to
> imagine, either...
> > It's reasonable to assume that mnesia can safely handle
> > databases of a number of gigabytes, perhaps (tounge in
> > cheek) a hundred or so gigabytes, if one thinks
> > carefully, and the access patterns are favourable. But I
> > don't know of anyone who actually does that, and most
> > people who have databases that large tend to not want to
> > be guinea pigs. (:
> Right. The reason I ask, though, is that we need the nice
> real-time and distributed characteristics of mnesia, and our
> current use of MySQL is proving to be a performance
> bottleneck and a constant source of operational problems. I
> am just trying to decide whether the time is better spent
> redesigning our MySQL database and replication scheme, or
> pushing mnesia's limits (we use mnesia anyway for other,
> smaller tables, so having a single tool would be nice).
> Ulf, could you elaborate on what you mean by "favourable
> access patterns"?
> Is there actually anyone out there using mnesia in the 100GB
> Dominic Williams
> erlang-questions mailing list
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the erlang-questions