[erlang-questions] clarify: why is Mnesia not fit for large databases

[Juan Jose Comellas]

This may not be the response you're looking for, but have you tried
Postgres? We're using it for a database that's over 100 GB with very good
results.


On 12/11/07, Dominic Williams <erlang@REDACTED> wrote:
>
> Hi,
>
> Ulf Wiger wrote :
>
> > Mnesia doesn't, but dets files are currently limited to 2
> >  GB per file, and mnesia uses dets files for disc_only
> > tables. In a fragmented disc_only table, that would
> > amount to max 2 GB per fragment (which is not something
> > that mnesia will check or enforce, so living close to
> > that limit is inadvisable to say the least).
>
> Thanks, that confirms my understanding that the 2GB limit
> can be overcome with fragmented tables.
>
> Would monitoring the size of each fragment and raising an
> alarm when it approaches the 2GB limit be a simple way to
> stay out of trouble? A scheme in which additional fragments
> get automatically created does not seem all that hard to
> imagine, either...
>
> > It's reasonable to assume that mnesia can safely handle
> > databases of a number of gigabytes, perhaps (tounge in
> > cheek) a hundred or so gigabytes, if one thinks
> > carefully, and the access patterns are favourable. But I
> > don't know of anyone who actually does that, and most
> > people who have databases that large tend to not want to
> >  be guinea pigs. (:
>
> Right. The reason I ask, though, is that we need the nice
> real-time and distributed characteristics of mnesia, and our
> current use of MySQL is proving to be a performance
> bottleneck and a constant source of operational problems. I
> am just trying to decide whether the time is better spent
> redesigning our MySQL database and replication scheme, or
> pushing mnesia's limits (we use mnesia anyway for other,
> smaller tables, so having a single tool would be nice).
>
> Ulf, could you elaborate on what you mean by "favourable
> access patterns"?
>
> Is there actually anyone out there using mnesia in the 100GB
> range?
>
> Regards,
>
> Dominic Williams
> http://dominicwilliams.net
>
> ----
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071211/b14fb866/attachment.htm>