[erlang-questions] mnesia storage types

[Phillips, Christopher]

  I'll defer toŠwell, any and everyone on the current state of this, but I
know Ulf Wiger answered this a while back on this mailing list (I had the
very same question and so did a fair bit of searching). His answer to
whether 4 GB was the max size -


If you keep data in ram_copies or disc_copies, and run a 64-bit VM, tables
can be larger than that. How much data you can have in practice will e.g.
depend on your tolerance for long startup times (disc_copies must be
loaded into RAM at startup).

Disc_only_copies are in fact limited to 2 GB, and you _really_ don't want
to exceed that, as mnesia doesn't have a nice way of handling errors at
that level - the result will likely be inconsistencies in the database.
And even if you fragment your disc_only_copies tables, Dets repair can
result in unacceptably long restart times. You need to measure and
consider your requirements.





On 3/28/14, 7:00 AM, "erlang-questions-request@REDACTED"
<erlang-questions-request@REDACTED> wrote:

>
>Date: Thu, 27 Mar 2014 13:23:40 -0400
>From: Noah Schwartz <noah.schwartz1@REDACTED>
>To: erlang-questions@REDACTED
>Subject: [erlang-questions] mnesia storage types
>Message-ID:
>	<CAMEFtU-u4OHQAR47XOhZWMJVny0ghHQ646D14YkpSROBbWAbyQ@REDACTED>
>Content-Type: text/plain; charset="iso-8859-1"
>
>Hello,
>
>We use mnesia fairly extensively in our application and are interested in
>keeping track of table growth. We have one very big table thats already
>sharded and want to know well ahead of time if/when others need to be
>sharded.
>
>We started with a fairly simple approach of calling mnesia:table_info(Tab,
>memory) on each table on a regular interval. It seems that for
>disc_only_tables mnesia defers to dets:info/2 and for disc_copies or
>ram_copies mnesia defers to ets:info/2. dets returns the size in bytes and
>ets returns the size in words. We noticed that the words returned by ets
>didnt match up with the file sizes on disk when converted to bytes. This
>got us wondering a little more about the different file types -- DAT for
>dets and DCD/DCL for disc_copies. We also noticed that if you called
>dets:info(Tab, memory) on a disc_copies table it returned undefined. I was
>always under the impression that a disc_copies table used ets and dets
>under the hood but, after looking at the mnesia code that doesn't seem
>like
>the case.
>
>So my first question is, do tables with a storage type of disc_copies use
>dets? If so, are they limited by the same 4GB limit that dets is? If so,
>how can I properly measure the size on disk of a disc_copies table.
>
>Thanks in advance
>
>--
>Noah