[erlang-questions] How to speed up mnesia startup?

[Kyungho Yun]

Thanks for replies.
I just know why it takes so long time to start up mnesia.


It looks fine to work except long time mnesia startup.
But it's getting worse problem because the number of tables are increasing.


Most of tables have same personally.

The maximum size(whichever 2GB or 4GB) limit of dets was main problem to
use single table.
I needed short response from mnesia but I wasn't sure that fragmentation is
right choice.

so I used lots of tables instead of it.

Some of table's size are about 50M Bytes. (about 130k no_objects(same
no_keys))
Some of table's size are about 30K Bytes. (about 100 no_objects(same
no_keys))


Is the fragmentation only way to reduce mnesia startup time?

thanks


On Fri, Apr 13, 2012 at 2:53 PM, Ulf Wiger <ulf@REDACTED> wrote:

>
>
> On 13 Apr 2012, at 00:31, Richard O'Keefe wrote:
>
> >
> > On 12/04/2012, at 9:26 PM, Ulf Wiger wrote:
> >
> >>
> >> Hmm, the first thing that stands out is the inordinate number of tables.
> >>
> >> Out of curiosity, I tried figuring out what is considered a reasonable
> number of tables for other database management systems:
> >>
> >> - Oracle: over 10,000 tables considered insane
> >>  (
> http://asktom.oracle.com/pls/asktom/f?p=100:11:::::P11_QUESTION_ID:26039880025641
> )
> >
> > That page has no claim that *Oracle* cannot handle it.
> > The arguments given are
> > - that it seems extremely unlikely that people could "design,
> >   implement, or maintain" a system with 10,000 distinct tables
> >   "personally".
> …
>
> Fair enough. My point was mainly that 60k tables is well beyond what most
> database management systems are optimized for. The question is obviously
> here what _nmesia_ is good at, but I just wanted to illustrate that if you
> want to build a database with 60k tables, you shouldn't _assume_ that any
> given dbms will handle it well.
>
> As far as mnesia is concerned, there are a number of commonly used
> functions that are reasonable as long as the number of tables is somewhere
> in the hundreds.
>
> Take, for example, mnesia:wait_for_tables/2. Let's assume that our
> application actually needs those 60k tables to be loaded before it starts
> doing things.
>
> wait_for_tables/2 is basically implemented like this (lots of code
> snipped):
>
> wait_for_tables_init(From, Tabs) ->
>    process_flag(trap_exit, true),
>    ...
>    cast({sync_tabs, Tabs, self()}),
>            rec_tabs(Tabs, Tabs, From, Init)
>    end.
>
> rec_tabs([Tab | Tabs], AllTabs, From, Init) ->
>    receive
>        {?SERVER_NAME, {tab_synced, Tab}} ->
>            rec_tabs(Tabs, AllTabs, From, Init);
>        ...
>    end;
> rec_tabs([], _, _, Init) ->
>    unlink(Init),
>    ok.
>
> To begin with, if the objective was to support tens of thousand tables, it
> would be better to provide functions to efficiently query the 'table of
> tables'. Currently, there is no table of tables, but a system_info(tables),
> which provides them all as a list (including 'schema', which you often must
> delete from the list, since you can't, for example, do wait_for tables on
> the schema).
>
> In the code above, mnesia sends the 60k list of tables in a single
> message. It then loops through the list and does selective receive on the
> replies belonging to each. This is a convenient way to do it when the list
> of Tabs is short, but pretty awful for a very large list, since each
> iteration may end up scanning a rather large message queue.
>
> This was just one example. Mnesia was not designed with this sort of thing
> in mind.
>
> BR,
> Ulf W
>
> Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc.
> http://feuerlabs.com
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20120413/17bd6709/attachment.htm>