[erlang-questions] Mnesia fragmented tables: mnesia:dirty_next and mnesia:dirty_first
Thu Jun 10 16:56:15 CEST 2010
By that I mean how is a QLC cursor implemented?
The reason for using a disc only copies, fragmented table, is because the number of records could be rather large (10's of millions).
From: [mailto:] On Behalf Of Evans, Matthew
Sent: Thursday, June 10, 2010 10:33 AM
To: Igor Ribeiro Sucupira
Subject: RE: [erlang-questions] Mnesia fragmented tables: mnesia:dirty_next and mnesia:dirty_first
Thanks for the information.
I have another question about QLC. How is this constructed internally?
In my case the table is disc_only_copies, does QLC read X records into RAM? Or the whole table?
From: Igor Ribeiro Sucupira [mailto:]
Sent: Wednesday, June 09, 2010 8:14 PM
To: Evans, Matthew
Subject: Re: [erlang-questions] Mnesia fragmented tables: mnesia:dirty_next and mnesia:dirty_first
On Wed, Jun 9, 2010 at 7:59 PM, Evans, Matthew <> wrote:
> I have a fragmented database where I want to do an mnesia:dirty_first(Tab), and mnesia:dirty_next(Tab,Key).
> These don't appear to be supported.
The mnesia_frag module can probably handle that, if you call
mnesia:activity/4 like this:
F = mnesia:activity(sync_dirty, fun mnesia:first/1, [my_table], mnesia_frag).
mnesia:activity(sync_dirty, fun mnesia:next/2, [my_table, F], mnesia_frag).
But maybe it will be more efficient to use a cursor, after creating it
inside of a dirty operation:
Cursor = mnesia:activity(sync_dirty, fun qlc:cursor/1, [qlc:q([R || R
<- mnesia:table(my_table)])], mnesia_frag).
However, if you need to be able to handle errors (e.g. an unavailable
fragment) smoothly (retrieve all the data that is available), you
should actually do something like what you are already doing. Being
that the case, I recommend that you read this post:
> I have got around this by:
> 1) Using mnesia:table_info/2 to get the frag hash state.
> 2) Create a list of all my fragments (as atoms), in a specific order.
> The above data is saved in a state record.
> Then whenever I get a get_next I call mnesia_frag_hash:key_to_frag_number/2, and from the list of fragments I saved in 2, above, I can get the actual fragment to search in.
> I can then call mnesia:dirty_next(Fragment,Key). If I get '$end_of_table' I can move to the next fragment in the list of fragments and so on until the list of fragments is exhausted.
> Other than doing something similar with mnesia:dirty_slot/2 is there any other way to do a get_next traversal of a fragmented table?
"The secret of joy in work is contained in one word - excellence. To
know how to do something well is to enjoy it." - Pearl S. Buck.
erlang-questions (at) erlang.org mailing list.
To unsubscribe; mailto:
More information about the erlang-questions