Mnesia fragmentation

[Ulf Wiger]

On Thu, 15 Aug 2002, martin logan wrote:

>I was thinking that fragmentation would be the solution. Ideally
>each server would write all of its data to its local fragment
>but when queries are performed the table is treated as a whole.
>Perhaps I have missunderstood the purpose of fragmentation. It
>seems to me that without control over the fragments that I could
>incur a network hit on every write.

I have written a small patch to mnesia_frag.erl that will give
you control of the fragments. I've been given indications that
it, or some modified version of it, will eventually be included
in OTP... Perhaps you can give it a spin and offer some feedback?

The idea is to offer a mnesia_frag behaviour, where one can
map (Key -> FragmentNumber) and (FragmentNumber -> TableName).

A short example: I'm working (slowly) on an XML database solution
using fragmented tables. I want all elements in one XML document
to be stored in the same fragment, but all XML documents to be
spread out evenly using a hashing algorithm. The idea is that I
want to control the fragments for one table (xmerldb_obj), and
let the others use the default scheme.

-module(xmerldb_frag_cb).
-behaviour(mnesia_frag).

-export([n_to_frag_name/3,
         key_to_n/3]).


%% Returns name of fragment table
n_to_frag_name(Tab, 1, FH) ->
    Tab;
n_to_frag_name(Tab, N, FH) ->
    list_to_atom(atom_to_list(Tab) ++ "_frag" ++
                 integer_to_list(N)).


%% Returns fragment number
key_to_n(xmerldb_obj, {DocId, Pos}, FH) ->
    %% Keep the whole document together in one fragment.
    key_to_n_hash(DocId, FH);
key_to_n(Tab, Key, FH) ->
    %% this would e.g. be the node types "index".
    %% What is the best fragmentation semantic here?
    %% We start by using the default semantics.
    key_to_n_hash(Key, FH).

key_to_n_hash(Key, FH) ->
    L = mnesia_frag:n_doubles(FH),
    A = erlang:hash(Key, trunc(math:pow(2, L))),
    P = mnesia_frag:next_n_split(FH),
    if
        A < P ->
            erlang:hash(Key, trunc(math:pow(2, L + 1)));
        true ->
            A
    end.


In the standard mnesia_frag_cb (implementing today's behaviour),

key_to_n(Tab, Key, FH) ->
    key_to_n_hash(Key, FH).


I can mail you the code, if you want to test it.

Known flaw: match_object() and select() will always search all
fragments. It would be nice to also have a callback function that
can inspect the pattern and determine whether the search can be
limited to only one or a few fragments.

/Uffe
-- 
Ulf Wiger, Senior Specialist,
   / / /   Architecture & Design of Carrier-Class Software
  / / /    Strategic Product & System Management
 / / /     Ericsson Telecom AB, ATM Multiservice Networks