mnesia power index -- What about fragmentation?

[Valentin Micic]

Speaking of indexes...
Am I wrong in saying that indexing for fragmented tables can be improoved? I
did a brief test once (on R9) and notice that fragment number allocation for
index table is the same as for key -- in other words, if KEY says that
record goes to fragment number 5, the index is also going to be stored in
the index fragment number 5. This causes a linear performance degradation --
more fragments one has, worse index-based lookup gets. Wouldn't it be better
if index fragments are calculated based index value? Hmmm... this is too
obvious. It must be that was doing something wrong? Was I?

V.

----- Original Message -----
From: "Ulf Wiger (AL/EAB)" <ulf.wiger@REDACTED>
To: <erlang-questions@REDACTED>
Sent: Friday, February 18, 2005 2:02 PM
Subject: RE: mnesia power index



Ulf Wiger (AL/EAB) wrote:

>   ** Oops! The following functions don't work right!
>
> 40> mnesia:dirty_index_first(test4,{1,name}).
> {{{"armstrong","joe"},2},[]}
> 41> mnesia:dirty_index_next(test4,{1,name},{{"armstrong","joe"},2}).
> {{{"armstrong","joe"},4},[]}
>
>   ** They do work with old-style indexes, and should work with
>   ** unordered indexes. I will fix this.

Now fixed. If anyone wants a patch, let me know.

Alternating next and prev on a bag table yields very
strange results - so using these functions on an unordered
index (at least when stepping back and forth) can sometimes
cause surprises. I will not attempt to fix this.

139> mnesia:dirty_index_first(test4,{1,name}).
{{{"armstrong","joe"},4},
 [{test4,2,"Joe","Armstrong","the quick brown fox"},
  {test4,4,"joe","armstrong","jumps over the lazy dog"}]}
140> mnesia:dirty_index_last(test4,{1,name}).
{{{"wiger","ulf"},3},
 [{test4,1,"Ulf","Wiger","The Quick Brown Fox"},
  {test4,3,"ulf","wiger","JUMPS OVER THE LAZY DOG"}]}
141> mnesia:dirty_index_next(test4,{1,name},element(1,v(139))).
{{{"wiger","ulf"},3},
 [{test4,1,"Ulf","Wiger","The Quick Brown Fox"},
  {test4,3,"ulf","wiger","JUMPS OVER THE LAZY DOG"}]}
142> mnesia:dirty_index_next(test4,{1,name},element(1,v(141))).
'$end_of_table'
143> mnesia:dirty_index_prev(test4,{1,name},element(1,v(140))).
{{{"armstrong","joe"},4},
 [{test4,2,"Joe","Armstrong","the quick brown fox"},
  {test4,4,"joe","armstrong","jumps over the lazy dog"}]}
144> mnesia:dirty_index_prev(test4,{1,name},element(1,v(143))).
'$end_of_table'


> - The index callbacks execute in the 'commit' phase, so there's
>    no possibility to abort (this will cause mnesia to dump core).
>    They simply have to work. Oh, and keep them side-effect free.

I decided to address this with a function:

mnesia_index:test_index_fun(Tab, Index, Object)

(The function returns whatever the indexing callback returns,
or exits if the callback exits.)

and also the sometimes more convenient function:

mnesia_index:test_indexes(Tab, Object) -> ok

(verifies that each index function returns a list
of values for the given object.)

By calling this within the transaction, you should get
a transaction abort rather than a nasty crash during
commit. I thought I'd build this into 'rdbms' if the
functionality actually becomes incorporated into mnesia.
Then, it will be transparent to users of 'rdbms'.

BTW, the functions work for all index types, not just callbacks.

/Uffe