[erlang-questions] Residual indexes with mnesia

[Christopher Faulet]

Salazard Rémy a écrit :
> I have a problem with a mnesia table of type 'bag' with indexed fields.
> 
> When I'm using mnesia:dirty_delete_object/2, sometimes the indexes are
> removed and sometimes not :(
[...]
> 
> Do I make a mistake or is it a bug?

Hi,

IMHO, this is a bug and a leak.

Firstly, I will explain shortly the format of indexes tables. I hope
there are no mistakes in my explanation.

For mnesia table, built with the record #mytbl{one, two, three}, the
element identified by 'one' is the implicit key.
We can add extra indexes on this table (For example, we can add an index
on the element identified by 'two'). When we do that, a table per index
is created. It will contain fields with the following syntax: {index, key}

We are going to use the following example to illustrate our purpose:

We insert

  {mytbl, key_1, index_1, data1}  (1)
  {mytbl, key_1, index_2, data2}  (2)
  {mytbl, key_1, index_2, data3}  (3)
  {mytbl, key_2, index_1, data4}  (4)

in the mnesia table named 'mytbl', defined from the record #mytbl{one,
two, three} and with an index on second element.
The associated indexes table will be:

  {index_1, key_1} -> associated with record  (1)
  {index_2, key_1} -> associated with records (2) and (3)
  {index_1, key_2} -> associated with record  (4)

Now, when we remove a record, we must remove these associated indexes
too.  But, with a table of type 'bag', we can have multiple records with
the same pair {index, key} (in our example, {index_2, key_1} is shared
by records (2) and (3)). So, to remove an index, we must be sure that
there is no other records that use the same pair {index, key}.

In our example, if we remove the record (1) or (4), we also implicitly
remove the associated index, i.e, respectively, {index_1, key_1} or
{index_1, key_2}. But, if we remove the record (2), the record (3) still
uses the pair {index_2, key_1}. So this index must not be removed
immediately. It will be removed only with the record (3).

From this point, we can try to explain the suspected bug.

In the file mnesia_index.erl which deals with indexes, we can find the
following code:

[...]
1 |del_object_bag(Tab, Key, Obj, Pos, Ixt, undefined) ->
2 |   Old = mnesia_lib:db_get(Tab, Key),
3 |   del_object_bag(Tab, Key, Obj, Pos, Ixt, Old);
4 |%% If Tab type is bag we need remove index identifier if Tab
5 |%% contains less than 2 elements.
6 |del_object_bag(_Tab, Key, Obj, Pos, Ixt, Old) when length(Old) < 2 ->
7 |   del_ixes(Ixt, [Obj], Pos, Key);
8 |del_object_bag(_Tab, _Key, _Obj, _Pos, _Ixt, _Old) -> ok.
[...]

Currently, to remove an index, a request is performed in the mnesia
table with the record key (line 2). If there are more than one result
(guard on line 6), we suppose that there is another record(s) that
shares the same index and we do not remove it (line 8). Else, we try to
remove it (line 7).

Come back on our example to understand what it happens.

If we remove the record (4), the associated index (identified by
{index_1, key_2}) is removed, because there is no other records with the
same key. all is ok.

Nevertheless, if we remove the record (1), the associated index
(identified by {index_1, key_1}) is not removed, because there is other
records with the same key (records (2) and (3)). Once the record (1)
removed, there is no more opportunity to remove the pair {index_1, key_1}.

IMHO, there is a bug here. We must refine the selection. We must only
keep records with the same key *AND* the same index. the line 2 returns
all records with the key of the one we want to remove. But, it seems
likely there are records with different indexes.

To fix this bug, we can replace the line 2 by these two lines:

IxKey = element(Pos, Obj),
Old = [X || X <-  mnesia_lib:db_get(Tab, Key), element(Pos, X) =:= IxKey],


It slows down the deletion of indexes and, maybe, it exists faster
solutions (perhaps with some mnesia internal facilities). But this hack
works. I made a patch that seems to work with the releases from R10B9 to
R11B4.

Regards,
-- 
Christopher Faulet
http://www.capflam.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mnesia_index.patch
Type: text/x-patch
Size: 582 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20070329/f6a22ab1/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 252 bytes
Desc: OpenPGP digital signature
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20070329/f6a22ab1/attachment-0001.bin>