[erlang-patches] New index type in mnesia (new feature)

[Ulf Wiger]

The mnesia extension presented at the EUC 2012,

  http://www.erlang-factory.com/upload/presentations/601/EUC2012-009.pdf

actually contains a solutions for new index types, primarily ordered indexes.

It is still evolving and will soon appear in a public repository.

The improvement in terms of indexing is that you can specify an index as

  {Pos, ordered | bag}

where ram_copies and disc_copies support both types, but disc_only_copies supports only bag.

When combined with backend plugins, the plugin gets to define which kinds of indexes it supports, as well as a callback function for generating index values. The user interface will also be extended to allow user-defined index functions, so that derived indexes can be created. A slight problem is that indexes are tied to attribute positions, and derived indexes really shouldn't be.

(Strictly speaking, a backend plugin can specify other index types, but mnesia will only allow an index type on a table if it's supported by all types of backends used by that table).

The addition was necessary since leveldb supports ordered_set semantics, but not bag semantics. We have since then implemented bag semantics on top of leveldb, but mainly for compatibility - not for performance.

Ordered indexes have good performance characteristics. The representation is {{IxValue, Key}}, and relies on the facts that (1) insert and delete are both very efficient, and (2) a match on {{IxVal, '_'}} in an ordered set is an efficient operation.

BR,
Ulf W

On 28 Jun 2013, at 12:22, Fredrik wrote:

> On 06/19/2013 06:51 PM, Aleksander Nycz wrote:
>> 
>> Hello, 
>> 
>> Mnesia gives possibility to create table indexes, when 
>> the user wants to frequently use some other field 
>> than the key field to look up records. 
>> 
>> Current index solution in mnesia uses ets table (type bag or duplicated_bag) to maintain mapping: 
>> Indexed field value -> Primary key value. 
>> 

Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc.
http://feuerlabs.com