[erlang-questions] mnesia_leveldb + prefix keys

[Dániel Szoboszlay]

Until the optimisations mentioned by Mikael are implemented, you can reach
good performance by writing the loop manually:

loop(mnesia:next(table, #key{jid={<<"user1">>, <<"localhost">>}, timestamp
= {1,2,3}}), []).

loop(K = #key{jid={<<"user1">>, <<"localhost">>}}, Acc) ->
  loop(mnesia:next(table, K), mnesia:read(table, K) ++ Acc);
loop(_, Acc) -> Acc.

On Tue, 22 Jan 2019 at 13:41 Mikael Pettersson <mikpelinux@REDACTED> wrote:

> On Tue, Jan 22, 2019 at 11:32 AM Matthias Rieber <ml-erlang@REDACTED>
> wrote:
> >
> > Hello,
> >
> > I've a question regarding the prefix keys in mnesia_eleveldb. I have a
> > record like this:
> >
> > -record(key, {
> >   jid       :: { binary(), binary() },
> >   timestamp :: erlang:timestamp()
> > }).
> >
> > -record(table, {
> >   key = key{},
> >   payload :: term()
> > }).
> >
> >
> > As far as I understand these queries are fast:
> >
> > mnesia:match_object(
> >   #table{key=#key{jid={<<"user1">>, <<"localhost">>},
> >                   timestamp = '_'},
> >          payload='_'})
> >
> > Since the keys are ordered I expected that query to be fast:
> >
> > mnesia:select(table, ets:fun2ms(
> >  fun(#table{key=#key{jid={<<"user1">>, <<"localhost">>},
> >                      timestamp = Timstamp},
> >             payload=P}) when Timstamp > {1,2,3} -> P
> >  end)
> >
> > But performance tests show that this will read all records (probably only
> > the ones with the matching jid). Is it possible to select this in an
> > efficient manner?
>
> Your observations are correct.  The second case is a "range select"
> which LevelDB can support efficiently, but mnesia_eleveldb doesn't
> implement that optimization yet.  I've had a ticket to implement this
> for a while now, but the initial prototype didn't work and I haven't
> had time to up-prioritize it yet.
>
> Meanwhile we've implemented another optimization in mnesia_eleveldb:
> we no longer store the tags of records in mnesia_eleveldb tables as
> those tags are invariant and redundant.  This reduces I/O and CPU
> usage.
>
> /Mikael
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20190122/644ee41e/attachment.htm>