[erlang-questions] how: Purging old records from Mnesia table

[Gleb Peregud]

On 3/6/08, Bengt Kleberg <bengt.kleberg@REDACTED> wrote:
> Greetings,
>
> While only a small part of the total answer I would recommend that you
> could look into the module timer. It has apply_after/4 and it does not
> have the unsigned int problem.
>
>
> bengt
>
> On Thu, 2008-03-06 at 12:38 +0100, Gleb Peregud wrote:
> > Hello.
> >
> > I would like to implement some algorithm of purging old data from
> > mnesia table. Expiration date is written inside each record (could be
> > 0 for permanent data).
> >
> > I have few ideas how to implement it:
> > 0) Every few seconds iterate over whole table and purge old records.
> > It is implemented in this way now and it eats about 35% of CPU on
> > every purge cycle for about 10 000 entries.
> > Will index on expiration field speed up selecting? Is it better to use
> > qlc? Will it speed give speed up if i use mnesia:index_match_object()
> > ?
> >
> > 1) Use erlang:send_after() to send messages to purge individual
> > records (with special care to timeouts greater then max value of
> > unsigned int). Will Erlang runtime handle efficiently potentially up
> > to few millions of timers?
> >
> > 2) Use some sort of queue to manage sorted list of records to be
> > purged. Make special process which will receive information about new
> > records, insert into this queue and sleep until first record is to be
> > purged. Probably gb_trees will handle it. Are there better ADS for
> > this task?
> >
> > I'll be greateful for any advices and responses :)
> >
> > Best Regards,
> > --
> > Gleb Peregud
> > http://gleber.pl/
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://www.erlang.org/mailman/listinfo/erlang-questions
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>

After checking source of timer module i can see it is implemented
using ordered_set ets table. This table is used as a queue and
gen_server module is pulling data at necessary moments (it is the same
as my idea in point 3). It seems to be a good choice :)

Hence it is enough to use timer:send_after()

--
Gleb Peregud
http://gleber.pl/

Every minute is to be grasped.
Time waits for nobody.
-- Inscription on a Zen Gong