Mnesia Speed Optimization

Younès HAFRI yhafri@REDACTED
Fri Jan 16 14:35:38 CET 2004


Hi,
I have a great optimization problem with Mnesia.
My database is not replicated and its works in the local node.
My Box configuration : PC, Processor  999Mhz, RAM 512Mo, Suse Linux 8.2

I want to write records if its not already exists in the database.

For example:
insert_if_not_exists(aaaa) -> ok
insert_if_not_exists(bbbb) -> ok
insert_if_not_exists(cccc) -> ok
insert_if_not_exists(aaaa) -> no insertion (already exists)
insert_if_not_exists(dddd) -> ok
....

I've a huge list of records, 1 million exactlly. And this is my "insert 
if not exists" function:
---------------------------------------------------------------------------- 

insert_if_not_exists(ListOf Records) ->
 lists:foreach(fun (Record) ->
                       case mnesia:dirty_read({table_index, Record}) of
                           []          -> 
mnesia:dirty_write(#table_index{field1= Record});
                           [Exists] -> ok
                       end
               end, ListOf Records).
---------------------------------------------------------------------------

This is some results:
insertion of 10000      records take 0.02   seconds in average
insertion of 10000      records take 0.19   seconds in average
insertion of 100000    records take 2.27   seconds in average
insertion of 1 million records take 23.2   seconds in average

The insertion time of 1 million records (23.2 sec) is linear but is too 
much for my.

Could you help me to speed up this function please?
Is there a method (parameters to set) to tune Mnesia.

I need help really!!!
Thank you in advance

Best Regards
Younès




More information about the erlang-questions mailing list