Mnesia Speed Optimization
Younès HAFRI
yhafri@REDACTED
Fri Jan 16 14:35:38 CET 2004
Hi,
I have a great optimization problem with Mnesia.
My database is not replicated and its works in the local node.
My Box configuration : PC, Processor 999Mhz, RAM 512Mo, Suse Linux 8.2
I want to write records if its not already exists in the database.
For example:
insert_if_not_exists(aaaa) -> ok
insert_if_not_exists(bbbb) -> ok
insert_if_not_exists(cccc) -> ok
insert_if_not_exists(aaaa) -> no insertion (already exists)
insert_if_not_exists(dddd) -> ok
....
I've a huge list of records, 1 million exactlly. And this is my "insert
if not exists" function:
----------------------------------------------------------------------------
insert_if_not_exists(ListOf Records) ->
lists:foreach(fun (Record) ->
case mnesia:dirty_read({table_index, Record}) of
[] ->
mnesia:dirty_write(#table_index{field1= Record});
[Exists] -> ok
end
end, ListOf Records).
---------------------------------------------------------------------------
This is some results:
insertion of 10000 records take 0.02 seconds in average
insertion of 10000 records take 0.19 seconds in average
insertion of 100000 records take 2.27 seconds in average
insertion of 1 million records take 23.2 seconds in average
The insertion time of 1 million records (23.2 sec) is linear but is too
much for my.
Could you help me to speed up this function please?
Is there a method (parameters to set) to tune Mnesia.
I need help really!!!
Thank you in advance
Best Regards
Younès
More information about the erlang-questions
mailing list