[erlang-questions] mnesia:write slow down

Janos Hary janos.n.hary@REDACTED
Fri Aug 23 23:51:34 CEST 2013


All,

 

My application uses mnesia for data storage. I saw a very quick performance
degradation as data started to build up.

I wrote some simple test cases and measured the write performance. I
inserted 1.3million records and compared the times needed to insert the
first and last 10000. The best I could reach was 22% time increase, but in
more lifelike situations it reached 73%. I used only one process to avoid
locking problems.

 

Finally I found the possible reason: my record looks like #img{dbid, ...},
where dbid is a continuous counter. If I change dbid to a random number the
insert time for the first and last records are nearly the same (actually it
varies in a range but there is no systematic slow down). This problem seems
to affect fragmented tables even worst. 

Is it a known problem? What shall I do if the continuous dbid is an
important part of my design? Shall I convert it to binary and prepend some
salt?

 

I'm happy to provide my test code or time measurements if it helps finding
the cause, but please note, I'm not concerned about the overall performance,
I just can't release the app if it is slowing down so fast, as the
requirement is to handle many more records.

 

Thanks for your time in advance.

Janos

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20130823/b69339d4/attachment.htm>


More information about the erlang-questions mailing list