[erlang-questions] Mnesia Query Performance

[Evans, Matthew]

Hi,

I changed your read function to look like this instead:

read([]) ->
    ok;
read([Key|Keys]) ->
    {atomic, _} =
        tx(fun() ->
                   mnesia:read(test_record, Key)
           end),
    read(Keys).

Before the change my test mirrored yours:

[{ordered_set,500,17},
 {ordered_set,1000,43},
 {ordered_set,2000,139},
 {ordered_set,4000,492},
 {ordered_set,8000,1882},
 {ordered_set,16000,7156},
 {set,500,14},
 {set,1000,49},
 {set,2000,140},
 {set,4000,493},
 {set,8000,1846},
 {set,16000,7123}]


After the change it looks like:

[{ordered_set,500,28},
 {ordered_set,1000,41},
 {ordered_set,2000,82},
 {ordered_set,4000,166},
 {ordered_set,8000,335},
 {ordered_set,16000,689},
 {set,500,20},
 {set,1000,41},
 {set,2000,84},
 {set,4000,167},
 {set,8000,337},
 {set,16000,701}]

I think that handling the whole "large" list within a single transaction is the cause of the slowdown. Not sure why.

Matt


-----Original Message-----
From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Rob Harrop
Sent: Thursday, March 10, 2011 10:03 AM
To: erlang-questions@REDACTED
Subject: [erlang-questions] Mnesia Query Performance

Hi,

I've been doing a bit of tuning on our use of mnesia in RabbitMQ and I'm 
a bit puzzled by the fact that read performance in transactions appears 
to be quadratic (or worse).

I ran a test in which I do only reads (no writes, no matching) on both 
set and ordered_set tables. From the results, it is clear that 
transaction execution time does not increase linearly:

[{ordered_set,500,11},
  {ordered_set,1000,34},
  {ordered_set,2000,120},
  {ordered_set,4000,418},
  {ordered_set,8000,1800},
  {ordered_set,16000,9981},
  {set,500,11},
  {set,1000,35},
  {set,2000,115},
  {set,4000,406},
  {set,8000,1808},
  {set,16000,9804}]

Here the second slot in each tuple is the number of reads and the third 
slot is tx execution time in milliseconds.

I have attached the test code.

Am I doing something horribly wrong, or is this expected behaviour? If 
this is expected behaviour are there are known/accepted workarounds?

Kind Regards,

Rob Harrop
RabbitMQ