[erlang-questions] Mnesia Query Performance

[Rob Harrop]

Hi Matt,

Thanks for taking the time to look at this. 

I should've said that we'd reached a similar conclusion that deleting our records one-by-one was significantly faster than doing them all in one transaction.

Is this the recommended solution? It's a real shame to lose the transaction protection here.

Regards,

Rob



On 10 Mar 2011, at 15:25, "Evans, Matthew" <mevans@REDACTED> wrote:

> Hi,
> 
> I changed your read function to look like this instead:
> 
> read([]) ->
>    ok;
> read([Key|Keys]) ->
>    {atomic, _} =
>        tx(fun() ->
>                   mnesia:read(test_record, Key)
>           end),
>    read(Keys).
> 
> Before the change my test mirrored yours:
> 
> [{ordered_set,500,17},
> {ordered_set,1000,43},
> {ordered_set,2000,139},
> {ordered_set,4000,492},
> {ordered_set,8000,1882},
> {ordered_set,16000,7156},
> {set,500,14},
> {set,1000,49},
> {set,2000,140},
> {set,4000,493},
> {set,8000,1846},
> {set,16000,7123}]
> 
> 
> After the change it looks like:
> 
> [{ordered_set,500,28},
> {ordered_set,1000,41},
> {ordered_set,2000,82},
> {ordered_set,4000,166},
> {ordered_set,8000,335},
> {ordered_set,16000,689},
> {set,500,20},
> {set,1000,41},
> {set,2000,84},
> {set,4000,167},
> {set,8000,337},
> {set,16000,701}]
> 
> I think that handling the whole "large" list within a single transaction is the cause of the slowdown. Not sure why.
> 
> Matt
> 
> 
> -----Original Message-----
> From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Rob Harrop
> Sent: Thursday, March 10, 2011 10:03 AM
> To: erlang-questions@REDACTED
> Subject: [erlang-questions] Mnesia Query Performance
> 
> Hi,
> 
> I've been doing a bit of tuning on our use of mnesia in RabbitMQ and I'm 
> a bit puzzled by the fact that read performance in transactions appears 
> to be quadratic (or worse).
> 
> I ran a test in which I do only reads (no writes, no matching) on both 
> set and ordered_set tables. From the results, it is clear that 
> transaction execution time does not increase linearly:
> 
> [{ordered_set,500,11},
>  {ordered_set,1000,34},
>  {ordered_set,2000,120},
>  {ordered_set,4000,418},
>  {ordered_set,8000,1800},
>  {ordered_set,16000,9981},
>  {set,500,11},
>  {set,1000,35},
>  {set,2000,115},
>  {set,4000,406},
>  {set,8000,1808},
>  {set,16000,9804}]
> 
> Here the second slot in each tuple is the number of reads and the third 
> slot is tx execution time in milliseconds.
> 
> I have attached the test code.
> 
> Am I doing something horribly wrong, or is this expected behaviour? If 
> this is expected behaviour are there are known/accepted workarounds?
> 
> Kind Regards,
> 
> Rob Harrop
> RabbitMQ
> 
> ________________________________________________________________
> erlang-questions (at) erlang.org mailing list.
> See http://www.erlang.org/faq.html
> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
>