[erlang-questions] Mnesia Query Performance

[Dan Gudmundsson]

What happens if you grab a table lock the first thing in the transaction?

/Dan

On Thu, Mar 10, 2011 at 5:55 PM, Rob Harrop <rob@REDACTED> wrote:
> Hi Matt,
>
> Thanks for taking the time to look at this.
>
> I should've said that we'd reached a similar conclusion that deleting our records one-by-one was significantly faster than doing them all in one transaction.
>
> Is this the recommended solution? It's a real shame to lose the transaction protection here.
>
> Regards,
>
> Rob
>
>
>
> On 10 Mar 2011, at 15:25, "Evans, Matthew" <mevans@REDACTED> wrote:
>
>> Hi,
>>
>> I changed your read function to look like this instead:
>>
>> read([]) ->
>>    ok;
>> read([Key|Keys]) ->
>>    {atomic, _} =
>>        tx(fun() ->
>>                   mnesia:read(test_record, Key)
>>           end),
>>    read(Keys).
>>
>> Before the change my test mirrored yours:
>>
>> [{ordered_set,500,17},
>> {ordered_set,1000,43},
>> {ordered_set,2000,139},
>> {ordered_set,4000,492},
>> {ordered_set,8000,1882},
>> {ordered_set,16000,7156},
>> {set,500,14},
>> {set,1000,49},
>> {set,2000,140},
>> {set,4000,493},
>> {set,8000,1846},
>> {set,16000,7123}]
>>
>>
>> After the change it looks like:
>>
>> [{ordered_set,500,28},
>> {ordered_set,1000,41},
>> {ordered_set,2000,82},
>> {ordered_set,4000,166},
>> {ordered_set,8000,335},
>> {ordered_set,16000,689},
>> {set,500,20},
>> {set,1000,41},
>> {set,2000,84},
>> {set,4000,167},
>> {set,8000,337},
>> {set,16000,701}]
>>
>> I think that handling the whole "large" list within a single transaction is the cause of the slowdown. Not sure why.
>>
>> Matt
>>
>>
>> -----Original Message-----
>> From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Rob Harrop
>> Sent: Thursday, March 10, 2011 10:03 AM
>> To: erlang-questions@REDACTED
>> Subject: [erlang-questions] Mnesia Query Performance
>>
>> Hi,
>>
>> I've been doing a bit of tuning on our use of mnesia in RabbitMQ and I'm
>> a bit puzzled by the fact that read performance in transactions appears
>> to be quadratic (or worse).
>>
>> I ran a test in which I do only reads (no writes, no matching) on both
>> set and ordered_set tables. From the results, it is clear that
>> transaction execution time does not increase linearly:
>>
>> [{ordered_set,500,11},
>>  {ordered_set,1000,34},
>>  {ordered_set,2000,120},
>>  {ordered_set,4000,418},
>>  {ordered_set,8000,1800},
>>  {ordered_set,16000,9981},
>>  {set,500,11},
>>  {set,1000,35},
>>  {set,2000,115},
>>  {set,4000,406},
>>  {set,8000,1808},
>>  {set,16000,9804}]
>>
>> Here the second slot in each tuple is the number of reads and the third
>> slot is tx execution time in milliseconds.
>>
>> I have attached the test code.
>>
>> Am I doing something horribly wrong, or is this expected behaviour? If
>> this is expected behaviour are there are known/accepted workarounds?
>>
>> Kind Regards,
>>
>> Rob Harrop
>> RabbitMQ
>>
>> ________________________________________________________________
>> erlang-questions (at) erlang.org mailing list.
>> See http://www.erlang.org/faq.html
>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
>>
>
> ________________________________________________________________
> erlang-questions (at) erlang.org mailing list.
> See http://www.erlang.org/faq.html
> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
>
>