[erlang-questions] Extremely poor Mnesia performance

[Dana.RUBINO@REDACTED]

Hi Edwin,

Thanks and good idea - I am actually investigating that route right this moment!

I think I could definitely do it, the problem being that after an initial bootstrap of the Mnesia tables I will be in a situation where I will be receiving live/real-time updates to the data. These unfortunately will need to be reflected in mnesia so that the queries performed against it are fully up-to-date whilst still enabling real-time response rates to the queries themselves.

A bit of a mouthful I know!

Many Thanks,
Dan

________________________________
From: emofine@REDACTED [mailto:emofine@REDACTED] On Behalf Of Edwin Fine
Sent: 30 September 2008 17:48
To: RUBINO, Dana, GBM
Cc: taavi@REDACTED; erlang-questions@REDACTED
Subject: Re: [erlang-questions] Extremely poor Mnesia performance

Is it possible to denormalize the data so that you don't have so many tables? Mnesia can behave as a hierarchical database because you can store arbitrarily complex terms, including what amounts to trees (lists of tuples of lists of.....). If your queries are likely to remain the same over time and you don't need ad-hoc queries, could you consider this approach? The problem with any hierarchy, of course, is that if you get it wrong it can be hell on wheels to fix.

On Tue, Sep 30, 2008 at 12:23 PM, <Dana.RUBINO@REDACTED<mailto:Dana.RUBINO@REDACTED>> wrote:
Thanks Taavi,

That wasn't the answer I was looking for! :-D

Oh dear, well that leaves my plan for using Mnesia in tatters.

I actually need an in memory db which at least has the core subset of RDBMS features. So I am essentially just looking for a small in memory DB for real-time access.

Has anyone here had any experience using something like KDB with Erlang?

Perhaps I could switch to using KDB as an alternative?

Many Thanks,
Dan

-----Original Message-----
From: Taavi Talvik [mailto:taavi@REDACTED<mailto:taavi@REDACTED>]
Sent: 30 September 2008 17:19
To: RUBINO, Dana, GBM
Cc: erlang-questions@REDACTED<mailto:erlang-questions@REDACTED>
Subject: Re: [erlang-questions] Extremely poor Mnesia performance


On Sep 30, 2008, at 5:58 PM, <Dana.RUBINO@REDACTED<mailto:Dana.RUBINO@REDACTED>> wrote:

>
> Hey all,
>
> I am doing some benchmarking with Mnesia at the moment and am pretty
> disappointed to say the least.
>
> I'm hoping I am doing something wrong.
>
> Running a 5 table join below: (two of the tables have ~50k rows the
> rest a couple of hundred)

Probably you are not doing anything wrong. From mnesia introduction (http://www.erlang.org/doc/apps/mnesia/
Mnesia_chap1.html#1),
it design goals are:
===
In telecommunications applications there are different needs from the features provided by traditional DBMSs. The applications now implemented in the Erlang language need a mixture of a broad range of features, which generally are not satisfied by traditional DBMSs.
Mnesia is designed with requirements like the following in mind:

       * Fast real-time key/value lookup
       * Complicated non real-time queries mainly for operation and maintenance
       * Distributed data due to distributed applications
       * High fault tolerance
       * Dynamic re-configuration
       * Complex objects
===

Mnesia is not designed as replacement for relational database. It is optimised for fast key-value lookup and distribution.

With qlc:info(Qh) you can get more information, how query is actually handled.

best regards,
taavi

***********************************************************************************
The Royal Bank of Scotland plc. Registered in Scotland No 90312. Registered Office: 36 St Andrew Square, Edinburgh EH2 2YB.
Authorised and regulated by the Financial Services Authority

This e-mail message is confidential and for use by the
addressee only. If the message is received by anyone other
than the addressee, please return the message to the sender
by replying to it and then delete the message from your
computer. Internet e-mails are not necessarily secure. The
Royal Bank of Scotland plc does not accept responsibility for
changes made to this message after it was sent.

Whilst all reasonable care has been taken to avoid the
transmission of viruses, it is the responsibility of the recipient to
ensure that the onward transmission, opening or use of this
message and any attachments will not adversely affect its
systems or data. No responsibility is accepted by The
Royal Bank of Scotland plc in this regard and the recipient should carry
out such virus and other checks as it considers appropriate.
Visit our websites at:
www.rbs.com<http://www.rbs.com>
www.rbs.com/gbm<http://www.rbs.com/gbm>
www.rbsgc.com<http://www.rbsgc.com>
***********************************************************************************

_______________________________________________
erlang-questions mailing list
erlang-questions@REDACTED<mailto:erlang-questions@REDACTED>
http://www.erlang.org/mailman/listinfo/erlang-questions



***********************************************************************************
The Royal Bank of Scotland plc. Registered in Scotland No 90312. Registered Office: 36 St Andrew Square, Edinburgh EH2 2YB. 
Authorised and regulated by the Financial Services Authority 
 
This e-mail message is confidential and for use by the 
addressee only. If the message is received by anyone other 
than the addressee, please return the message to the sender 
by replying to it and then delete the message from your 
computer. Internet e-mails are not necessarily secure. The 
Royal Bank of Scotland plc does not accept responsibility for 
changes made to this message after it was sent. 

Whilst all reasonable care has been taken to avoid the 
transmission of viruses, it is the responsibility of the recipient to 
ensure that the onward transmission, opening or use of this 
message and any attachments will not adversely affect its 
systems or data. No responsibility is accepted by The 
Royal Bank of Scotland plc in this regard and the recipient should carry 
out such virus and other checks as it considers appropriate. 
Visit our websites at: 
www.rbs.com
www.rbs.com/gbm
www.rbsgc.com
***********************************************************************************
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20080930/b22c9e8f/attachment.htm>