Mnesia and Oracle

[Valentin Micic]

Well, we're using mnesia with dets stroring there-about 80GB data set.
IMHO, if you want to develop a cache to front RDBMS, what would mnesia do 
for you that ETS wouldn't? Other than processing overhead, of course.
And my main issue would be data-integrity.

V.

----- Original Message ----- 
From: "Ryan Rawson" <ryanobjc@REDACTED>
To: "Valentin Micic" <valentin@REDACTED>
Cc: "Yariv Sadan" <yarivvv@REDACTED>; "Inswitch Solutions" 
<erlang@REDACTED>; <erlang-questions@REDACTED>
Sent: Thursday, August 10, 2006 9:54 AM
Subject: Re: Mnesia and Oracle


> What about my 50 gb data set?
>
> What about my 100 gb data set?
>
> Ultimately I think a major value is using mnesia as a coherent cache
> and some SQL backend as a master storage.
>
> -ryan
>
>
> On 8/10/06, Valentin Micic <valentin@REDACTED> wrote:
>> I do not think combining mnesia and MySQL is a good choice, no matter 
>> what
>> justification one may put forward. If nothing else, it complicates
>> maintenance, confuses people, and may (potentialy) compromise both
>> products... not to mention a data integrity issues intrinsic to such an
>> implementation.
>> Mnesia's dets tables work reasonably well, even with huge data sets. The
>> problem(s) starts when you start deleting records -- free list grows to 
>> the
>> point that (IMHO) takes quite a long time to write it to disk. This 
>> becomes
>> more obvious when one uses a lots of fragments. Thus, even if you shut 
>> the
>> database regularly, it might corrupt few fragments. My guess: mensia
>> controller does not give enough time to all dets processes (one per
>> fragment) to flush their respective free lists to dist.
>>
>> A question for Erlang/OTP team: how can one prevent this from happening?
>>
>> I'm looking more (and more) to Berkeley DB for storage. Anybody, how does 
>> it
>> compare to dets?
>>
>> Valentin.
>>
>>
>> ----- Original Message -----
>> From: "Yariv Sadan" <yarivvv@REDACTED>
>> To: "Inswitch Solutions" <erlang@REDACTED>
>> Cc: <erlang-questions@REDACTED>
>> Sent: Tuesday, August 08, 2006 11:51 PM
>> Subject: Re: Mnesia and Oracle
>>
>>
>> > Hi Eduardo,
>> >
>> > I haven't had production experience with these databases, but there
>> > are a couple of things I found by research that are keeping me from
>> > using Mnesia exclusively in my application:
>> >
>> > - Mnesia disc storage, based on dets, has a couple of drawbacks when
>> > handling very large (many gigs) datasets: potentially long repair
>> > times and memory consumption that grows with data fragmentation.
>> > - QLC, the query engine for Mnesia, doesn't currently optimize joins.
>> > If your queries involve joining big tables, they can take a long time
>> > to execute.
>> >
>> > The join optimizations are planned for a future R11 OTP release, but
>> > there are no plans to change dets AFAIK.
>> >
>> > Depending on your application, these issues may not be a big problem.
>> > For the application I'm building, I'm planning on using both MySQL and
>> > Mnesia, where MySQL will be used for storing high-volume data and
>> > Mnesia for "live" session data.
>> >
>> > Hope this helps!
>> >
>> > Regards,
>> > Yariv
>> >
>> > On 8/8/06, Inswitch Solutions <erlang@REDACTED> wrote:
>> >>
>> >>
>> >>
>> >> Hi,
>> >>
>> >> I'm already working with Oracle and Mnesia, and I'd  like to hear
>> >> experiences of the Erlang community about these  databases.
>> >> When deciding over Oracle or Mnesia database for an  Erlang, or non
>> >> Erlang, based real-time system which factors are in favour  in one 
>> >> over
>> >> the other (performance...?) ?.
>> >>
>> >>
>> >> thanks, Eduardo
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> Prepaid Expertise -        Programmable Switches
>> >> Powered by Ericsson Licensed Technology
>> >> Eng.        Eduardo Figoli - Development Center - IN Switch Solutions
>> >> Inc.
>> >> Headquarters - Miami-U.S.A. Tel: 1305-3578076 Fax:        1305-7686260
>> >> Development Center - Montevideo - Uruguay Tel/Fax:        5982-7104457
>> >> e-mail: eduardo@REDACTED
>> >>
>> >>
>>
>>
>