Mnesia disk performance (was RE: multi-attribute mnesia indexes?)

[Ulf Wiger]

On Tue, 2 Jan 2001, Shawn Pearce wrote:

>Andi Kleen <ak@REDACTED> scrawled:
>> On Tue, Jan 02, 2001 at 03:49:16PM -0500, Shawn Pearce wrote:
>> > Oracle does this with their database and it is a big performance
>> > booster.  The other thing they do is allow a table to be striped
>> > across multiple disks by making a table exist in multiple file system
>> > files at once.  (They stripe disk allocations across the files.)  This
>> > does help to manage larger tables as well.
>> 
>> Near all modern OS can do that themselves using volume managers and software
>> RAID -- it would probably be a waste of time to implement it in Mnesia too.
>
>This is true, and I agree.  However, it does allow Oracle to easily
>handle >2GB datafiles on Unixes that cannot deal with it.  It also
>lets you stick to 32 bit file offsets by adding a file number
>``prefix''.

I must confess that I haven't kept up with OS vs DBMS design in the
past few years, but it used to be commonly accepted that you simply
couldn't build a really fast DBMS on top of the standard file and
memory management provided by the leading operating systems --
certainly not if you wanted similar behaviour across multiple
platforms. I don't know if it's still true...

One DBMS I worked with was Cincom's SUPRA. I remember that you had the
choice upon installing the database whether you wanted it to reside on
a normal file system (good for testing) or on a raw partition (good
for speed). In the case of the raw partition, SUPRA would use its own
file I/O driver.

I agree that the problem is worse for mnesia, since it must deal with
variable size objects. I know that work is ongoing to rewrite dets for
much better performance. One of the biggies is that an Erlang program
shall be able to perform multiple disk operations in one instruction
to the file driver. This should be a big booster for dets.

/Uffe
-- 
Ulf Wiger                                    tfn: +46  8 719 81 95
Senior System Architect                      mob: +46 70 519 81 95
Strategic Product & System Management    ATM Multiservice Networks
Data Backbone & Optical Services Division      Ericsson Telecom AB