early warning - new rdbms

[Eranga Udesh]

Hi,

No New RDBMS yet?

Also regarding the per process memory issue in Linux/Unix came up in the
mailing list lately, how can we deal with a large Erlang Mnesia DB, which
may have to hold data, say over 5-6 GB of size (5 million records)? Since
mnesia disk_copies DB keeps all the data in memory, it's going to be a
problem, isn't it?

Regards,
- Eranga



-----Original Message-----
From: Ulf Wiger (AL/EAB) [mailto:ulf.wiger@REDACTED] 
Sent: Friday, February 24, 2006 3:03 PM
To: Eranga Udesh
Subject: RE: early warning - new rdbms


I'm hoping to be able to upload a snapshot 
sometime this week. It will not be fully 
functional, but the integrity checking 
should work, at least.

Regards,
Ulf W 

> -----Original Message-----
> From: Eranga Udesh [mailto:casper2000a@REDACTED] 
> Sent: den 24 februari 2006 04:35
> To: Ulf Wiger (AL/EAB)
> Subject: RE: early warning - new rdbms
> 
> Hi,
> 
> Where's the RDBMS system you talk about? Could I get quick 
> access, since I am designing a system and I prefer to design 
> it on top of a RDBMS system, instead of a local Mnesia DB.
> 
> Pls advice asap.
> 
> Thanks,
> - Eranga
> 
> 
> 
> -----Original Message-----
> From: owner-erlang-questions@REDACTED
> [mailto:owner-erlang-questions@REDACTED] On Behalf Of Ulf 
> Wiger (AL/EAB)
> Sent: Thursday, February 09, 2006 9:15 PM
> To: Chaitanya Chalasani
> Cc: erlang-questions@REDACTED
> Subject: RE: early warning - new rdbms
> 
> 
> Ok, but I've been sidetracked for a few days.
> I'm still doing some cleanups. I will let you all know as 
> soon as I have something.
> 
> (Again, I was unprepared for so many takers.
> I had expected to have to announce it a few times before 
> anyone took the bait.  ;)
> 
> Regards,
> Ulf W 
> 
> > -----Original Message-----
> > From: Chaitanya Chalasani [mailto:chaitanya.chalasani@REDACTED]
> > Sent: den 9 februari 2006 13:44
> > To: Ulf Wiger (AL/EAB)
> > Cc: erlang-questions@REDACTED
> > Subject: Re: early warning - new rdbms
> > 
> > I would like to beta-test as well.
> > 
> > On Wednesday 25 January 2006 14:16, Ulf Wiger (AL/EAB) wrote:
> > > I thought I'd let the cat out of the bag a little...
> > >
> > > If anyone wants to beta-test or help out with some of the more 
> > > advanced problems, let me know.
> > >
> > >
> > > I've come pretty far along with a new version of rdbms. It
> > has several
> > > nice features, and I think it's about to make the transition from 
> > > 'somewhat interesting' to 'actually useful':
> > >
> > > - JIT compilation of verification code. The overhead
> > >   for type and bounds checking is now only a few (~5)
> > >   microseconds per write operation.
> > > - The parameterized indexes that I hacked into mnesia
> > >   before are now part of rdbms. This include ordered
> > >   indexes and fragmented indexes (i.e. hashed on
> > >   index value - so they should scale well.)
> > > - Rdbms will handle fragmented tables transparently
> > >   (And actually handles plain tables with less overhead
> > >   than mnesia_frag does.) The overhead for using the
> > >   rdbms access module (compared to no access module)
> > >   on a plain transaction is in the order of 20
> > >   microseconds on my 1 GHz SunBLADE.
> > > - Rdbms hooks into the transaction handling in such a
> > >   way that it can automatically rebuild the verification
> > >   code as soon as a schema transaction commits.
> > > - A readable file format for schema definitions, trying
> > >   to establish a structured way to create large mnesia
> > >   databases. I've also added a 'group' concept to be
> > >   able to group tables into corresponding subsystems,
> > >   since I thought this might be helpful in large
> > >   systems.
> > >
> > >
> > > I'm planning to release rdbms with OTP R11, since it 
> requires some 
> > > changes to mnesia that (hopefully) will make it into R11. R11 is 
> > > planned for May.
> > >
> > > Some of the (fairly minor) changes to mnesia so far:
> > >
> > > - The access module can hook into the transaction
> > >   flow by providing callbacks for begin_activity()
> > >   and end_activity(). Rdbms uses this for proper
> > >   handling of abort and commit triggers as well as
> > >   loop detection in referential integrity checks.
> > >   It also allows rdbms to detect schema changes.
> > > - An 'on-load' hook allows rdbms to build indexes
> > >   the first time a table is loaded.
> > > - A low-level access API for foreign tables. My
> > >   first foreign table attempt was a 'disk_log'.
> > >   It makes it possible to properly log events
> > >   inside a transaction context. You also get
> > >   replicated logs almost for free, as well as
> > >   (if you want to) fragmented logs. (:
> > >   My next attempt at a foreign table is a read-
> > >   only file system (doesn't have to be read-only,
> > >   but I thought I'd start with that.) Thus
> > >   the experiments with converting regexps to
> > >   the select pattern syntax.
> > >
> > > One interesting experiment might be to define an ISAM table
> > type for
> > > really large ordered sets on disk. Combining it with 
> rdbms, you can
> > > type- specify the data types and then convert to whatever 
> format is 
> > > expected by the ISAM files.
> > >
> > > Some questions to those who might be interested:
> > >
> > > - I'd like to break compatibility with the old
> > >   rdbms in some respects. Is this a problem for
> > >   anyone? (speak now or forever hold your peace)
> > > - Do you have any suggestions or feature requests?
> > > - Do you want to help out?
> > >
> > > Regards,
> > > Uffe
> > 
> > --
> > Chaitanya Chalasani
> > 
> 
> 
>