early warning - new rdbms

Serge Aleynikov <>
Wed Jan 25 23:41:31 CET 2006


I have a Mnesia feature request, though not specifically related to rdbms.

It would be helpful to have some kind of a "Change Number" (CN) in 
Mnesia that gets incremented every time there's a change in the data. 
This way if there's a network loss followed by subsequent restoration, 
it may not be necessary to restart/resynch mnesia nodes if they 
determine that the CN hasn't changed from the last known SCN (that is no 
data changes took place during network outage).

Regards,

Serge


Ulf Wiger (AL/EAB) wrote:
> I thought I'd let the cat out of the bag a little...
> 
> If anyone wants to beta-test or help out with some
> of the more advanced problems, let me know.
> 
> 
> I've come pretty far along with a new version of 
> rdbms. It has several nice features, and I think
> it's about to make the transition from 'somewhat
> interesting' to 'actually useful':
> 
> - JIT compilation of verification code. The overhead
>   for type and bounds checking is now only a few (~5)
>   microseconds per write operation.
> - The parameterized indexes that I hacked into mnesia
>   before are now part of rdbms. This include ordered
>   indexes and fragmented indexes (i.e. hashed on 
>   index value - so they should scale well.)
> - Rdbms will handle fragmented tables transparently
>   (And actually handles plain tables with less overhead
>   than mnesia_frag does.) The overhead for using the
>   rdbms access module (compared to no access module)
>   on a plain transaction is in the order of 20 
>   microseconds on my 1 GHz SunBLADE.
> - Rdbms hooks into the transaction handling in such a
>   way that it can automatically rebuild the verification
>   code as soon as a schema transaction commits.
> - A readable file format for schema definitions, trying
>   to establish a structured way to create large mnesia
>   databases. I've also added a 'group' concept to be
>   able to group tables into corresponding subsystems,
>   since I thought this might be helpful in large 
>   systems.
> 
> 
> I'm planning to release rdbms with OTP R11, since it
> requires some changes to mnesia that (hopefully) will
> make it into R11. R11 is planned for May.
> 
> Some of the (fairly minor) changes to mnesia so far:
> 
> - The access module can hook into the transaction
>   flow by providing callbacks for begin_activity()
>   and end_activity(). Rdbms uses this for proper
>   handling of abort and commit triggers as well as
>   loop detection in referential integrity checks.
>   It also allows rdbms to detect schema changes.
> - An 'on-load' hook allows rdbms to build indexes
>   the first time a table is loaded.
> - A low-level access API for foreign tables. My 
>   first foreign table attempt was a 'disk_log'. 
>   It makes it possible to properly log events 
>   inside a transaction context. You also get 
>   replicated logs almost for free, as well as 
>   (if you want to) fragmented logs. (:
>   My next attempt at a foreign table is a read-
>   only file system (doesn't have to be read-only,
>   but I thought I'd start with that.) Thus
>   the experiments with converting regexps to 
>   the select pattern syntax.
> 
> One interesting experiment might be to define 
> an ISAM table type for really large ordered sets
> on disk. Combining it with rdbms, you can type-
> specify the data types and then convert to 
> whatever format is expected by the ISAM files.
> 
> Some questions to those who might be interested:
> 
> - I'd like to break compatibility with the old
>   rdbms in some respects. Is this a problem for
>   anyone? (speak now or forever hold your peace)
> - Do you have any suggestions or feature requests?
> - Do you want to help out?
> 
> Regards,
> Uffe
> 

-- 
Serge Aleynikov
R&D Telecom, IDT Corp.
Tel: (973) 438-3436
Fax: (973) 438-1464




More information about the erlang-questions mailing list