early warning - new rdbms

[ke.han]

Ulf,
I am two weeks away from v1.0 of an app for a client.  It uses mnesia, 
yaws, ajax.  I would love to review what you have to apply it to the 
next version.
My plan is to pull together a simple framework based on mnesia, 
ajax/json, plain_fsm (your latest one), and yaws to take things a level 
or two of maintainability beyond what I've done in version 1 of this app.
thanks for putting me on the beta group.
ke han

Ulf Wiger (AL/EAB) wrote:
> I thought I'd let the cat out of the bag a little...
> 
> If anyone wants to beta-test or help out with some
> of the more advanced problems, let me know.
> 
> 
> I've come pretty far along with a new version of 
> rdbms. It has several nice features, and I think
> it's about to make the transition from 'somewhat
> interesting' to 'actually useful':
> 
> - JIT compilation of verification code. The overhead
>   for type and bounds checking is now only a few (~5)
>   microseconds per write operation.
> - The parameterized indexes that I hacked into mnesia
>   before are now part of rdbms. This include ordered
>   indexes and fragmented indexes (i.e. hashed on 
>   index value - so they should scale well.)
> - Rdbms will handle fragmented tables transparently
>   (And actually handles plain tables with less overhead
>   than mnesia_frag does.) The overhead for using the
>   rdbms access module (compared to no access module)
>   on a plain transaction is in the order of 20 
>   microseconds on my 1 GHz SunBLADE.
> - Rdbms hooks into the transaction handling in such a
>   way that it can automatically rebuild the verification
>   code as soon as a schema transaction commits.
> - A readable file format for schema definitions, trying
>   to establish a structured way to create large mnesia
>   databases. I've also added a 'group' concept to be
>   able to group tables into corresponding subsystems,
>   since I thought this might be helpful in large 
>   systems.
> 
> 
> I'm planning to release rdbms with OTP R11, since it
> requires some changes to mnesia that (hopefully) will
> make it into R11. R11 is planned for May.
> 
> Some of the (fairly minor) changes to mnesia so far:
> 
> - The access module can hook into the transaction
>   flow by providing callbacks for begin_activity()
>   and end_activity(). Rdbms uses this for proper
>   handling of abort and commit triggers as well as
>   loop detection in referential integrity checks.
>   It also allows rdbms to detect schema changes.
> - An 'on-load' hook allows rdbms to build indexes
>   the first time a table is loaded.
> - A low-level access API for foreign tables. My 
>   first foreign table attempt was a 'disk_log'. 
>   It makes it possible to properly log events 
>   inside a transaction context. You also get 
>   replicated logs almost for free, as well as 
>   (if you want to) fragmented logs. (:
>   My next attempt at a foreign table is a read-
>   only file system (doesn't have to be read-only,
>   but I thought I'd start with that.) Thus
>   the experiments with converting regexps to 
>   the select pattern syntax.
> 
> One interesting experiment might be to define 
> an ISAM table type for really large ordered sets
> on disk. Combining it with rdbms, you can type-
> specify the data types and then convert to 
> whatever format is expected by the ISAM files.
> 
> Some questions to those who might be interested:
> 
> - I'd like to break compatibility with the old
>   rdbms in some respects. Is this a problem for
>   anyone? (speak now or forever hold your peace)
> - Do you have any suggestions or feature requests?
> - Do you want to help out?
> 
> Regards,
> Uffe
>