Watching mnesia transactions

[Bernard Duggan]

Hi all,
    Bit of a complex mnesia question here.  First, I need to describe
what we want to do:

    For reasons of interacting with a legacy system (which I'll call
"L") we want to be able to track changes to certain of our mnesia
tables, and send those changes off so L can update its own copy of the
data.  Sadly, it's thoroughly impractical to remove the local copy in L,
so we're pretty much stuck with this approach for now.  I'll explain the
approaches we've tried so far and the problems we've had - hopefully
someone can either suggest where our thinking has gone astray, or an
entirely different system :)


Attempt 1:
Using mnesia:subscribe({mnesia_table_event...}), we watch for table
events and send them off to L.  That's very nice and simple, except
there's no way we can see to group transaction events together - every
table event has its own ActivityID, certainly, but how do you know when
all the events for a given ActivityID have arrived?  This seems like a
pretty serious limitation on what otherwise is a very handy system.


Attempt 2:
Use the mnesia_access callback module and
mnesia:activity(transaction...).  Here, we've done the following:
* Prior to each mnesia:activity call, we set up an ETS table and throw
it in the process dictionary. 
* In the transaction function we first empty the ETS table (to ensure
that if the transaction restarts, the table will be cleared)
* In the activity callback functions we record to the ETS table each
write/delete event that takes place on mnesia.
* When the transaction successfully completes, we can dump that table to
get a full record of write events, grouped as a transaction, and throw
those to L.

We were really happy with that implementation until I realised a minor
problem with it this morning:
The posting of events to L happens /after/ the transaction completes. 
Consider the case of two transactions, T1 and T2 in separate processes
which both modify the same row.  You could then have the case where:
* T1 runs
* T2 runs
* T2 sends its change to L
* T1 sends its older change to L
If, alternatively, we do the posting inside the transaction, even as the
last operation, it seems like it might be possible for the transaction
to restart after the post and get us in all kinds of trouble.

We've come up with a number of elaborate modifications to this plan to
resolve the issue, but I can't help but think we're missing something
obvious (or non-obvious).  After all, mnesia knows full well what's gone
on and in what order, so why do we seem to need to go to such lengths to
reconstruct that information?


I've slightly simplified this to keep it relatively short, so I'll
apologise in advance if any of your questions are answered with "yeah we
already thought of that and with won't work/doesn't apply because blah
blah" :)

Thanks for reading this far :)

Cheers,

Bernard