mnesia replication

Chandrashekhar Mullaparthi chandrashekhar.mullaparthi@REDACTED
Wed Aug 10 18:56:58 CEST 2005

On 10 Aug 2005, at 17:06, Serge Aleynikov wrote:

> mnesia:del_table_copy/2, seems to have a restriction that the node 
> from which the replica is to be deleted should *not* have schema 
> stored on disk:
> ~/tmp>erl -sname a
> ~/tmp>erl -sname b -mnesia extra_db_nodes \[a@REDACTED\]
> (a@REDACTED)1> mnesia:create_schema([a@REDACTED, b@REDACTED]).
> ok
> (a@REDACTED)2> mnesia:start().
> ok
> (b@REDACTED)1> mnesia:start().
> ok
> (a@REDACTED)3> mnesia:create_table(test,
>   [{disc_copies, [a@REDACTED]},{ram_copies, [b@REDACTED]}]).
> {atomic,ok}
> (b@REDACTED)2> q().
> ok
> (a@REDACTED)4> mnesia:del_table_copy(test, b@REDACTED).
> {aborted,{not_active,"All replicas on diskfull nodes are not active 
> yet",
>                      test,
>                      [b@REDACTED]}}

What I suggested (and I thought that's what you asked for) was a way to 
remove the whole RAM copy node out of the schema. If that is what you 
want, the first argument to the mnesia:del_table_copy/2 function must 
be the name schema and not 'test'. In the above example, you are trying 
to delete a table from a node where mnesia is not running. It will 
obviously not work.

> ------------
> What's the advantage/disadvantage of creating the disk-based schema on 
> each node vs. keeping it on a subset of nodes?

In your case, there isn't much of an advantage if you are always going 
to start your RAM copy nodes with the extra_db_nodes sepcified.

> In the configuration below does it make sense to keep the schema on 
> disk only on the Master/Standby nodes?

Consider the situation when both your master/standby nodes are down. 
When your ram-copy node comes up, it will not be able to fetch a schema 
from anywhere and most likely refuse to start - assuming you wait for 
certain tables to be available. Even if you don't wait for tables, your 
function calls to mnesia will crash because the tables you are 
referring to will not be known to mnesia. If this is ok for your 
application then you can get away with keeping the schema on disk only 
on master/standby nodes.

> ------------
> On a separate note, I tried to emulate a disk crash on node 
> b@REDACTED with a disk-copy of the schema by wiping out the content 
> of mnesia directory, and attempting to recover that mnesia node.  Upon 
> startup of mnesia on b@REDACTED it crashed with a core dump:
>  ** FATAL ** Failed to merge schema: Incompatible schema storage types 
> (local). on b@REDACTED storage ram_copies, on a@REDACTED storage 
> disc_copies
> What I did next was I manually copied the "schema.DAT" file from 
> a@REDACTED, and started mnesia.
> ~/tmp>mkdir Mnesia.b\@devlinuxpro/
> ~/tmp>cp Mnesia.a\@devlinuxpro/schema.DAT Mnesia.b\@devlinuxpro/
> ~/tmp>erl -sname b -mnesia extra_db_nodes \[a@REDACTED\]
> (b@REDACTED)1> mnesia:start().
> ok
> Then mnesia:del_table_copy/2 worked alright from node a@REDACTED:
> (a@REDACTED)6> mnesia:del_table_copy(test, b@REDACTED).
> {atomic,ok}
> (b@REDACTED)2> mnesia:info().
> ...
> opt_disc. Directory "/home/serge/tmp/Mnesia.b@REDACTED" is used.
> use fallback at restart = false
> running db nodes   = [a@REDACTED,b@REDACTED]
> stopped db nodes   = []
> master node tables = []
> remote             = [test]
> ram_copies         = []
> disc_copies        = [schema]
> disc_only_copies   = []
> [{a@REDACTED,disc_copies}] = [test]
> [{a@REDACTED,disc_copies},{b@REDACTED,disc_copies}] = [schema]
> ...
> I am not sure if this is the right way to get rid of a faulty replica 
> of a table in case of a disk-hosted schema, but this approach seems to 
> work.

I think the right way of recovery is this.

1. Delete b@REDACTED from the mnesia schema using the 
mnesia:del_table_copy/2 command on a@REDACTED
2. Bring up b@REDACTED with the extra_db_nodes parameter
3. Do an mnesia:add_table_copy(test, b@REDACTED, ram_copies).


> Chandrashekhar Mullaparthi wrote:
>> On 9 Aug 2005, at 21:28, Serge Aleynikov wrote:
>>> Chandrashekhar Mullaparthi wrote:
>>>>> Is this a good approach in terms of avoiding extensive locking 
>>>>> overhead in trying to load multiple tables with preserving 
>>>>> referential integrity of data accross several tables?
>>>> Have you considered using sticky locks? They are quite useful if 
>>>> you are only ever going to write to mnesia on one node at a time.
>>> I am experimenting with several writing methods including 
>>> table-level locks, sticky locks, and backup-restore technique 
>>> (suggested by Vance in one of the former emails), but haven't come 
>>> to a conclusion yet which technique provides better performance in 
>>> case of the following mnesia configuration and size of the database 
>>> of about 700M:
>>>   Master   -------------- RamCopy1
>>>   DiskCopy ------+        ...
>>>     |            |        ...
>>>     |            +------- RamCopyN
>>>   Standby
>>>   DiskCopy
>>> All writes happen either at the Master or at Standby (if Master is 
>>> down) node.
>>> What I need to ensure is that:
>>> 2. It should be possible to delete table copies of RamCopyK node 
>>> from the mnesia schema on Master/Standby nodes, while the RamCopyK 
>>> host is off-line.
>> You can do this using mnesia:del_table_copy/3
>> mnesia:del_table_copy(schema, r@REDACTED).
>> Once you do this you are not supposed to bring back the node 
>> r@REDACTED with it's old database. The results are "undefined" 
>> according to the documentation.
>>> 5. The should be no replication of data between RamCopy nodes, only 
>>> from Master/Slave to RamCopy nodes.
>> AFAIK, this is not possible with mnesia. But as long as you are not 
>> going to perform updates on the RAM copy nodes, there will not be any 
>> replication traffic between these nodes - it will always be from 
>> master/slave to the ram copy nodes.
>>> I am not sure how to deal with #2 and #5...
>> hope this helps.
>> Chandru
> -- 
> Serge Aleynikov
> R&D Telecom, IDT Corp.
> Tel: (973) 438-3436
> Fax: (973) 438-1464
> serge@REDACTED

More information about the erlang-questions mailing list