mnesia replication

[Serge Aleynikov]

Thanks for your recommendations!

Chandrashekhar Mullaparthi wrote:
> On 10 Aug 2005, at 17:06, Serge Aleynikov wrote:
> 
>> mnesia:del_table_copy/2, seems to have a restriction that the node 
>> from which the replica is to be deleted should *not* have schema 
>> stored on disk:
>>
>> ~/tmp>erl -sname a
>> ~/tmp>erl -sname b -mnesia extra_db_nodes \[a@REDACTED\]
>>
>> (a@REDACTED)1> mnesia:create_schema([a@REDACTED, b@REDACTED]).
>> ok
>> (a@REDACTED)2> mnesia:start().
>> ok
>>
>> (b@REDACTED)1> mnesia:start().
>> ok
>>
>> (a@REDACTED)3> mnesia:create_table(test,
>>   [{disc_copies, [a@REDACTED]},{ram_copies, [b@REDACTED]}]).
>> {atomic,ok}
>>
>> (b@REDACTED)2> q().
>> ok
>>
>> (a@REDACTED)4> mnesia:del_table_copy(test, b@REDACTED).
>> {aborted,{not_active,"All replicas on diskfull nodes are not active yet",
>>                      test,
>>                      [b@REDACTED]}}
> 
> 
> What I suggested (and I thought that's what you asked for) was a way to 
> remove the whole RAM copy node out of the schema. If that is what you 
> want, the first argument to the mnesia:del_table_copy/2 function must be 
> the name schema and not 'test'. In the above example, you are trying to 
> delete a table from a node where mnesia is not running. It will 
> obviously not work.
> 
>> ------------
>>
>> What's the advantage/disadvantage of creating the disk-based schema on 
>> each node vs. keeping it on a subset of nodes?
> 
> 
> In your case, there isn't much of an advantage if you are always going 
> to start your RAM copy nodes with the extra_db_nodes sepcified.
> 
>>
>> In the configuration below does it make sense to keep the schema on 
>> disk only on the Master/Standby nodes?
> 
> 
> Consider the situation when both your master/standby nodes are down. 
> When your ram-copy node comes up, it will not be able to fetch a schema 
> from anywhere and most likely refuse to start - assuming you wait for 
> certain tables to be available. Even if you don't wait for tables, your 
> function calls to mnesia will crash because the tables you are referring 
> to will not be known to mnesia. If this is ok for your application then 
> you can get away with keeping the schema on disk only on master/standby 
> nodes.
> 
>>
>> ------------
>> On a separate note, I tried to emulate a disk crash on node 
>> b@REDACTED with a disk-copy of the schema by wiping out the content 
>> of mnesia directory, and attempting to recover that mnesia node.  Upon 
>> startup of mnesia on b@REDACTED it crashed with a core dump:
>>
>>  ** FATAL ** Failed to merge schema: Incompatible schema storage types 
>> (local). on b@REDACTED storage ram_copies, on a@REDACTED storage 
>> disc_copies
>>
>> What I did next was I manually copied the "schema.DAT" file from 
>> a@REDACTED, and started mnesia.
>>
>> ~/tmp>mkdir Mnesia.b\@devlinuxpro/
>> ~/tmp>cp Mnesia.a\@devlinuxpro/schema.DAT Mnesia.b\@devlinuxpro/
>>
>> ~/tmp>erl -sname b -mnesia extra_db_nodes \[a@REDACTED\]
>> (b@REDACTED)1> mnesia:start().
>> ok
>>
>> Then mnesia:del_table_copy/2 worked alright from node a@REDACTED:
>>
>> (a@REDACTED)6> mnesia:del_table_copy(test, b@REDACTED).
>> {atomic,ok}
>>
>> (b@REDACTED)2> mnesia:info().
>> ...
>> opt_disc. Directory "/home/serge/tmp/Mnesia.b@REDACTED" is used.
>> use fallback at restart = false
>> running db nodes   = [a@REDACTED,b@REDACTED]
>> stopped db nodes   = []
>> master node tables = []
>> remote             = [test]
>> ram_copies         = []
>> disc_copies        = [schema]
>> disc_only_copies   = []
>> [{a@REDACTED,disc_copies}] = [test]
>> [{a@REDACTED,disc_copies},{b@REDACTED,disc_copies}] = [schema]
>> ...
>>
>> I am not sure if this is the right way to get rid of a faulty replica 
>> of a table in case of a disk-hosted schema, but this approach seems to 
>> work.
> 
> 
> I think the right way of recovery is this.
> 
> 1. Delete b@REDACTED from the mnesia schema using the 
> mnesia:del_table_copy/2 command on a@REDACTED
> 2. Bring up b@REDACTED with the extra_db_nodes parameter
> 3. Do an mnesia:add_table_copy(test, b@REDACTED, ram_copies).
> 
> Chandru
> 
>>
>> Chandrashekhar Mullaparthi wrote:
>>
>>> On 9 Aug 2005, at 21:28, Serge Aleynikov wrote:
>>>
>>>> Chandrashekhar Mullaparthi wrote:
>>>>
>>>>>> Is this a good approach in terms of avoiding extensive locking 
>>>>>> overhead in trying to load multiple tables with preserving 
>>>>>> referential integrity of data accross several tables?
>>>>>
>>>>>
>>>>> Have you considered using sticky locks? They are quite useful if 
>>>>> you are only ever going to write to mnesia on one node at a time.
>>>>
>>>>
>>>>
>>>> I am experimenting with several writing methods including 
>>>> table-level locks, sticky locks, and backup-restore technique 
>>>> (suggested by Vance in one of the former emails), but haven't come 
>>>> to a conclusion yet which technique provides better performance in 
>>>> case of the following mnesia configuration and size of the database 
>>>> of about 700M:
>>>>
>>>>   Master   -------------- RamCopy1
>>>>   DiskCopy ------+        ...
>>>>     |            |        ...
>>>>     |            +------- RamCopyN
>>>>   Standby
>>>>   DiskCopy
>>>>
>>>> All writes happen either at the Master or at Standby (if Master is 
>>>> down) node.
>>>>
>>>> What I need to ensure is that:
>>>>
>>>> 2. It should be possible to delete table copies of RamCopyK node 
>>>> from the mnesia schema on Master/Standby nodes, while the RamCopyK 
>>>> host is off-line.
>>>
>>> You can do this using mnesia:del_table_copy/3
>>> mnesia:del_table_copy(schema, r@REDACTED).
>>> Once you do this you are not supposed to bring back the node 
>>> r@REDACTED with it's old database. The results are "undefined" 
>>> according to the documentation.
>>>
>>>>
>>>> 5. The should be no replication of data between RamCopy nodes, only 
>>>> from Master/Slave to RamCopy nodes.
>>>
>>> AFAIK, this is not possible with mnesia. But as long as you are not 
>>> going to perform updates on the RAM copy nodes, there will not be any 
>>> replication traffic between these nodes - it will always be from 
>>> master/slave to the ram copy nodes.
>>>
>>>> I am not sure how to deal with #2 and #5...
>>>
>>> hope this helps.
>>> Chandru
>>
>>
>> -- 
>> Serge Aleynikov
>> R&D Telecom, IDT Corp.
>> Tel: (973) 438-3436
>> Fax: (973) 438-1464
>> serge@REDACTED
>>
> 
> 

-- 
Serge Aleynikov
R&D Telecom, IDT Corp.
Tel: (973) 438-3436
Fax: (973) 438-1464
serge@REDACTED