[erlang-questions] Mnesia Fragmented Tables, del_table_copy

[Håkan Mattsson]

2011/1/28 Tessaro Alexej <atessaro@REDACTED>:
> Ok, so I can remove the last fragment using change_table_copy(Tab, del_frag),

No. You should use the mnesia:change_table_frag/2 function.

> but what am I supposed to do if I loose one node (let's say
> by physical disconnection) and the fragments in it are last replicas..
>
> In my opinion, the on-top application logic should continue to work but
> the data contained in the lost node fragments will not be considered...right?

If you want a fault tolerant system where you want the system to continue
to work despite that some node has crashed, you need to ensure that you
have redundant resources. For short you need to replicate your Mnesia
tables.

> Well, for example, if I try to call (after loosing one node and his fragments):
>
> mnesia:activity(async_dirty, TableInfo, [Table, size], mnesia_frag)
>
> where TableInfo is
>
> TableInfo = fun(TName, TInfo)   ->
>        mnesia:table_info(TName, TInfo)
> end
>
> I am looking for the whole table size (the sum of the table fragments
> sizes), but the function aborts when considering the lost fragments.
> In my case I would like the mnesia table_info fun to return me the table
> size anyway (simply adding 0 records to the table size accumulator when
> considering not existing fragments).

It is simply so that as all fragments not are available, Mnesia cannot compute
a correct table size and therefore the function will fail. Firstly the
behavior is
consistent with how ordinary Mnesia tables works: the table size can only be
computed if at least one replica of the table is online. Secondly it
is consistent
with how hash tables works: the table size can only be computed if all buckets
are available.

> But this is only a personal need in a personal application context,
> where I prefer to maintain ram only copies with no replicas.

You are trying to use fragmented tables in a way that they not was intended
to be used. I think that it would be better for you if you used ordinary Mnesia
tables and rolled your own fragmention logic on top of them. In fact
it I did not
get the impression that you actually need a distributed hash table (fragmented
table) as you wanted to remove arbitrary buckets (table fragments). That is a
rather unusual use of a hash table.

/Håkan

> On Thu, 2011-01-27 at 22:36 +0100, Håkan Mattsson wrote:
>> A fragmented table in Mnesia is a big distributed hash table where each bucket
>> is implemented as a "normal" Mnesia table. When you insert a record in the
>> fragmented table, a hash function is used to compute which bucket (table
>> fragment) to put the record in. The linear hashing algorithm that is used allows
>> the hash table to shrink by deleting the last bucket (table fragment) and
>> dynamically rehash the records in that bucket and move them to other buckets.
>> The algorithm does however not cope with deletion of arbitrary buckets.
>>
>> If you want to shrink the fragmented table (delete the last bucket) you need to
>> use the function mnesia:change_table_frag(Tab, del_frag) as described in the
>> user's guide: http://www.erlang.org/doc/apps/mnesia/Mnesia_chap5.html.
>>
>> Each bucket (table fragment) can be replicated to several nodes. Replicas
>> are deleted with the function mnesia:del_table_copy(Frag, Node) and it is
>> not possible to use this function to delete the last replica.
>>
>> In your case you had 4 fragments and you tried to delete the last replica
>> of the second fragment. This is not allowed. But it would have been allowed
>> to delete the fourth fragment (and move its records) by invoking
>>
>>   mnesia:change_table_frag(constResourceData, del_frag)
>>
>> /Håkan
>>
>> PS.
>>   The function mnesia:change_table_frag/2 is only mentioned in the user's
>>   guide. It is not documented in the reference manual. Sorry for that.
>>
>> On Thu, Jan 27, 2011 at 6:18 PM, Tessaro Alexej <atessaro@REDACTED> wrote:
>> > Thank you for your reply.
>> >
>> > In my case the table is dynamically growing (by adding new fragments
>> > when the existing ones reach some sort of threshold).
>> >
>> > Cause the contained data have no critical relevance (but real-time
>> > reading and writing times constraints), I thought to maintain ram only
>> > copies with no replicas.
>> >
>> > So, when a node disconnects from the cluster cloud, I would expect the
>> > data it was containing to disappear from the cloud (and the requests
>> > involving those records to fail).
>> >
>> > I expected mnesia_frag system to detect not running mnesia nodes, thus
>> > not considering them when calling mnesia activity functions on the whole
>> > table.
>> >
>> > Am I missing something?
>> >
>> > On Wed, 2011-01-26 at 14:45 -0200, Igor Ribeiro Sucupira wrote:
>> >> I haven't looked at it in detail, but I know it doesn't make much
>> >> sense to delete the second fragment (leaving no other copies of it) in
>> >> a table that has 4 fragments. It would break your fragmented table
>> >> (writing keys would not be possible in some cases).
>> >>
>> >> Best regards.
>> >> Igor.
>> >>
>> >> On Wed, Jan 19, 2011 at 11:31 AM, Tessaro Alexej <atessaro@REDACTED> wrote:
>> >> > Hello,
>> >> >
>> >> > I have started two communicating mnesia nodes and created one fragmented
>> >> > table with fragments stored in RAM only (no other replicas) and schema
>> >> > stored on disk on both nodes:
>> >> >
>> >> > node1 : constResourceData_frag3 (ram), constResourceData_frag4 (ram),
>> >> > schema (disk)
>> >> >
>> >> > node2 : constResourceData (ram), constResourceData_frag2 (ram), schema
>> >> > (disk)
>> >> >
>> >> > Now, from node1, I tried to call
>> >> > mnesia:del_table_copy(constResourceData_frag2, node2)
>> >> >
>> >> > but it fails saying:
>> >> >
>> >> > node1> Mnesia(node1): Last replica deleted in table
>> >> > constResourceData_frag2
>> >> > node1>{aborted,{no_exists,constResourceData_frag2,frag_properties,
>> >> >                    frag_hash}}
>> >> > node1> Mnesia(node1): Transaction {tid,75,<0.371.0>} calling
>> >> > #Fun<mnesia_schema.15.79614902> with [] failed:
>> >> >  {aborted,{no_exists,constResourceData_frag2,frag_properties,frag_hash}}
>> >> >
>> >> > Looking at the mnesia_schema.erl source code, I found that the failure
>> >> > seems to happen when it tries to delete the "whole_table" in
>> >> > make_delete_table/2 and it runs the following check:
>> >> >
>> >> > %% Check if it is a base table
>> >> > nesia_frag:lookup_frag_hash(Tab),
>> >> >
>> >> > Why is my attempt of removing the fragment failing?
>> >> >
>> >> > My erlang version is R12B.
>> >> >
>> >> > Thank you
>>
>> ________________________________________________________________
>> erlang-questions (at) erlang.org mailing list.
>> See http://www.erlang.org/faq.html
>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
>>
>
> --
> Tessaro Alexej, Volunia Italia
>
> DISCLAIMER: This e-mail and any attachment is for authorised use by
> the intended recipient(s) only. It may contain proprietary material,
> confidential information and/or be subject to legal privilege. It
> should not be copied, disclosed to, retained or used by, any other
> party. If you are not an intended recipient then please promptly
> delete this e-mail and any attachment and all copies and inform
> the sender. Thank you.
>
>
> ________________________________________________________________
> erlang-questions (at) erlang.org mailing list.
> See http://www.erlang.org/faq.html
> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
>
>