[erlang-questions] Mnesia Fragmented Tables, del_table_copy

Fri Jan 28 15:16:44 CET 2011

Thanks for your help!

On Fri, 2011-01-28 at 10:26 +0100, Håkan Mattsson wrote:
> 2011/1/28 Tessaro Alexej <atessaro@REDACTED>:
> > Ok, so I can remove the last fragment using change_table_copy(Tab, del_frag),
> 
> No. You should use the mnesia:change_table_frag/2 function.
> 
> > but what am I supposed to do if I loose one node (let's say
> > by physical disconnection) and the fragments in it are last replicas..
> >
> > In my opinion, the on-top application logic should continue to work but
> > the data contained in the lost node fragments will not be considered...right?
> 
> If you want a fault tolerant system where you want the system to continue
> to work despite that some node has crashed, you need to ensure that you
> have redundant resources. For short you need to replicate your Mnesia
> tables.
> 
> > Well, for example, if I try to call (after loosing one node and his fragments):
> >
> > mnesia:activity(async_dirty, TableInfo, [Table, size], mnesia_frag)
> >
> > where TableInfo is
> >
> > TableInfo = fun(TName, TInfo)   ->
> >        mnesia:table_info(TName, TInfo)
> > end
> >
> > I am looking for the whole table size (the sum of the table fragments
> > sizes), but the function aborts when considering the lost fragments.
> > In my case I would like the mnesia table_info fun to return me the table
> > size anyway (simply adding 0 records to the table size accumulator when
> > considering not existing fragments).
> 
> It is simply so that as all fragments not are available, Mnesia cannot compute
> a correct table size and therefore the function will fail. Firstly the
> behavior is
> consistent with how ordinary Mnesia tables works: the table size can only be
> computed if at least one replica of the table is online. Secondly it
> is consistent
> with how hash tables works: the table size can only be computed if all buckets
> are available.
> 
> > But this is only a personal need in a personal application context,
> > where I prefer to maintain ram only copies with no replicas.
> 
> You are trying to use fragmented tables in a way that they not was intended
> to be used. I think that it would be better for you if you used ordinary Mnesia
> tables and rolled your own fragmention logic on top of them. In fact
> it I did not
> get the impression that you actually need a distributed hash table (fragmented
> table) as you wanted to remove arbitrary buckets (table fragments). That is a
> rather unusual use of a hash table.
> 
> /Håkan
> 
> > On Thu, 2011-01-27 at 22:36 +0100, Håkan Mattsson wrote:
> >> A fragmented table in Mnesia is a big distributed hash table where each bucket
> >> is implemented as a "normal" Mnesia table. When you insert a record in the
> >> fragmented table, a hash function is used to compute which bucket (table
> >> fragment) to put the record in. The linear hashing algorithm that is used allows
> >> the hash table to shrink by deleting the last bucket (table fragment) and
> >> dynamically rehash the records in that bucket and move them to other buckets.
> >> The algorithm does however not cope with deletion of arbitrary buckets.
> >>
> >> If you want to shrink the fragmented table (delete the last bucket) you need to
> >> use the function mnesia:change_table_frag(Tab, del_frag) as described in the
> >> user's guide: http://www.erlang.org/doc/apps/mnesia/Mnesia_chap5.html.
> >>
> >> Each bucket (table fragment) can be replicated to several nodes. Replicas
> >> are deleted with the function mnesia:del_table_copy(Frag, Node) and it is
> >> not possible to use this function to delete the last replica.
> >>
> >> In your case you had 4 fragments and you tried to delete the last replica
> >> of the second fragment. This is not allowed. But it would have been allowed
> >> to delete the fourth fragment (and move its records) by invoking
> >>
> >>   mnesia:change_table_frag(constResourceData, del_frag)
> >>
> >> /Håkan
> >>
> >> PS.
> >>   The function mnesia:change_table_frag/2 is only mentioned in the user's
> >>   guide. It is not documented in the reference manual. Sorry for that.
> >>
> >> On Thu, Jan 27, 2011 at 6:18 PM, Tessaro Alexej <atessaro@REDACTED> wrote:
> >> > Thank you for your reply.
> >> >
> >> > In my case the table is dynamically growing (by adding new fragments
> >> > when the existing ones reach some sort of threshold).
> >> >
> >> > Cause the contained data have no critical relevance (but real-time
> >> > reading and writing times constraints), I thought to maintain ram only
> >> > copies with no replicas.
> >> >
> >> > So, when a node disconnects from the cluster cloud, I would expect the
> >> > data it was containing to disappear from the cloud (and the requests
> >> > involving those records to fail).
> >> >
> >> > I expected mnesia_frag system to detect not running mnesia nodes, thus
> >> > not considering them when calling mnesia activity functions on the whole
> >> > table.
> >> >
> >> > Am I missing something?
> >> >
> >> > On Wed, 2011-01-26 at 14:45 -0200, Igor Ribeiro Sucupira wrote:
> >> >> I haven't looked at it in detail, but I know it doesn't make much
> >> >> sense to delete the second fragment (leaving no other copies of it) in
> >> >> a table that has 4 fragments. It would break your fragmented table
> >> >> (writing keys would not be possible in some cases).
> >> >>
> >> >> Best regards.
> >> >> Igor.
> >> >>
> >> >> On Wed, Jan 19, 2011 at 11:31 AM, Tessaro Alexej <atessaro@REDACTED> wrote:
> >> >> > Hello,
> >> >> >
> >> >> > I have started two communicating mnesia nodes and created one fragmented
> >> >> > table with fragments stored in RAM only (no other replicas) and schema
> >> >> > stored on disk on both nodes:
> >> >> >
> >> >> > node1 : constResourceData_frag3 (ram), constResourceData_frag4 (ram),
> >> >> > schema (disk)
> >> >> >
> >> >> > node2 : constResourceData (ram), constResourceData_frag2 (ram), schema
> >> >> > (disk)
> >> >> >
> >> >> > Now, from node1, I tried to call
> >> >> > mnesia:del_table_copy(constResourceData_frag2, node2)
> >> >> >
> >> >> > but it fails saying:
> >> >> >
> >> >> > node1> Mnesia(node1): Last replica deleted in table
> >> >> > constResourceData_frag2
> >> >> > node1>{aborted,{no_exists,constResourceData_frag2,frag_properties,
> >> >> >                    frag_hash}}
> >> >> > node1> Mnesia(node1): Transaction {tid,75,<0.371.0>} calling
> >> >> > #Fun<mnesia_schema.15.79614902> with [] failed:
> >> >> >  {aborted,{no_exists,constResourceData_frag2,frag_properties,frag_hash}}
> >> >> >
> >> >> > Looking at the mnesia_schema.erl source code, I found that the failure
> >> >> > seems to happen when it tries to delete the "whole_table" in
> >> >> > make_delete_table/2 and it runs the following check:
> >> >> >
> >> >> > %% Check if it is a base table
> >> >> > nesia_frag:lookup_frag_hash(Tab),
> >> >> >
> >> >> > Why is my attempt of removing the fragment failing?
> >> >> >
> >> >> > My erlang version is R12B.
> >> >> >
> >> >> > Thank you
> >>
> >> ________________________________________________________________
> >> erlang-questions (at) erlang.org mailing list.
> >> See http://www.erlang.org/faq.html
> >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
> >>
> >
> > --
> > Tessaro Alexej, Volunia Italia
> >
> > DISCLAIMER: This e-mail and any attachment is for authorised use by
> > the intended recipient(s) only. It may contain proprietary material,
> > confidential information and/or be subject to legal privilege. It
> > should not be copied, disclosed to, retained or used by, any other
> > party. If you are not an intended recipient then please promptly
> > delete this e-mail and any attachment and all copies and inform
> > the sender. Thank you.
> >
> >
> > ________________________________________________________________
> > erlang-questions (at) erlang.org mailing list.
> > See http://www.erlang.org/faq.html
> > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
> >
> >
> 
> ________________________________________________________________
> erlang-questions (at) erlang.org mailing list.
> See http://www.erlang.org/faq.html
> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
> 

-- 
Tessaro Alexej, Volunia Italia

DISCLAIMER: This e-mail and any attachment is for authorised use by
the intended recipient(s) only. It may contain proprietary material,
confidential information and/or be subject to legal privilege. It
should not be copied, disclosed to, retained or used by, any other
party. If you are not an intended recipient then please promptly
delete this e-mail and any attachment and all copies and inform
the sender. Thank you.