[erlang-questions] Can one mnesia table fragment be further fragmented?
Fri Jun 20 07:23:06 CEST 2008
Thanks ,Paul. I decide to replace the mnesia_frag_hash with my own hash
module .With this customized hash module,I hope I can explicitly pass a
fragment number to make the fragment splited.
I decide to use a mnesia table or other persitent-able data structure in
erlang which can globally seen by all nodes to store the mapping relations
between key value range and fragment(Eg. [1..999]-> frag
1,[1000-1999]->frag2), so when I need to split a specifed fragment ,I can
rejust the mapping relation table.
Here's my plan,any suggestions are welcome.
2008/6/19, Paul Mineiro <>:
> Re: picking what gets split, not with the default frag_hash. You can see
> what it's doing in mnesia_frag_hash.erl ... it basically splits the
> fragments in order. This means that unless there is a power-of-two number
> of fragments they will be of different magnitudes. You could leverage
> that for your load balancing by placing bigger fragments on less noded
> nodes, etc. In fact, maybe this is why you are seeing
> an uneven distribution to begin with?
> -- p
> p.z. If you are feeling your wheaties, you can make your own frag hash,
> and specify the module in the frag_properties when you create the table.
> Don't be tempted to just use phash2 or something like that; in order to go
> from N fragments to M fragments you have to pass through all the
> intermediate stages N + 1, ..., M - 1, M ; you do not want to rehash all
> the data in all the buckets at each step.
> On Thu, 19 Jun 2008, devdoer bird wrote:
> > Thanks.
> > Thant means ,I need fragment table many times not for load balance but
> > decreasing one specific node's (the node is not as powerful as others )
> > load.
> > Eg. Node A has 2 fragments,Node B has 2 fragments. Each fragment has
> > records(say 100 records each fragment).
> > Now the Node B is heavily loaded.If I adding one or two fragments in
> > A , the load of Node B won't be
> > decreased ,for the records from Node A will be splited and moved to the
> > fragments,the records on Node A is 50+50+50+50=200 ,the total number of
> > Node B's record is still 200.
> > In order to decreas the load on Node B,I have to calcaulate carefully how
> > many new fragments should be added.
> > If there's a way for mnesia to specify which node's fragment should be
> > splited when addin a new fragment,the problems above will be solved.Take
> > example above ,when adding a new fragment,I can tell mnesia to split
> > B's fragment.
> > 2008/6/19, Scott Lystig Fritchie <>:
> > >
> > > devdoer bird <> wrote:
> > >
> > > dd> I spilit one mnesia table into 3 fragments which distributed on 3
> > > dd> machines named A,B and C,but now the the machine "A" is very busy,
> > >
> > > dd> so I decide split the fragment on the machine "A" into small
> > > dd> fragments.Does mnesia support this?
> > >
> > > No, that isn't possible with Mnesia, not as you wish/state above.
> > >
> > > If you have a table 'tab' split into fragments 'tab', 'tab_frag2', and
> > > 'tab_frag3', you can add more fragments: 'tab_frag4', 'tab_frag5', etc.
> > > You can choose to put only 1 fragment on node A, 4 fragments on node B,
> > > and 4 fragments on node C. Assuming that access to the table is evenly
> > > distributed over all fragments, then node A should be getting only 1/9
> > > of the load, and B & C will get 4/9 of the load.
> > >
> > > -Scott
> > >
> In an artificial world, only extremists live naturally.
> -- Paul Graham
> erlang-questions mailing list
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the erlang-questions