mneisa table fragmentation vs manually splitting

[vim]

hi Z, thanks for your reply,

> why do you claim the application should know how to use a specific node to write to a specific key?

i need to know in which cluster resides the key i need. i.e. if i have
partitioned my database to 100 clusters, in need to keep a map which
tell me on which cluster i can found a specific key.

this is not related to the master-slave scheme, even if i have just 1
machine on each cluster, i need to do the same. the master-slave
scheme is a just a possible enhancement, so i can have hot-backup for
each key-value.

on the other hand, if i use mnesia fragmented database feature, mnesia
(as far as i understand) should keep track on where each key resides,
releasing me from keeping that key mapping.

the disadvantage i fear is the up-scale limit of connecting many
machines together without partitioning into clusters. from what i have
read here and there, erlang shouldn't be connected together on more
than few hundreds active machine. * if anyone has numbers of this
practical limit, please advise *

> What are the master-slave relations you assume in the 10 machine cluster?

master-slave refers to a scheme where i connect all the mnesia
instances on the relevant machines, so they are synced. but i choose
split up my dataset, where for a given key i only do writes through
specific machine/node. on that kind of scheme i get fault-tolerance
because if the master (the writer) fails, i always have a backup
(slave) with the most up to date data. and i can assign one of the
slave to be a master if the master fails. furthermore, i don't get
into many problems of multiple write, and syncing them, when you use
master-master scheme (more than 1 writer)

vim.

On Oct 1, 12:31 am, Zohan <matan.l...@REDACTED> wrote:
> hi,
>
> For the manual split option you describe, why do you claim
> the application should know how to use a specific node to write to a
> specific key,
> in order to use master-slave duplicated database scheme?
> What are the master-slave relations you assume in the 10 machine
> cluster?
>
> BR,
> Z.
>
> On Sep 27, 8:40 am, vim <ofer.aff...@REDACTED> wrote:
>
>
>
>
>
> > hi,
>
> > i am working on a erlang based distributed architecture, at the end i
> > intend to use it on EC2. my application included a database, and since
> > i want to spread the load on the database, i thought of splitting the
> > table.
>
> > the question is should i choose mnesia built-in feature 'fragmented
> > table' over manually splitting the database into clusters?
>
> > i.e. suppose i have a simple key-value sets which i need to store in
> > my database. and suppose i have 1000 keys, and i did some measurements
> > and i want only 10 keys on each machine (for better performance) so i
> > need 100 machines.  thus, if i understand correctly the
> > documentation, i have at least two options:
>
> > 1. manual split:
> > i can set up clusters of 10 machines each. on each machine i will have
> > a node with a running mnesia, and on a such cluster all the mnesia
> > nodes will be connected. furthermore, i can use master-slave
> > duplicated database scheme, if my application will know how to use a
> > specific node to write to a specific key. i assume here that *all* the
> > nodes in my world (100) are erlang connected, and on each 10 machine
> > cluster there is a connected (shared) mnesia database.
>
> > 2. table fragmentation:
> > i can connect all my 100 machines with single mnesia schema, and
> > fragment the table into 10 pieces, where each cluster of 10 machines
> > will share each piece. again, i can use the same master-slave scheme i
> > described above to access my database.
>
> > am i correct with those assumptions?
> > should i prefer one method over the other?
>
> > thanks for your help,
> > vim.
>
> > ________________________________________________________________
> > erlang-questions mailing list. Seehttp://www.erlang.org/faq.html
> > erlang-questions (at) erlang.org
>
> ________________________________________________________________
> erlang-questions mailing list. Seehttp://www.erlang.org/faq.html
> erlang-questions (at) erlang.org