[erlang-questions] sending data down the wire in mysql vs. mnesia

[Paul Mineiro]

I don't find this answer satisfactory in the case of distributed
fragmented mnesia tables.

I've actually had to rewrite computations in map-reduce form with servers
on each node doing operations on fragments that are local to the node (I
still use mnesia for storage for operational benefits) and then collecting
results (similar to gen_server:multi_call/4, but using pg2 and a custom
hash function known both to the client and mnesia).  I was hoping to
implement such computations in straight mnesia (less chances for me to
screw something up) but noticed it was easy to start seeing 20 mbits of
sustained I/O between my boxes even at low query rates for my application.

It'd be nice to somehow within mnesia extend the concept of fragmented
tables to reduce network I/O for incrementally computable aggregate
functions.

-- p


On Sat, 1 Dec 2007, Lev Walkin wrote:

> Andrew Arrow wrote:
> > I have a PHP + mysql app now that uses a mysql stored procedure to
> > avoid sending lots of data "down the wire."  That is, the stored
> > procedure iterates over thousands of rows, determines the value the
> > app needs and sends back just the value.  This avoids the thousands of
> > rows coming down the wire to have PHP compute the value.
> >
> > In moving this system to erlang + mnesia do I have to worry about the
> > same issue?  It seems I will have to store the thousands of items in a
> > set in a mnesia table, retrieve all the rows, and have erlang compute
> > the value.  There's no way to do the iteration inside mnesia like
> > there is in mysql is there?
>
>
> You do this "stored procedure" thing in MySQL primarily to avoid
> inter-process communication and serialization/deserialization.
>
> In Erlang+Mnesia case, the Mnesia table runs inside Erlang VM,
> therefore avoiding inter-process communication issues. Also,
> there is some advantage in not having to perform serialization
> and deserialization, although with *ts tables (which Mnesia
> is based upon) there is some copying going on anyway.
>
> So, the short answer is no, you don't do it in Mnesia. Primarily
> because your surrounding Erlang code precisely acts like a "stored
> procedure" around Mnesia tables, running in the same VM.
>
>
> --
> Lev Walkin
> vlm@REDACTED
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>

Optimism is an essential ingredient of innovation. How else can the
individual favor change over security?

  -- Robert Noyce