[erlang-questions] sending data down the wire in mysql vs. mnesia

[Paul Mineiro]

With a fragmented MySQL table, I can execute a stored procedure
on each MySQL fragment, and the reduce the collection of results in the
client.  The only thing that travels over the wire is the collection of
results, one for each fragment.

I'm looking for something similar from Mnesia.  Erlang has all the
building blocks to put it together (so I have), but I can see an
opportunity to come up with something standard within Mnesia which is
reasonably general and would return me to a more location-transparent
style of programming.

-- p

On Sat, 1 Dec 2007, Lev Walkin wrote:

> Paul Mineiro wrote:
> > I don't find this answer satisfactory in the case of distributed
> > fragmented mnesia tables.
> >
> > I've actually had to rewrite computations in map-reduce form with servers
> > on each node doing operations on fragments that are local to the node (I
> > still use mnesia for storage for operational benefits) and then collecting
> > results (similar to gen_server:multi_call/4, but using pg2 and a custom
> > hash function known both to the client and mnesia).  I was hoping to
> > implement such computations in straight mnesia (less chances for me to
> > screw something up) but noticed it was easy to start seeing 20 mbits of
> > sustained I/O between my boxes even at low query rates for my application.
> >
> > It'd be nice to somehow within mnesia extend the concept of fragmented
> > tables to reduce network I/O for incrementally computable
> > aggregate functions.
>
>
> How would you do that with fragmented MySQL tables? Stored procedures
> won't work in this case either. Apples and Oranges?
>
>
>
>
> > -- p
> >
> > On Sat, 1 Dec 2007, Lev Walkin wrote:
> >
> >> Andrew Arrow wrote:
> >>> I have a PHP + mysql app now that uses a mysql stored procedure to
> >>> avoid sending lots of data "down the wire."  That is, the stored
> >>> procedure iterates over thousands of rows, determines the value the
> >>> app needs and sends back just the value.  This avoids the thousands of
> >>> rows coming down the wire to have PHP compute the value.
> >>>
> >>> In moving this system to erlang + mnesia do I have to worry about the
> >>> same issue?  It seems I will have to store the thousands of items in a
> >>> set in a mnesia table, retrieve all the rows, and have erlang compute
> >>> the value.  There's no way to do the iteration inside mnesia like
> >>> there is in mysql is there?
> >>
> >> You do this "stored procedure" thing in MySQL primarily to avoid
> >> inter-process communication and serialization/deserialization.
> >>
> >> In Erlang+Mnesia case, the Mnesia table runs inside Erlang VM,
> >> therefore avoiding inter-process communication issues. Also,
> >> there is some advantage in not having to perform serialization
> >> and deserialization, although with *ts tables (which Mnesia
> >> is based upon) there is some copying going on anyway.
> >>
> >> So, the short answer is no, you don't do it in Mnesia. Primarily
> >> because your surrounding Erlang code precisely acts like a "stored
> >> procedure" around Mnesia tables, running in the same VM.
> >>
> >>
> >> --
> >> Lev Walkin
> >> vlm@REDACTED
> >> _______________________________________________
> >> erlang-questions mailing list
> >> erlang-questions@REDACTED
> >> http://www.erlang.org/mailman/listinfo/erlang-questions
> >>
> >
> > Optimism is an essential ingredient of innovation. How else can the
> > individual favor change over security?
> >
> >   -- Robert Noyce
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>

Optimism is an essential ingredient of innovation. How else can the
individual favor change over security?

  -- Robert Noyce