[erlang-questions] sending data down the wire in mysql vs.mnesia

[David Mercer]

By the way, this was a helpful exchange.  Thank-you for including the list
on your replies.

DBM

-----Original Message-----
From: erlang-questions-bounces@REDACTED
[mailto:erlang-questions-bounces@REDACTED] On Behalf Of Lev Walkin
Sent: Saturday, December 01, 2007 16:57
To: Paul Mineiro
Cc: erlang-questions; Andrew Arrow
Subject: Re: [erlang-questions] sending data down the wire in mysql
vs.mnesia

Paul Mineiro wrote:
> With a fragmented MySQL table, I can execute a stored procedure
> on each MySQL fragment, and the reduce the collection of results in the
> client.  The only thing that travels over the wire is the collection of
> results, one for each fragment.
> 
> I'm looking for something similar from Mnesia.  Erlang has all the
> building blocks to put it together (so I have), but I can see an
> opportunity to come up with something standard within Mnesia which is
> reasonably general and would return me to a more location-transparent
> style of programming.

My point though is that Erlang _is_ the stored procedure around Mnesia.
What one does is runs a function mapped across several Mnesia nodes,
doing local computations, getting single results back. No difference.

> -- p
> 
> On Sat, 1 Dec 2007, Lev Walkin wrote:
> 
>> Paul Mineiro wrote:
>>> I don't find this answer satisfactory in the case of distributed
>>> fragmented mnesia tables.
>>>
>>> I've actually had to rewrite computations in map-reduce form with
servers
>>> on each node doing operations on fragments that are local to the node (I
>>> still use mnesia for storage for operational benefits) and then
collecting
>>> results (similar to gen_server:multi_call/4, but using pg2 and a custom
>>> hash function known both to the client and mnesia).  I was hoping to
>>> implement such computations in straight mnesia (less chances for me to
>>> screw something up) but noticed it was easy to start seeing 20 mbits of
>>> sustained I/O between my boxes even at low query rates for my
application.
>>>
>>> It'd be nice to somehow within mnesia extend the concept of fragmented
>>> tables to reduce network I/O for incrementally computable
>>> aggregate functions.
>>
>> How would you do that with fragmented MySQL tables? Stored procedures
>> won't work in this case either. Apples and Oranges?
>>
>>
>>
>>
>>> -- p
>>>
>>> On Sat, 1 Dec 2007, Lev Walkin wrote:
>>>
>>>> Andrew Arrow wrote:
>>>>> I have a PHP + mysql app now that uses a mysql stored procedure to
>>>>> avoid sending lots of data "down the wire."  That is, the stored
>>>>> procedure iterates over thousands of rows, determines the value the
>>>>> app needs and sends back just the value.  This avoids the thousands of
>>>>> rows coming down the wire to have PHP compute the value.
>>>>>
>>>>> In moving this system to erlang + mnesia do I have to worry about the
>>>>> same issue?  It seems I will have to store the thousands of items in a
>>>>> set in a mnesia table, retrieve all the rows, and have erlang compute
>>>>> the value.  There's no way to do the iteration inside mnesia like
>>>>> there is in mysql is there?
>>>> You do this "stored procedure" thing in MySQL primarily to avoid
>>>> inter-process communication and serialization/deserialization.
>>>>
>>>> In Erlang+Mnesia case, the Mnesia table runs inside Erlang VM,
>>>> therefore avoiding inter-process communication issues. Also,
>>>> there is some advantage in not having to perform serialization
>>>> and deserialization, although with *ts tables (which Mnesia
>>>> is based upon) there is some copying going on anyway.
>>>>
>>>> So, the short answer is no, you don't do it in Mnesia. Primarily
>>>> because your surrounding Erlang code precisely acts like a "stored
>>>> procedure" around Mnesia tables, running in the same VM.
>>>>
>>>>
>>>> --
>>>> Lev Walkin
>>>> vlm@REDACTED
>>>> _______________________________________________
>>>> erlang-questions mailing list
>>>> erlang-questions@REDACTED
>>>> http://www.erlang.org/mailman/listinfo/erlang-questions
>>>>
>>> Optimism is an essential ingredient of innovation. How else can the
>>> individual favor change over security?
>>>
>>>   -- Robert Noyce
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://www.erlang.org/mailman/listinfo/erlang-questions
>>
> 
> Optimism is an essential ingredient of innovation. How else can the
> individual favor change over security?
> 
>   -- Robert Noyce

_______________________________________________
erlang-questions mailing list
erlang-questions@REDACTED
http://www.erlang.org/mailman/listinfo/erlang-questions