[erlang-questions] sending data down the wire in mysql vs.mnesia
Tue Dec 4 16:32:29 CET 2007
By the way, this was a helpful exchange. Thank-you for including the list
on your replies.
[mailto:] On Behalf Of Lev Walkin
Sent: Saturday, December 01, 2007 16:57
To: Paul Mineiro
Cc: erlang-questions; Andrew Arrow
Subject: Re: [erlang-questions] sending data down the wire in mysql
Paul Mineiro wrote:
> With a fragmented MySQL table, I can execute a stored procedure
> on each MySQL fragment, and the reduce the collection of results in the
> client. The only thing that travels over the wire is the collection of
> results, one for each fragment.
> I'm looking for something similar from Mnesia. Erlang has all the
> building blocks to put it together (so I have), but I can see an
> opportunity to come up with something standard within Mnesia which is
> reasonably general and would return me to a more location-transparent
> style of programming.
My point though is that Erlang _is_ the stored procedure around Mnesia.
What one does is runs a function mapped across several Mnesia nodes,
doing local computations, getting single results back. No difference.
> -- p
> On Sat, 1 Dec 2007, Lev Walkin wrote:
>> Paul Mineiro wrote:
>>> I don't find this answer satisfactory in the case of distributed
>>> fragmented mnesia tables.
>>> I've actually had to rewrite computations in map-reduce form with
>>> on each node doing operations on fragments that are local to the node (I
>>> still use mnesia for storage for operational benefits) and then
>>> results (similar to gen_server:multi_call/4, but using pg2 and a custom
>>> hash function known both to the client and mnesia). I was hoping to
>>> implement such computations in straight mnesia (less chances for me to
>>> screw something up) but noticed it was easy to start seeing 20 mbits of
>>> sustained I/O between my boxes even at low query rates for my
>>> It'd be nice to somehow within mnesia extend the concept of fragmented
>>> tables to reduce network I/O for incrementally computable
>>> aggregate functions.
>> How would you do that with fragmented MySQL tables? Stored procedures
>> won't work in this case either. Apples and Oranges?
>>> -- p
>>> On Sat, 1 Dec 2007, Lev Walkin wrote:
>>>> Andrew Arrow wrote:
>>>>> I have a PHP + mysql app now that uses a mysql stored procedure to
>>>>> avoid sending lots of data "down the wire." That is, the stored
>>>>> procedure iterates over thousands of rows, determines the value the
>>>>> app needs and sends back just the value. This avoids the thousands of
>>>>> rows coming down the wire to have PHP compute the value.
>>>>> In moving this system to erlang + mnesia do I have to worry about the
>>>>> same issue? It seems I will have to store the thousands of items in a
>>>>> set in a mnesia table, retrieve all the rows, and have erlang compute
>>>>> the value. There's no way to do the iteration inside mnesia like
>>>>> there is in mysql is there?
>>>> You do this "stored procedure" thing in MySQL primarily to avoid
>>>> inter-process communication and serialization/deserialization.
>>>> In Erlang+Mnesia case, the Mnesia table runs inside Erlang VM,
>>>> therefore avoiding inter-process communication issues. Also,
>>>> there is some advantage in not having to perform serialization
>>>> and deserialization, although with *ts tables (which Mnesia
>>>> is based upon) there is some copying going on anyway.
>>>> So, the short answer is no, you don't do it in Mnesia. Primarily
>>>> because your surrounding Erlang code precisely acts like a "stored
>>>> procedure" around Mnesia tables, running in the same VM.
>>>> Lev Walkin
>>>> erlang-questions mailing list
>>> Optimism is an essential ingredient of innovation. How else can the
>>> individual favor change over security?
>>> -- Robert Noyce
>> erlang-questions mailing list
> Optimism is an essential ingredient of innovation. How else can the
> individual favor change over security?
> -- Robert Noyce
erlang-questions mailing list
More information about the erlang-questions