[erlang-questions] sending data down the wire in mysql vs. mnesia
Sat Dec 1 23:36:29 CET 2007
Paul Mineiro wrote:
> I don't find this answer satisfactory in the case of distributed
> fragmented mnesia tables.
> I've actually had to rewrite computations in map-reduce form with servers
> on each node doing operations on fragments that are local to the node (I
> still use mnesia for storage for operational benefits) and then collecting
> results (similar to gen_server:multi_call/4, but using pg2 and a custom
> hash function known both to the client and mnesia). I was hoping to
> implement such computations in straight mnesia (less chances for me to
> screw something up) but noticed it was easy to start seeing 20 mbits of
> sustained I/O between my boxes even at low query rates for my application.
> It'd be nice to somehow within mnesia extend the concept of fragmented
> tables to reduce network I/O for incrementally computable
> aggregate functions.
How would you do that with fragmented MySQL tables? Stored procedures
won't work in this case either. Apples and Oranges?
> -- p
> On Sat, 1 Dec 2007, Lev Walkin wrote:
>> Andrew Arrow wrote:
>>> I have a PHP + mysql app now that uses a mysql stored procedure to
>>> avoid sending lots of data "down the wire." That is, the stored
>>> procedure iterates over thousands of rows, determines the value the
>>> app needs and sends back just the value. This avoids the thousands of
>>> rows coming down the wire to have PHP compute the value.
>>> In moving this system to erlang + mnesia do I have to worry about the
>>> same issue? It seems I will have to store the thousands of items in a
>>> set in a mnesia table, retrieve all the rows, and have erlang compute
>>> the value. There's no way to do the iteration inside mnesia like
>>> there is in mysql is there?
>> You do this "stored procedure" thing in MySQL primarily to avoid
>> inter-process communication and serialization/deserialization.
>> In Erlang+Mnesia case, the Mnesia table runs inside Erlang VM,
>> therefore avoiding inter-process communication issues. Also,
>> there is some advantage in not having to perform serialization
>> and deserialization, although with *ts tables (which Mnesia
>> is based upon) there is some copying going on anyway.
>> So, the short answer is no, you don't do it in Mnesia. Primarily
>> because your surrounding Erlang code precisely acts like a "stored
>> procedure" around Mnesia tables, running in the same VM.
>> Lev Walkin
>> erlang-questions mailing list
> Optimism is an essential ingredient of innovation. How else can the
> individual favor change over security?
> -- Robert Noyce
More information about the erlang-questions