[erlang-questions] sending data down the wire in mysql vs. mnesia

[Lev Walkin]

Paul Mineiro wrote:
> I don't find this answer satisfactory in the case of distributed
> fragmented mnesia tables.
> 
> I've actually had to rewrite computations in map-reduce form with servers
> on each node doing operations on fragments that are local to the node (I
> still use mnesia for storage for operational benefits) and then collecting
> results (similar to gen_server:multi_call/4, but using pg2 and a custom
> hash function known both to the client and mnesia).  I was hoping to
> implement such computations in straight mnesia (less chances for me to
> screw something up) but noticed it was easy to start seeing 20 mbits of
> sustained I/O between my boxes even at low query rates for my application.
> 
> It'd be nice to somehow within mnesia extend the concept of fragmented
> tables to reduce network I/O for incrementally computable
> aggregate functions.


How would you do that with fragmented MySQL tables? Stored procedures
won't work in this case either. Apples and Oranges?




> -- p
> 
> On Sat, 1 Dec 2007, Lev Walkin wrote:
> 
>> Andrew Arrow wrote:
>>> I have a PHP + mysql app now that uses a mysql stored procedure to
>>> avoid sending lots of data "down the wire."  That is, the stored
>>> procedure iterates over thousands of rows, determines the value the
>>> app needs and sends back just the value.  This avoids the thousands of
>>> rows coming down the wire to have PHP compute the value.
>>>
>>> In moving this system to erlang + mnesia do I have to worry about the
>>> same issue?  It seems I will have to store the thousands of items in a
>>> set in a mnesia table, retrieve all the rows, and have erlang compute
>>> the value.  There's no way to do the iteration inside mnesia like
>>> there is in mysql is there?
>>
>> You do this "stored procedure" thing in MySQL primarily to avoid
>> inter-process communication and serialization/deserialization.
>>
>> In Erlang+Mnesia case, the Mnesia table runs inside Erlang VM,
>> therefore avoiding inter-process communication issues. Also,
>> there is some advantage in not having to perform serialization
>> and deserialization, although with *ts tables (which Mnesia
>> is based upon) there is some copying going on anyway.
>>
>> So, the short answer is no, you don't do it in Mnesia. Primarily
>> because your surrounding Erlang code precisely acts like a "stored
>> procedure" around Mnesia tables, running in the same VM.
>>
>>
>> --
>> Lev Walkin
>> vlm@REDACTED
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://www.erlang.org/mailman/listinfo/erlang-questions
>>
> 
> Optimism is an essential ingredient of innovation. How else can the
> individual favor change over security?
> 
>   -- Robert Noyce