[erlang-questions] mnesia async vs sync

[Dan Gudmundsson]

Try using mnesia:write instead of mnesia:dirty_write inside your fun.


On Mon, Mar 14, 2016 at 12:39 PM Roberto Ostinelli <roberto@REDACTED>
wrote:

> So,
> I took the time to benchmark a little more. I'm hoping that some kind soul
> can help me understand this here.
>
> As said, fundamentally I don't see any speed difference when using
> sync_dirty and async_dirty.
> The more nodes you add, the slower insertions will be even if the docs for
> async_dirty state [1]:
>
> "By passing the same "fun" as an argument to the function
> mnesia:sync_dirty(Fun [, Args]), it is performed in almost the same context
> as the function mnesia:async_dirty/1,2. The difference is that the
> operations are performed synchronously. The caller waits for the updates to
> be performed on all active replicas."
>
> "The functions wait for the operation to be performed on one node but not
> the others. If the table resides locally, no waiting occurs."
>
>
> These are the different types of write that I've used and their benchmarks:
>
>
> *mnesia:dirty_write/1*
>
> write_bench(Count) ->
>     F = fun(N) ->
>         mnesia:dirty_write(#test_table{key = N, value = N})
>     end,
>     lists:foreach(F, lists:seq(1, Count)).
>
>
> 1> mnesia_test:write(['1@REDACTED']).
> Written in 1.494185 sec, at a rate of 334630.5845661682/sec
>
> 1> mnesia_test:write(['1@REDACTED','2@REDACTED']).
> Written in 2.865135 sec, at a rate of 174511.84673671573/sec
>
> 1> mnesia_test:write(['1@REDACTED','2@REDACTED','3@REDACTED','
> 4@REDACTED']).
> Written in 77.847125 sec, at a rate of 6422.844774293207/sec
>
>
>
> *mnesia:activity/2 with sync_dirty*
>
> write_bench(Count) ->
>     F = fun(N) ->
>         mnesia:activity(sync_dirty, fun() ->
>             mnesia:dirty_write(#test_table{key = N, value = N})
>         end)
>     end,
>     lists:foreach(F, lists:seq(1, Count)).
>
>
> 1> mnesia_test:write(['1@REDACTED']).
> Written in 1.79734 sec, at a rate of 278188.8791213682/sec
>
> 1> mnesia_test:write(['1@REDACTED','2@REDACTED']).
> Written in 3.570957 sec, at a rate of 140018.48804116095/sec
>
> 1> mnesia_test:write(['1@REDACTED','2@REDACTED','3@REDACTED','
> 4@REDACTED']).
> Written in 92.945978 sec, at a rate of 5379.468921183443/sec
>
>
>
> *mnesia:activity/2 with async_dirty*
>
> write_bench(Count) ->
>     F = fun(N) ->
>         mnesia:activity(async_dirty, fun() ->
>             mnesia:dirty_write(#test_table{key = N, value = N})
>         end)
>     end,
>     lists:foreach(F, lists:seq(1, Count)).
>
> 1> mnesia_test:write(['1@REDACTED']).
> Written in 1.638162 sec, at a rate of 305220.11864516453/sec
>
> 1> mnesia_test:write(['1@REDACTED','2@REDACTED']).
> Written in 3.255289 sec, at a rate of 153596.19376344158/sec
>
> 1> mnesia_test:write(['1@REDACTED','2@REDACTED','3@REDACTED','
> 4@REDACTED']).
> Written in 98.841335 sec, at a rate of 5058.61237102878/sec
>
>
>
>
> *mnesia:async_dirty/1*
>
> write_bench(Count) ->
>     F = fun(N) ->
>         mnesia:async_dirty(fun() ->
>             mnesia:dirty_write(#test_table{key = N, value = N})
>         end)
>     end,
>     lists:foreach(F, lists:seq(1, Count)).
>
> 1> mnesia_test:write(['1@REDACTED']).
> Written in 1.688114 sec, at a rate of 296188.5275520492/sec
>
> 1> mnesia_test:write(['1@REDACTED','2@REDACTED']).
> Written in 3.166962 sec, at a rate of 157880.0124535754/sec
>
> 1> mnesia_test:write(['1@REDACTED','2@REDACTED','3@REDACTED','
> 4@REDACTED']).
> Written in 93.074646 sec, at a rate of 5372.032250329483/sec
>
>
>
> Any ideas?
>
> Best,
> r.
>
> [1] http://erlang.org/doc/apps/mnesia/Mnesia_chap4.html
>
>
> %%%%%%%%%%%%%%%%%%%%%%% FULL MODULE %%%%%%%%%%%%%%%%%%%%%%%
>
> -module(mnesia_test).
> -compile(export_all).
>
> -record(test_table, {
>     key = undefined :: any(),
>     value = undefined :: any()
> }).
>
> write(Nodes) ->
>     Count = 500000,
>
>     connect_nodes(Nodes),
>     start_mnesia_on(Nodes),
>     create_table_in(Nodes),
>
>     {Time, _} = timer:tc(?MODULE, write_bench, [Count]),
>
>     io:format("Written in ~p sec, at a rate of ~p/sec~n", [
>         Time/1000000,
>         Count/Time*1000000
>     ]).
>
> connect_nodes(Nodes) ->
>     [true = net_kernel:connect_node(Node) || Node <- Nodes].
>
> start_mnesia_on(Nodes) ->
>     [rpc:call(Node, application, start, [mnesia]) || Node <- Nodes].
>
> create_table_in(Nodes) ->
>     mnesia:change_config(extra_db_nodes, Nodes),
>     mnesia:create_table(test_table, [
>         {type, set},
>         {ram_copies, Nodes},
>         {attributes, record_info(fields, test_table)},
>         {storage_properties, [{ets, [{read_concurrency, true}]}]}
>     ]).
>
> write_bench(Count) ->
>     F = fun(N) ->
>         mnesia:activity(ets, fun() ->
>             mnesia:dirty_write(#test_table{key = N, value = N})
>         end)
>     end,
>     lists:foreach(F, lists:seq(1, Count)).
>
> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
>
>
> On Fri, Mar 11, 2016 at 12:25 AM, Ryan <zzantozz@REDACTED> wrote:
>
>> On 03/10/2016 05:10 PM, Roberto Ostinelli wrote:
>>
>> Thank you for your input.
>>
>> This is interesting since the documentation [1] states:
>>
>> "By passing the same "fun" as an argument to the function mnesia:sync_dirty(Fun
>> [, Args]) <http://erlang.org/doc/man/mnesia.html#sync_dirty-2>, it is
>> performed in almost the same context as the function
>> mnesia:async_dirty/1,2
>> <http://erlang.org/doc/man/mnesia.html#async_dirty-2>. The difference is
>> that the operations are performed synchronously. The caller waits for the
>> updates to be performed on all active replicas."
>>
>> Maybe I'm not reading this right?
>>
>> I agree that those docs imply an async_dirty call doesn't wait for
>> anything beyond the one node. It specifically says, under async_dirty,
>> that, "The functions wait for the operation to be performed on one node but
>> not the others. If the table resides locally, no waiting occurs."
>>
>> I may have misinterpreted the code I was looking at, or the docs could be
>> misleading. I'll wait eagerly for someone more well-informed to come along
>> and shed some light on the situation.
>>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20160314/ec494603/attachment.htm>