[erlang-questions] mnesia async vs sync

Roberto Ostinelli roberto@REDACTED
Mon Mar 14 12:39:28 CET 2016


So,
I took the time to benchmark a little more. I'm hoping that some kind soul
can help me understand this here.

As said, fundamentally I don't see any speed difference when using
sync_dirty and async_dirty.
The more nodes you add, the slower insertions will be even if the docs for
async_dirty state [1]:

"By passing the same "fun" as an argument to the function
mnesia:sync_dirty(Fun [, Args]), it is performed in almost the same context
as the function mnesia:async_dirty/1,2. The difference is that the
operations are performed synchronously. The caller waits for the updates to
be performed on all active replicas."
"The functions wait for the operation to be performed on one node but not
the others. If the table resides locally, no waiting occurs."


These are the different types of write that I've used and their benchmarks:


*mnesia:dirty_write/1*

write_bench(Count) ->
    F = fun(N) ->
        mnesia:dirty_write(#test_table{key = N, value = N})
    end,
    lists:foreach(F, lists:seq(1, Count)).


1> mnesia_test:write(['1@REDACTED']).
Written in 1.494185 sec, at a rate of 334630.5845661682/sec

1> mnesia_test:write(['1@REDACTED','2@REDACTED']).
Written in 2.865135 sec, at a rate of 174511.84673671573/sec

1> mnesia_test:write(['1@REDACTED','2@REDACTED','3@REDACTED','4@REDACTED
']).
Written in 77.847125 sec, at a rate of 6422.844774293207/sec



*mnesia:activity/2 with sync_dirty*

write_bench(Count) ->
    F = fun(N) ->
        mnesia:activity(sync_dirty, fun() ->
            mnesia:dirty_write(#test_table{key = N, value = N})
        end)
    end,
    lists:foreach(F, lists:seq(1, Count)).


1> mnesia_test:write(['1@REDACTED']).
Written in 1.79734 sec, at a rate of 278188.8791213682/sec

1> mnesia_test:write(['1@REDACTED','2@REDACTED']).
Written in 3.570957 sec, at a rate of 140018.48804116095/sec

1> mnesia_test:write(['1@REDACTED','2@REDACTED','3@REDACTED','4@REDACTED
']).
Written in 92.945978 sec, at a rate of 5379.468921183443/sec



*mnesia:activity/2 with async_dirty*

write_bench(Count) ->
    F = fun(N) ->
        mnesia:activity(async_dirty, fun() ->
            mnesia:dirty_write(#test_table{key = N, value = N})
        end)
    end,
    lists:foreach(F, lists:seq(1, Count)).

1> mnesia_test:write(['1@REDACTED']).
Written in 1.638162 sec, at a rate of 305220.11864516453/sec

1> mnesia_test:write(['1@REDACTED','2@REDACTED']).
Written in 3.255289 sec, at a rate of 153596.19376344158/sec

1> mnesia_test:write(['1@REDACTED','2@REDACTED','3@REDACTED','4@REDACTED
']).
Written in 98.841335 sec, at a rate of 5058.61237102878/sec




*mnesia:async_dirty/1*

write_bench(Count) ->
    F = fun(N) ->
        mnesia:async_dirty(fun() ->
            mnesia:dirty_write(#test_table{key = N, value = N})
        end)
    end,
    lists:foreach(F, lists:seq(1, Count)).

1> mnesia_test:write(['1@REDACTED']).
Written in 1.688114 sec, at a rate of 296188.5275520492/sec

1> mnesia_test:write(['1@REDACTED','2@REDACTED']).
Written in 3.166962 sec, at a rate of 157880.0124535754/sec

1> mnesia_test:write(['1@REDACTED','2@REDACTED','3@REDACTED','4@REDACTED
']).
Written in 93.074646 sec, at a rate of 5372.032250329483/sec



Any ideas?

Best,
r.

[1] http://erlang.org/doc/apps/mnesia/Mnesia_chap4.html


%%%%%%%%%%%%%%%%%%%%%%% FULL MODULE %%%%%%%%%%%%%%%%%%%%%%%

-module(mnesia_test).
-compile(export_all).

-record(test_table, {
    key = undefined :: any(),
    value = undefined :: any()
}).

write(Nodes) ->
    Count = 500000,

    connect_nodes(Nodes),
    start_mnesia_on(Nodes),
    create_table_in(Nodes),

    {Time, _} = timer:tc(?MODULE, write_bench, [Count]),

    io:format("Written in ~p sec, at a rate of ~p/sec~n", [
        Time/1000000,
        Count/Time*1000000
    ]).

connect_nodes(Nodes) ->
    [true = net_kernel:connect_node(Node) || Node <- Nodes].

start_mnesia_on(Nodes) ->
    [rpc:call(Node, application, start, [mnesia]) || Node <- Nodes].

create_table_in(Nodes) ->
    mnesia:change_config(extra_db_nodes, Nodes),
    mnesia:create_table(test_table, [
        {type, set},
        {ram_copies, Nodes},
        {attributes, record_info(fields, test_table)},
        {storage_properties, [{ets, [{read_concurrency, true}]}]}
    ]).

write_bench(Count) ->
    F = fun(N) ->
        mnesia:activity(ets, fun() ->
            mnesia:dirty_write(#test_table{key = N, value = N})
        end)
    end,
    lists:foreach(F, lists:seq(1, Count)).

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%


On Fri, Mar 11, 2016 at 12:25 AM, Ryan <zzantozz@REDACTED> wrote:

> On 03/10/2016 05:10 PM, Roberto Ostinelli wrote:
>
> Thank you for your input.
>
> This is interesting since the documentation [1] states:
>
> "By passing the same "fun" as an argument to the function mnesia:sync_dirty(Fun
> [, Args]) <http://erlang.org/doc/man/mnesia.html#sync_dirty-2>, it is
> performed in almost the same context as the function
> mnesia:async_dirty/1,2
> <http://erlang.org/doc/man/mnesia.html#async_dirty-2>. The difference is
> that the operations are performed synchronously. The caller waits for the
> updates to be performed on all active replicas."
>
> Maybe I'm not reading this right?
>
> I agree that those docs imply an async_dirty call doesn't wait for
> anything beyond the one node. It specifically says, under async_dirty,
> that, "The functions wait for the operation to be performed on one node but
> not the others. If the table resides locally, no waiting occurs."
>
> I may have misinterpreted the code I was looking at, or the docs could be
> misleading. I'll wait eagerly for someone more well-informed to come along
> and shed some light on the situation.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20160314/0f16c88c/attachment.htm>


More information about the erlang-questions mailing list