<div dir="ltr">Rewriting into a module and passing through eministat:<br><div><br>-module(maps_get).<br><br>-export([datasets/0, t/0]).<br><br>without(Map) -><br> K1 = {a, 1},<br> K2 = {a, 2},<br> K3 = {a, 3},<br> #{K1 := V1, K2 := V2, K3 := V3} = Map,<br> {V1, V2, V3}.<br><br>with(Map) -><br> {maps:get({a,1}, Map),<br> maps:get({a,2}, Map),<br> maps:get({a,3}, Map)}.<br><br>datasets() -><br> Map = #{{a, 1} => 100, {a, 2} => 200, {a, 3} => 300},<br> [eministat:s("without", fun() -> without(Map) end, 100, us),<br> eministat:s("with", fun() -> with(Map) end, 100, us)].<br><br>t() -><br> [Without | With] = datasets(),<br> eministat:x(95.0, Without, With).<br><br></div><div>yields the following result:<br><br>9> maps_get:t().<br>x without<br>+ with<br>+--------------------------------------------------------------------------+<br>| * * + x x|<br>| * * |<br>| * x |<br>| * x |<br>| * x |<br>| * x |<br>| * x |<br>| * x |<br>| * x |<br>| * x |<br>| * x |<br>| * x |<br>| * x |<br>| * x |<br>| * x |<br>| * x |<br>| * x |<br>| * x |<br>| * x |<br>| * x |<br>| * x |<br>| * x |<br>| * x |<br>| * x |<br>| * x |<br>| * x |<br>| * x |<br>| * x |<br>| * x |<br>| * x |<br>| * x |<br>| * x |<br>| * x |<br>| * x |<br>| * x |<br>| + x |<br>| + x |<br>| + x |<br>| + x |<br>| + x |<br>| + x |<br>| + x |<br>| + x |<br>| + x |<br>| + x |<br>| + x |<br>| + x |<br>| + x |<br>| + x |<br>| + x |<br>| + x |<br>| + x |<br>| + x |<br>| + x |<br>| + x |<br>| + x |<br>| + x |<br>| + x |<br>| + x |<br>| + x |<br>| + x |<br>| + x |<br>| + x |<br>| + |<br>| + |<br>| + |<br>| + |<br>| + |<br>| + |<br>| + |<br>| + |<br>| + |<br>| + |<br>| + |<br>| + |<br>| + |<br>| + |<br>| + |<br>| + |<br>| + |<br>| + |<br>| + |<br>| + |<br>| + |<br>| + |<br>| + |<br>| + |<br>| + |<br>| + |<br>| + |<br>| + |<br>| + |<br>| + |<br>| + |<br>| + |<br>| + |<br>| + |<br>||________A_M_______| |<br>| |_A_| |<br>+--------------------------------------------------------------------------+<br>Dataset: x N=100 CI=95.0000<br>Statistic Value [ Bias] (Bootstrapped LB‥UB)<br>Min: 0.00000e+0<br>1st Qu. 0.00000e+0<br>Median: 1.00000<br>3rd Qu. 1.00000<br>Max: 10.0000<br>Average: 0.820000 [ -8.45000e-4] ( 0.630000 ‥ 1.23000)<br>Std. Dev: 1.33621 [ -7.87598e-2] ( 0.482418 ‥ 2.33919)<br><br>Outliers: 0/2 = 2 (μ=0.819155, σ=1.25745)<br> Outlier variance: 0.989942 (severe, the data set is probably unusable)<br><br>------<br><br>Dataset: + N=100 CI=95.0000<br>Statistic Value [ Bias] (Bootstrapped LB‥UB)<br>Min: 0.00000e+0<br>1st Qu. 0.00000e+0<br>Median: 0.00000e+0<br>3rd Qu. 0.00000e+0<br>Max: 2.00000<br>Average: 4.00000e-2 [ -4.33000e-4] ( 0.00000e+0 ‥ 0.110000)<br>Std. Dev: 0.242878 [ -2.09268e-2] ( 0.100000 ‥ 0.455716)<br><br>Outliers: 0/3 = 3 (μ=3.95670e-2, σ=0.221952)<br> Outlier variance: 0.989996 (severe, the data set is probably unusable)<br><br>Difference at 95.0% confidence<br> -0.780000 ± 0.266188<br> -95.1220% ± 32.4620%<br> (Student's t, pooled s = 0.960324)<br>------<br><br></div><div>The result is largely unusable. We can't measure at the microsecond level precisely enough to conclude anything. You will probably have to run several rounds of the selection in order to get a useful value which isn't severely pertubed by e.g. garbage collection or taking an interrupt on the OSX machine I ran this on. You will have to do something about the outlier variance.<br><br></div><div>Also note that the confidence intervals overlaps a lot. Even though we have significant difference at 95.0% confidence, the difference is probably affected too much by measurement errors to be conclusive. A test could easily run 1000 rounds of the operation:<br><br></div><div>Altering the code to 10000 rounds and 25 trials yields the following code:<br><br>-module(maps_get).<br><br>-export([datasets/0, t/0]).<br><br>without(0, _) -> ok;<br>without(K, Map) -><br> K1 = {a, 1},<br> K2 = {a, 2},<br> K3 = {a, 3},<br> #{K1 := V1, K2 := V2, K3 := V3} = Map,<br> {V1, V2, V3},<br> without(K-1, Map).<br><br>with(0, _) -> ok;<br>with(K, Map) -><br> {maps:get({a,1}, Map),<br> maps:get({a,2}, Map),<br> maps:get({a,3}, Map)},<br> with(K-1, Map).<br><br>datasets() -><br> Map = #{{a, 1} => 100, {a, 2} => 200, {a, 3} => 300},<br> [eministat:s("without", fun() -> without(10000, Map) end, 25, us),<br> eministat:s("with", fun() -> with(10000, Map) end, 25, us)].<br><br>t() -><br> [Without | With] = datasets(),<br> eministat:x(95.0, Without, With).<br><br></div><div>And the following output from eministat:<br><br>11> maps_get:t().<br>x without<br>+ with<br>+--------------------------------------------------------------------------+<br>| x xx x x + +|<br>| x x + |<br>| x x + |<br>| x + |<br>| x + |<br>| x + |<br>| x + |<br>| x + |<br>| x + |<br>| x + |<br>| x + |<br>| x + |<br>| x + |<br>| x + |<br>| x + |<br>| x + |<br>| x + |<br>| x + |<br>| x + |<br>| + |<br>| + |<br>| + |<br>| + |<br>| + |<br>||MA__| |<br>| |A| |<br>+--------------------------------------------------------------------------+<br>Dataset: x N=25 CI=95.0000<br>Statistic Value [ Bias] (Bootstrapped LB‥UB)<br>Min: 408.000<br>1st Qu. 410.000<br>Median: 411.000<br>3rd Qu. 412.000<br>Max: 460.000<br>Average: 418.880 [ -2.47040e-2] ( 413.760 ‥ 426.680)<br>Std. Dev: 16.3078 [ -0.661582] ( 10.6226 ‥ 21.5513)<br><br>Outliers: 0/6 = 6 (μ=418.855, σ=15.6462)<br> Outlier variance: 0.149444 (moderate)<br><br>------<br><br>Dataset: + N=25 CI=95.0000<br>Statistic Value [ Bias] (Bootstrapped LB‥UB)<br>Min: 860.000<br>1st Qu. 860.000<br>Median: 861.000<br>3rd Qu. 861.000<br>Max: 902.000<br>Average: 862.440 [ -9.31200e-3] ( 860.680 ‥ 869.040)<br>Std. Dev: 8.26680 [ -1.72872] ( 0.583095 ‥ 16.8891)<br><br>Outliers: 0/1 = 1 (μ=862.431, σ=6.53808)<br> Outlier variance: 3.84000e-2 (slight)<br><br>Difference at 95.0% confidence<br> 443.560 ± 7.35359<br> 105.892% ± 1.75554%<br> (Student's t, pooled s = 12.9283)<br>------<br><br>ok<br>12><br><br></div><div>Conclusion: maps:get/2 is likely around 443us (+/- 7.4us) slower in this test than ignoring it. But also note that in both cases the compiler is warning us about term construction which is unused. So the compiler might optimize stuff away. Though running a loop with no effect as a control ticks out at 170us, so it is unlikely.<br><br></div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, May 18, 2016 at 1:55 PM, Dmitry Belyaev <span dir="ltr"><<a href="mailto:be.dmitry@gmail.com" target="_blank">be.dmitry@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div>As a first step try a compiled version, not a test in a shell. <br><br><div class="gmail_quote"><span class="">On 18 May 2016 1:35:49 PM AEST, Avinash Dhumane <<a href="mailto:nistrigunya@gmail.com" target="_blank">nistrigunya@gmail.com</a>> wrote:</span><blockquote class="gmail_quote" style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr"><span class=""><div>For my algorithm trading application, gains even in 5 to 10 microseconds matter. <br><br>I used maps extensively to improve expressiveness of the design. <br><br>But, here is a small observation about maps. I am not sure if the performance variation seen is indeed true, or if my test case is wrong. In either case, please point me to the right way of efficiently using of maps. <br><br>avinash@veda:~/tws$ erl<br>Erlang/OTP 18 [erts-7.3] [source] [64-bit] [smp:2:2] [async-threads:10] [hipe] [kernel-poll:false]<br><br>Eshell V7.3 (abort with ^G)<br><br></div><div>%%% compute average of timer:tc() return values (timings)<br><br></div><div>1> Avg = fun(PerfList) -> Sum = lists:sum(lists:map(fun({Time, _}) -> Time end, PerfList)), Len = erlang:length(PerfList), Sum div Len end.<br>#Fun<erl_eval.6.50752066><br><br></div><div>%%% a test map of size 3<br><br></div></span><div>2> Map = #{{a, 1} => 100, {a, 2} =>
; 200,
{a, 3} => 300}.<div><div class="h5"><br>#{{a,1} => 100,{a,2} => 200,{a,3} => 300}<br><br></div></div></div><div><div class="h5"><div>%%% performance: without maps:get()<br><br></div><div>3> PerfList1 = lists:map(fun(_) -> timer:tc(fun() -> K1 = {a, 1}, K2 = {a, 2}, K3 = {a, 3}, #{K1 := V1, K2 := V2, K3 := V3} = Map, {V1, V2, V3} end) end, lists:seq(1,100)).<br>[{111,{100,200,300}},<br> {81,{100,200,300}},<br> {88,{100,200,300}},<br></div>.....details deleted.....<br><div> {91,{100,200,...}},<br> {83,{100,...}},<br> {85,{...}},<br> {107,...},<br> {...}|...]<br><br></div><div>%%% performance: with maps:get()<br><br></div><div>4> PerfList2 = lists:map(fun(_) -> timer:tc(fun() -> {maps:get({a,1}, Map), maps:get({a,2}, Map), maps:get({a,3}, Map)} end) end, lists:seq(1, 100)).<br>[{13,{100,200,300}},<br> {12,{100,200,300}},<br> {33,{100,200,300}},<br>.....details deleted.....<br> {13,{100,200,...}},<br> {12,{100,...}},<br> {12,{...}},<br> {12,...},<br> {...}|...]<br><br></div><div>%%% compare performance: maps:get() is 7-times faster<br><br></div><div>5> {Avg(PerfList1), Avg(PerfList2)}.<br>{89,13}<br>6><br></div></div></div></div>
<p style="margin-top:2.5em;margin-bottom:1em;border-bottom:1px solid #000"></p><pre><hr><span class=""><br>erlang-questions mailing list<br><a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a><br><a href="http://erlang.org/mailman/listinfo/erlang-questions" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a><br></span></pre></blockquote></div><span class="HOEnZb"><font color="#888888"><br>
-- <br>
Best wishes, <br>
Dmitry Belyaev</font></span></div><br>_______________________________________________<br>
erlang-questions mailing list<br>
<a href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a><br>
<a href="http://erlang.org/mailman/listinfo/erlang-questions" rel="noreferrer" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a><br>
<br></blockquote></div><br><br clear="all"><br>-- <br><div class="gmail_signature">J.</div>
</div>