<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=SV link=blue vlink=purple style='word-wrap:break-word'><div class=WordSection1><p class=MsoNormal><span lang=EN-US style='mso-fareast-language:EN-US'>One thing to be aware of when doing benchmarks on ETS set vs ordered_set is that ordered_set may get boosted cache performance if the keys are accessed in term order. The same internal tree nodes are visited over and over again as term adjacent key are most often close to each other in the tree.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='mso-fareast-language:EN-US'>So if that is not your typical traffic case, the benchmark should access the keys in some random order.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='mso-fareast-language:EN-US'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='mso-fareast-language:EN-US'>/Sverker<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='mso-fareast-language:EN-US'> <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='mso-fareast-language:EN-US'><o:p> </o:p></span></p><div><div style='border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm'><p class=MsoNormal style='margin-left:65.2pt'><b><span lang=EN-US>From:</span></b><span lang=EN-US> erlang-questions <erlang-questions-bounces@erlang.org> <b>On Behalf Of </b>Jacob<br><b>Sent:</b> den 26 februari 2021 16:35<br><b>To:</b> Dan Gudmundsson <dangud@gmail.com><br><b>Cc:</b> Questions erlang-questions <erlang-questions@erlang.org><br><b>Subject:</b> Re: Performance of mnesia:select/2<o:p></o:p></span></p></div></div><p class=MsoNormal style='margin-left:65.2pt'><o:p> </o:p></p><p style='margin-left:65.2pt'><o:p> </o:p></p><div><p class=MsoNormal style='margin-left:65.2pt'>On 2/26/21 3:53 PM, Dan Gudmundsson wrote:<o:p></o:p></p></div><blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'><div><p class=MsoNormal style='margin-left:65.2pt'>Interesting, and times do you get for 500 ets lookup on that data?<o:p></o:p></p></div></blockquote><p style='margin-left:65.2pt'>Oh, I really should have tested that as well:<o:p></o:p></p><p style='margin-left:65.2pt'>timer:tc(fun () -> lists:map(fun(K) -> ets:lookup(t, K) end, Keys) end).<o:p></o:p></p><p style='margin-left:65.2pt'>* using lookups: [set, {keypos, 2}]: 3148 us<br>* using lookup: [ordered_set, {keypos, 2}]: 3423 us<o:p></o:p></p><p style='margin-left:65.2pt'>I have repeated the select/pattern runs to have comparable results:<o:p></o:p></p><p style='margin-left:65.2pt'>* using pattern, [ordered_set, {keypos, 2}]: 3173 us<br>* using pattern, [set, {keypos, 2}]: 8091 us<o:p></o:p></p><p style='margin-left:65.2pt'>Note that there was quite a high jitter on the measurements so I have compute the mean value of 3 measurements each.<o:p></o:p></p><p style='margin-left:65.2pt'>So lookups and pattern/ordered_set were too close to really come to a winner here. I had the impression that the variance was slightly higher with lookup, but digging into this would require a better measurement setup than what I have been using.<o:p></o:p></p><p style='margin-left:65.2pt'><o:p> </o:p></p><blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'><p class=MsoNormal style='margin-left:65.2pt'> <o:p></o:p></p><div><div><p class=MsoNormal style='margin-left:65.2pt'>On Fri, Feb 26, 2021 at 3:12 PM Jacob <<a href="mailto:jacob01@gmx.net">jacob01@gmx.net</a>> wrote:<o:p></o:p></p></div><blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'><p class=MsoNormal style='margin-left:65.2pt'>Hi,<br><br>assuming that the match spec compiler does clever things with patterns,<br>I'd use select with the following<br><br> MatchExpression = [ {{'_', K, '_'}, [], ['$_']} || K <- Keys ]<br><br>If did some quick measurements with plain ETS, timer:tc and 500 keys out<br>of 1000000 table entries and got:<br><br> * using pattern, [ordered_set]: 4532605 us<br> * using pattern, [set] : 4645525 us<br> * using pattern, [ordered_set, {keypos, 2}]: 3826 us (!!!)<br> * using pattern, [set, {keypos, 2}]: 5714 us (!!!)<br><br> * using guards, [ordered_set]: 12542928 us<br> * using guards, [set]: 12310452 us<br> * using guards, [ordered_set, {keypos, 2}]: 12365477 us<br> * using guards, [set, {keypos, 2}]: 12277839 us<br><br>I have initialised the DB with [ ets:insert(t, {N, N,<br>integer_to_list(N)}) || N <- lists:seq(1, 1000000) ].<br><br>I don't know though, how this will translate to Mnesia, but I'd give<br>select with pattern on the primary key a try.<br><br>/Jacob<br><br><br>On 2/26/21 11:03 AM, Vance Shipley wrote:<br>> If I need to lookup a list of keys which is the better approach? Why?<br>><br>> Fselect = fun(Keys) -><br>> MatchHead = {'_', '$1', '$2'},<br>> F = fun(Key) -><br>> {'=:=', '$1', Key}<br>> end,<br>> MatchConditions = [list_to_tuple(['or' | lists:map(F, Keys)]),<br>> MatchBody = ['$_'],<br>> MatchFunction = {MatchHead, MatchConditions, MatchBody},<br>> MatchExpression = [MatchFunction],<br>> mnesia:select(Table, MatchExpression)<br>> end,<br>> mnesia:transaction(Fselect, [Keys]).<br>><br>> Fread = fun F([Key | T], Acc) -><br>> [R] = mnesia:read(Table, Key),<br>> F(T, [R | Acc]);<br>> F([], Acc) -><br>> lists:reverse(Acc)<br>> end,<br>> mnesia:transaction(Fread, [Keys. []]).<br>><br>><br>> --<br>> -Vance<o:p></o:p></p></blockquote></div></blockquote></div></body></html>