I used a simple thing like:<div><br></div><div><div><div>test_iter(Mod, Fun, Args, Iters) -></div><div> test_iter(Mod, Fun, Args, now(), Iters, Iters).</div><div><br></div><div>test_iter(_Mod, _Fun, _Args, Start, Iters, 0) -></div>
<div> Iters/(timer:now_diff(now(), Start)/1000000);</div><div><br></div><div>test_iter(Mod, Fun, Args, Start, Iters, CountDown) -></div><div> erlang:apply(Mod, Fun, Args),</div><div> test_iter(Mod, Fun, Args, Start, Iters, CountDown-1).</div>
</div><div><br></div><div>And was just looking at total iterations per sec. I would just repeat this several times until I found a relatively stable reading. Sure, there was variation, but I'm looking to simulate the pattern I use in this library, which is iterating through and copying many small bits of text (json keys and values.)</div>
<div><br></div><div>Since there was such a large difference in overall performance (string being more than twice as fast), I didn't feel the need to be more precise before posing the question.</div><div><br></div><div>
E.g.<br><br></div><div><div>20> json:test_iter(json, test_bin_copy, [<<"hi, my name is erik">>], 1000).</div><div>232018.5614849188</div><div>21> json:test_iter(json, test_bin_copy, [<<"hi, my name is erik">>], 1000).</div>
<div>224870.69934787496</div><div>22> json:test_iter(json, test_bin_copy, [<<"hi, my name is erik">>], 1000).</div><div>226193.16896629723</div><div>23> </div><div>23> json:test_iter(json, test_string_copy, ["hi, my name is erik"], 100000).</div>
<div>650732.3993154295</div><div>24> json:test_iter(json, test_string_copy, ["hi, my name is erik"], 100000).</div><div>608076.4716970806</div><div>25> json:test_iter(json, test_string_copy, ["hi, my name is erik"], 100000).</div>
<div>567359.7912115968</div></div><div><br></div><div>Many of the follow up observations and questions have been stimulating, so I'm now interested as well in a more detailed analysis.</div><div><br></div><div>However, in the end what I'm looking at is the differences in performance between list and binary string processing under what I believe is idiomatic Erlang for such problems.</div>
<div><br></div><div>Thanks,</div><div><br></div><div>Erik.</div><br><div class="gmail_quote">On Tue, Oct 23, 2012 at 1:07 AM, Martynas Pumputis <span dir="ltr"><<a href="mailto:martynasp@gmail.com" target="_blank">martynasp@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi,<br>
<br>
Could you show the exact steps of your simulation? Binary version should be faster, because some extra memory allocation is avoided per each iteration and large binaries aren't being copied.<br>
<br>
Take a look at: <a href="http://www.erlang.org/doc/efficiency_guide/binaryhandling.html" target="_blank">http://www.erlang.org/doc/<u></u>efficiency_guide/<u></u>binaryhandling.html</a><br>
<br>
Martynas<div><div class="h5"><br>
<br>
On 10/23/2012 12:18 AM, Erik Pearson wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="h5">
Hi,<br>
I've read from advice given many years ago that processing binaries byte<br>
by byte (e.g. a recursive parser), performance is better using a list to<br>
accumulate the bytes, rather than using binary concatenation. So<br>
[B|Accum] rather than <<Accum/binary, B>>. There seems to be<br>
a consensus however, on the efficiency of Binaries compared to List<br>
strings.<br>
<br>
My own quick test, which was just to copy a list or binary element by<br>
element, showed much better performance for the list version. The test<br>
was basically to pass an arbitrary string or binary, and copy it some<br>
number of thousands of times, and output the complete copies per second.<br>
<br>
I tried list based accumulation for a binary, using binary destructuring<br>
in the function head, and that sped things up, but it was still slower<br>
than the equivalent list string copy.<br>
<br>
Are there any tips for binaries? Of is this not a good use case for<br>
binaries.<br>
<br>
test_bin_copy(Bin) -><br>
test_bin_copy(Bin, <<>>).<br>
test_bin_copy(<<>>, Accum) -><br>
Accum;<br>
test_bin_copy(<<Char, Rest/binary>>, Accum) -><br>
test_bin_copy(Rest, <<Accum/binary, Char>>).<br>
<br>
test_string_copy(Bin) -><br>
test_string_copy(Bin, []).<br>
test_string_copy([], Accum) -><br>
lists:reverse(Accum);<br>
test_string_copy([Char|Rest], Accum) -><br>
test_string_copy(Rest, [Char|Accum]).<br>
<br>
For what its worth this is part of a json module. The current practice<br>
in json libraries seems to favor binaries, so I assumed there were<br>
inherent performance advantages. I can imagine, e.g., that an empty<br>
binary would be stored as a modest sized buffer that would be appended<br>
in place until there was a need to expand it or copy (e.g. if an older<br>
version of it was being appended), and that operations on it would be<br>
fast compared to arbitrary consing (which is however highly optimized.)<br>
<br>
I think some of the favoritism for binaries in json libs is because it<br>
makes it easy to differentiate json strings (as erlang binaries) from<br>
json arrays (as erlang lists), but my implementation is using tagged<br>
tuples to contain each json value, so this is not a concern. Of course<br>
there are the memory concerns, but in my tests any memory concerns with<br>
list char size vs binary bytes is erased by the performance gains.<br>
<br>
I'm sure I've put my foot in my mouth at least once, but, anyway, advice<br>
appreciated.<br>
<br>
Thanks,<br>
Erik.<br>
<br>
<br></div></div>
______________________________<u></u>_________________<br>
erlang-questions mailing list<br>
<a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a><br>
<a href="http://erlang.org/mailman/listinfo/erlang-questions" target="_blank">http://erlang.org/mailman/<u></u>listinfo/erlang-questions</a><br>
<br>
</blockquote>
<br>
</blockquote></div><br></div>