[erlang-questions] list vs binary performarnce, destructuring and consing
Tue Oct 23 17:11:03 CEST 2012
I used a simple thing like:
test_iter(Mod, Fun, Args, Iters) ->
test_iter(Mod, Fun, Args, now(), Iters, Iters).
test_iter(_Mod, _Fun, _Args, Start, Iters, 0) ->
test_iter(Mod, Fun, Args, Start, Iters, CountDown) ->
erlang:apply(Mod, Fun, Args),
test_iter(Mod, Fun, Args, Start, Iters, CountDown-1).
And was just looking at total iterations per sec. I would just repeat this
several times until I found a relatively stable reading. Sure, there was
variation, but I'm looking to simulate the pattern I use in this library,
which is iterating through and copying many small bits of text (json keys
Since there was such a large difference in overall performance (string
being more than twice as fast), I didn't feel the need to be more precise
before posing the question.
20> json:test_iter(json, test_bin_copy, [<<"hi, my name is erik">>], 1000).
21> json:test_iter(json, test_bin_copy, [<<"hi, my name is erik">>], 1000).
22> json:test_iter(json, test_bin_copy, [<<"hi, my name is erik">>], 1000).
23> json:test_iter(json, test_string_copy, ["hi, my name is erik"], 100000).
24> json:test_iter(json, test_string_copy, ["hi, my name is erik"], 100000).
25> json:test_iter(json, test_string_copy, ["hi, my name is erik"], 100000).
Many of the follow up observations and questions have been stimulating, so
I'm now interested as well in a more detailed analysis.
However, in the end what I'm looking at is the differences in performance
between list and binary string processing under what I believe is idiomatic
Erlang for such problems.
On Tue, Oct 23, 2012 at 1:07 AM, Martynas Pumputis <>wrote:
> Could you show the exact steps of your simulation? Binary version should
> be faster, because some extra memory allocation is avoided per each
> iteration and large binaries aren't being copied.
> Take a look at: http://www.erlang.org/doc/**efficiency_guide/**
> On 10/23/2012 12:18 AM, Erik Pearson wrote:
>> I've read from advice given many years ago that processing binaries byte
>> by byte (e.g. a recursive parser), performance is better using a list to
>> accumulate the bytes, rather than using binary concatenation. So
>> [B|Accum] rather than <<Accum/binary, B>>. There seems to be
>> a consensus however, on the efficiency of Binaries compared to List
>> My own quick test, which was just to copy a list or binary element by
>> element, showed much better performance for the list version. The test
>> was basically to pass an arbitrary string or binary, and copy it some
>> number of thousands of times, and output the complete copies per second.
>> I tried list based accumulation for a binary, using binary destructuring
>> in the function head, and that sped things up, but it was still slower
>> than the equivalent list string copy.
>> Are there any tips for binaries? Of is this not a good use case for
>> test_bin_copy(Bin) ->
>> test_bin_copy(Bin, <<>>).
>> test_bin_copy(<<>>, Accum) ->
>> test_bin_copy(<<Char, Rest/binary>>, Accum) ->
>> test_bin_copy(Rest, <<Accum/binary, Char>>).
>> test_string_copy(Bin) ->
>> test_string_copy(Bin, ).
>> test_string_copy(, Accum) ->
>> test_string_copy([Char|Rest], Accum) ->
>> test_string_copy(Rest, [Char|Accum]).
>> For what its worth this is part of a json module. The current practice
>> in json libraries seems to favor binaries, so I assumed there were
>> inherent performance advantages. I can imagine, e.g., that an empty
>> binary would be stored as a modest sized buffer that would be appended
>> in place until there was a need to expand it or copy (e.g. if an older
>> version of it was being appended), and that operations on it would be
>> fast compared to arbitrary consing (which is however highly optimized.)
>> I think some of the favoritism for binaries in json libs is because it
>> makes it easy to differentiate json strings (as erlang binaries) from
>> json arrays (as erlang lists), but my implementation is using tagged
>> tuples to contain each json value, so this is not a concern. Of course
>> there are the memory concerns, but in my tests any memory concerns with
>> list char size vs binary bytes is erased by the performance gains.
>> I'm sure I've put my foot in my mouth at least once, but, anyway, advice
>> erlang-questions mailing list
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the erlang-questions