Performance of term_to_binary vs Bbinary_to_term

[Valentin Micic]

Yes, your (as well as everyone else’s) explanation may account for binary_to_term/1 being slower, I get it — it’s a story about complexity and all… however, something does not add up.

Consider the first example in my measurements:

The term being converted to external representation is  an atom ‘a’. 

So, conversion from an atom ‘a’ to its external representation:  <<131,100,0,1,97>> (using term_to_binary/1) is projected to run at the rate of more than 103 million conversions per second.

Since this term has a very simple external representation: <<131, 100,0, 1, 97>>; one would expect conversion back to its internal representation to be just marginally slower. 

And yet, binary_to_term/1 is an order of magnitude  (or more than 34 times) slower!

I don’t think that story of complexity can pass a red-face test here. 

Kind regards

V/

> On 07 Jun 2021, at 20:22, Antoine Koener <antoine.koener@REDACTED> wrote:
> 
> 
> Not sure that this will explain all, but reconstruction of terms needs more work than serialising, think about atoms than may need to be created, or complex structure that need allocation. 
> 
> Converting everything to binary don’t need any check at all, whereas the reverse is not true.
> 
> What do you think?
> 
> 
> 
>> On 7 Jun 2021, at 14:25, Valentin Micic <v@REDACTED> wrote:
>> 
>> Hmmm… I realised that my previous email may be seen as a comment rather than a question, so let me ask the question clearly.
>> 
>> Given that binary_to_term/1 is about two orders of magnitude slower than term_to_binary/1, is there anyone out there that may have a reasonable explanation for that?
>> 
>> Kind regards
>> 
>> V/
>> 
>>> On 06 Jun 2021, at 02:07, Valentin Micic <v@REDACTED <mailto:v@REDACTED>> wrote:
>>> 
>>> Hi all,
>>> 
>>> I did some performance measurement recently that included conversion of an arbitrary erlang term to its external binary representation via term_to_binary/1, as well as reversing the result using binary_to_term/1.
>>> 
>>> I’ve noticed that term_to_binary/1 is significantly faster than binary_to_term/1.
>>> 
>>> Also, I’ve observed that binary_to_term/1 performance gets considerably worse as complexity of specified term increases, whilst term_to_binary/1 maintains (more-less) steady performance.
>>> 
>>> (cig@REDACTED)40> tconvert:run( a, 10000000 ).
>>> 
>>> term_to_binary/1 RETURN VALUE:<<131,100,0,1,97>>
>>> REQUEST COUNT:10000000
>>> ELAPSED TIME (usec):97070
>>> TIME PER REQUEST (usec): 0.009707
>>> PROJECTED RATE (req/sec): 103018440.30081384
>>> 
>>> binary_to_term/1 RETURN VALUE:a
>>> REQUEST COUNT:10000000
>>> ELAPSED TIME (usec):3383483
>>> TIME PER REQUEST (usec): 0.3383483
>>> PROJECTED RATE (req/sec): 2955534.2822765773
>>> ok
>>> 
>>> (cig@REDACTED)41> tconvert:run( {a,<<1,2,3>>, b, [1,2,3], c, {1,2,3}, d, #{a=>1, b=>2, c=>3}}, 10000000 ).
>>> 
>>> term_to_binary/1 RETURN VALUE:<<131,104,8,100,0,1,97,109,0,0,0,3,1,2,3,100,0,1,
>>>                                 98,107,0,3,1,2,3,100,0,1,99,104,3,97,1,97,2,97,
>>>                                 3,100,0,1,100,116,0,0,0,3,100,0,1,97,97,1,100,
>>>                                 0,1,98,97,2,100,0,1,99,97,3>>
>>> REQUEST COUNT:10000000
>>> ELAPSED TIME (usec):97307
>>> TIME PER REQUEST (usec): 0.0097307
>>> PROJECTED RATE (req/sec): 102767529.57135664
>>> 
>>> binary_to_term/1 RETURN VALUE:{a,<<1,2,3>>,
>>>                                  b,
>>>                                  [1,2,3],
>>>                                  c,
>>>                                  {1,2,3},
>>>                                  d, 
>>>                                  #{a => 1,b => 2,c => 3}}
>>> REQUEST COUNT:10000000
>>> ELAPSED TIME (usec):8747426
>>> TIME PER REQUEST (usec): 0.8747426
>>> PROJECTED RATE (req/sec): 1143193.4377038456
>>> ok
>>> 
>>> 
>>> 
>>> I’ve performed testing on R21.1.
>>> Any thoughts?
>>> 
>>> V/
>> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20210608/77878cc3/attachment.htm>