Performance of term_to_binary vs Bbinary_to_term

[Mikael Pettersson]

On Tue, Jun 8, 2021 at 12:03 AM Valentin Micic <v@REDACTED> wrote:
> The term being converted to external representation is  an atom ‘a’.
>
> So, conversion from an atom ‘a’ to its external representation:  <<131,100,0,1,97>> (using term_to_binary/1) is projected to run at the rate of more than 103 million conversions per second.
>
> Since this term has a very simple external representation: <<131, 100,0, 1, 97>>; one would expect conversion back to its internal representation to be just marginally slower.

This expectation is unfortunately not true. Converting the external
representation of an atom, i.e. a sequence of characters in a binary
or a list, to its internal representation, is somewhat complex and
expensive. The reason is that an Erlang VM has a single unique handle
for every atom known to it. This is known as "interning" and it
reduces storage requirements and speeds up equality tests.
Implementation-wise it requires maintaining a mapping from strings to
those handles, typically via a hash table, and lookups or updates in
that table require synchronization from concurrent accesses. (Most
LISP/Scheme implementations use interning as well, and I would expect
Prolog implementations to do too.)

I took your tconvert.erl, applied Björn's modification, and then
modified it to allow passing in the encoding and decoding functions:

2> tconvert:run(a, 10000000, fun erlang:atom_to_binary/1, fun
erlang:binary_to_atom/1).

fun erlang:atom_to_binary/1 RETURN VALUE:<<"a">>
REQUEST COUNT:10000000
ELAPSED TIME (usec):267899
TIME PER REQUEST (usec): 0.0267899
PROJECTED RATE (req/sec): 37327500.289288126

fun erlang:binary_to_atom/1 RETURN VALUE:a
REQUEST COUNT:10000000
ELAPSED TIME (usec):753514
TIME PER REQUEST (usec): 0.0753514
PROJECTED RATE (req/sec): 13271153.555209326

(This is on an old Ivy Bridge desktop with OTP-24.0.2)

So in your case it's the interning of the atom that's the dominant
cost, not binary_to_term/1 in general.