[erlang-questions] 14 seconds from disaster (was: Re: No JSON/MAPS interoperability in 17.0?)

[Richard A. O'Keefe]

On 19/03/2014, at 6:56 AM, Kenneth Lundin wrote:
> Also note that we don't think that EEP-20 is a good idea in its present form.
> EEP-20 "Split the atoms" introduces two types of atoms, and is invisible on the language level, but for the VM implementation and the characteristics it is dramatic.

As EEP 20 notes, it describes a technique which has been implemented
and used in a concurrent logic programming language.

> Todays atoms are represented as immediates in todays VM meaning that no allocation on the heap is needed.

> Many places in the VM code relies on that fact which makes it easy to se atom. In EEP-20 some atoms will be boxed types stored on the heap and this will change the impact on memory handling an memory consumption dramatically. Performance will also be impacted, negatively.

It's certainly true that performance *might* be hurt,
but it is very far from obvious that it *must* be so.

On the other hand, I have evidence that *not* having collectable
atoms *does* have a huge negative performance impact.

I tried this on two machines with different CPUs and operating systems.

1> [if N rem 10000 == 0 -> io:write(N), io:nl(), A ; true -> A end
1> || N <- lists:seq(1,10000000),                   
1>     A <- [list_to_atom(integer_to_list(N))]].

This tries to create 10,000,000 atoms, reporting every 10,000.

After just 14 seconds 'beam.smp' goes to 95% CPU and stays there,
busy doing oodles of system calls (including pthead_cond_wait)
and getting nowhere.

*That* is a negative performance impact.

When people use <<"foo">> instead of 'foo',
they are in effect implementing EEP 20 by hand,
inconsistently, and without the benefit of the
performance hacks that can go into a low-level
implementation.