[erlang-questions] Memory usage in OTP 21

[Jesper Louis Andersen]

I'm pretty sure that's not it, but a function such as

Enum.to_list(1..10)

contains an enumeration which is a constant and to_list can be unfolded to
produce [1, ..., 10].

Since that is a constant it ends up as a literal in the beam bytecode and
thus it never ever generates any garbage when called. I'm pretty sure that
Erlang compiler is not smart enough to make this unfolding, but it doesn't
take a lot of work to make a compiler constant fold such a case. Especially
if it is common in code since compiler developers tend to target that.
Another common trick is if escape analysis shows the result doesn't outlive
its scope in which case data can be stack-allocated, making it far more
unlikely to produce heap bump allocation and thus trigger the GC.

This assumption can be verifed by disassembly of the beam bytecode and
looking for what the system is doing.

The negative allocation sounds strange to me though. That warrants
investigation in what the trace calls are returning IMO to verify it
happens at that level or lower.

On Thu, Jul 19, 2018 at 7:42 AM Devon Estes <devon.c.estes@REDACTED> wrote:

> Hey everyone,
>
> First off, I would like to apologize for the following code example in
> Elixir - it's the language I know best, and where we're having the problem.
>
> Anyway, I'm one of the maintainers of an Elixir benchmarking tool called
> Benchee. A few months ago we added memory measurement as a feature in our
> benchmarking tool. However, with the release of OTP 21, we're seeing some
> measurements that seem very strange. For example, according to our
> measurements, the following function uses 0 bytes of memory:
> `Enum.to_list(1..10)`. On OTP 20, this always uses between 350-380 byes
> depending on the platform. There are a few other instances of these kinds
> of functions which we believe should be using memory somewhere, but the
> results that we get back from our measurements say they are not using any
> memory, and all of them are around functions that in OTP 20 used very
> little memory. We are also seeing somewhat frequently what appears to be
> _negative_ net memory usage, which again seems really strange.
>
> So, is this some sort of optimization that we're missing, or is there
> somewhere else (maybe in a heap fragment?) that these structures might be
> stored? And if they are somewhere else, is it possible to measure this
> memory usage?
>
> At the moment we're measuring memory usage by using `erlang:trace/3` to
> listen to the garbage collection events, and calculate the used memory from
> there, and adding any remaining used memory on the heap at the end of the
> function (after the last GC run).
>
> Thanks again for any help y'all might be able to offer!
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>


-- 
J.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20180719/a45768ae/attachment.htm>