[erlang-questions] Memory usage in OTP 21

[Tobias Pfeiffer]

Hi there and thank you for your help!

I already disassembled it to make sure it's not inlined (I had this with
another benchmark) and it's not:

   {:function, :test, 0, 10,
    [
      {:label, 9},
      {:line, [{:location, 'nofile', 2}]},
      {:func_info, {:atom, Test}, {:atom, :test}, 0},
      {:label, 10},
      {:move, {:literal, 1..10}, {:x, 0}},
      {:line, [{:location, 'nofile', 3}]},
      {:call_ext_only, 1, {:extfunc, Enum, :to_list, 1}}
    ]},


Disassembled with this code:
https://github.com/PragTob/elixir_playground/blob/master/lib/asm.ex

I'll try to give some more background information on the problem we have
in the coming days.

Dank you everyone!
Tobi

On 07/19/2018 05:28 PM, Jesper Louis Andersen wrote:
> I'm pretty sure that's not it, but a function such as
> 
> Enum.to_list(1..10)
> 
> contains an enumeration which is a constant and to_list can be unfolded
> to produce [1, ..., 10].
> 
> Since that is a constant it ends up as a literal in the beam bytecode
> and thus it never ever generates any garbage when called. I'm pretty
> sure that Erlang compiler is not smart enough to make this unfolding,
> but it doesn't take a lot of work to make a compiler constant fold such
> a case. Especially if it is common in code since compiler developers
> tend to target that. Another common trick is if escape analysis shows
> the result doesn't outlive its scope in which case data can be
> stack-allocated, making it far more unlikely to produce heap bump
> allocation and thus trigger the GC.
> 
> This assumption can be verifed by disassembly of the beam bytecode and
> looking for what the system is doing.
> 
> The negative allocation sounds strange to me though. That warrants
> investigation in what the trace calls are returning IMO to verify it
> happens at that level or lower.
> 
> On Thu, Jul 19, 2018 at 7:42 AM Devon Estes <devon.c.estes@REDACTED
> <mailto:devon.c.estes@REDACTED>> wrote:
> 
>     Hey everyone,
> 
>     First off, I would like to apologize for the following code example
>     in Elixir - it's the language I know best, and where we're having
>     the problem.
> 
>     Anyway, I'm one of the maintainers of an Elixir benchmarking tool
>     called Benchee. A few months ago we added memory measurement as a
>     feature in our benchmarking tool. However, with the release of OTP
>     21, we're seeing some measurements that seem very strange. For
>     example, according to our measurements, the following function uses
>     0 bytes of memory: `Enum.to_list(1..10)`. On OTP 20, this always
>     uses between 350-380 byes depending on the platform. There are a few
>     other instances of these kinds of functions which we believe should
>     be using memory somewhere, but the results that we get back from our
>     measurements say they are not using any memory, and all of them are
>     around functions that in OTP 20 used very little memory. We are also
>     seeing somewhat frequently what appears to be _negative_ net memory
>     usage, which again seems really strange.
> 
>     So, is this some sort of optimization that we're missing, or is
>     there somewhere else (maybe in a heap fragment?) that these
>     structures might be stored? And if they are somewhere else, is it
>     possible to measure this memory usage?
> 
>     At the moment we're measuring memory usage by using `erlang:trace/3`
>     to listen to the garbage collection events, and calculate the used
>     memory from there, and adding any remaining used memory on the heap
>     at the end of the function (after the last GC run).
> 
>     Thanks again for any help y'all might be able to offer!
>     _______________________________________________
>     erlang-questions mailing list
>     erlang-questions@REDACTED <mailto:erlang-questions@REDACTED>
>     http://erlang.org/mailman/listinfo/erlang-questions
> 
> 
> 
> -- 
> J.
> 
> 
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
> 

-- 
http://www.pragtob.info/