[erlang-questions] Memory usage in OTP 21
Devon Estes
devon.c.estes@REDACTED
Fri Jul 27 20:35:17 CEST 2018
I now have an idea as to what's going on here. While looking at a function
that uses a little bit more memory, we're not getting any tracing messages
that indicate garbage collection is happening. And yet, when I check the
process's garbage collection info immediately after executing the function,
I see this:
[
old_heap_block_size: 610,
heap_block_size: 376,
mbuf_size: 0,
recent_size: 100,
stack_size: 14,
old_heap_size: 58,
heap_size: 177,
bin_vheap_size: 0,
bin_vheap_block_size: 46422,
bin_old_vheap_size: 0,
bin_old_vheap_block_size: 46422
]
The fact that the heap_block_size has been increased from 233 to 376 words,
and that there's an old_heap that's been allocated and now has data on it,
indicates that garbage collection happened, but our tracer process was
never sent a message. I tried flushing out the process mailbox for our
tracer to see if maybe we were just missing the message, and there was
still nothing in there after the function was run. I also tried on a much
larger function that generated a lot of data, and I did see plenty of
garbage collection messages sent to our tracer in that case, both minor and
major GC runs.
So, is it possible that the tracer is configured incorrectly? When we set
the tracer we're using `:erlang.trace(pid, true, [:garbage_collection,
tracer: self()])` (which I think is `erlang:trace(Pid, true,
[garbage_collection, {tracer, self()}])` in Erlang syntax if that's more
understandable for y'all).
This would explain the negative measurements, as there might have been some
data on the heap that was collected away, but we were never received any
messages about it. Also, this behavior is showing up in both OTP 20 and 21,
but it appears to be more pronounced in OTP 21.
I hope this helps jog some ideas as to what might be going on here, and
thanks again for any help y'all can offer.
Cheers,
Devon
On Sat, Jul 21, 2018 at 12:45 PM Tobias Pfeiffer <pragtob@REDACTED> wrote:
> Hi there and thank you for your help!
>
> I already disassembled it to make sure it's not inlined (I had this with
> another benchmark) and it's not:
>
> {:function, :test, 0, 10,
> [
> {:label, 9},
> {:line, [{:location, 'nofile', 2}]},
> {:func_info, {:atom, Test}, {:atom, :test}, 0},
> {:label, 10},
> {:move, {:literal, 1..10}, {:x, 0}},
> {:line, [{:location, 'nofile', 3}]},
> {:call_ext_only, 1, {:extfunc, Enum, :to_list, 1}}
> ]},
>
>
> Disassembled with this code:
> https://github.com/PragTob/elixir_playground/blob/master/lib/asm.ex
>
> I'll try to give some more background information on the problem we have
> in the coming days.
>
> Dank you everyone!
> Tobi
>
> On 07/19/2018 05:28 PM, Jesper Louis Andersen wrote:
> > I'm pretty sure that's not it, but a function such as
> >
> > Enum.to_list(1..10)
> >
> > contains an enumeration which is a constant and to_list can be unfolded
> > to produce [1, ..., 10].
> >
> > Since that is a constant it ends up as a literal in the beam bytecode
> > and thus it never ever generates any garbage when called. I'm pretty
> > sure that Erlang compiler is not smart enough to make this unfolding,
> > but it doesn't take a lot of work to make a compiler constant fold such
> > a case. Especially if it is common in code since compiler developers
> > tend to target that. Another common trick is if escape analysis shows
> > the result doesn't outlive its scope in which case data can be
> > stack-allocated, making it far more unlikely to produce heap bump
> > allocation and thus trigger the GC.
> >
> > This assumption can be verifed by disassembly of the beam bytecode and
> > looking for what the system is doing.
> >
> > The negative allocation sounds strange to me though. That warrants
> > investigation in what the trace calls are returning IMO to verify it
> > happens at that level or lower.
> >
> > On Thu, Jul 19, 2018 at 7:42 AM Devon Estes <devon.c.estes@REDACTED
> > <mailto:devon.c.estes@REDACTED>> wrote:
> >
> > Hey everyone,
> >
> > First off, I would like to apologize for the following code example
> > in Elixir - it's the language I know best, and where we're having
> > the problem.
> >
> > Anyway, I'm one of the maintainers of an Elixir benchmarking tool
> > called Benchee. A few months ago we added memory measurement as a
> > feature in our benchmarking tool. However, with the release of OTP
> > 21, we're seeing some measurements that seem very strange. For
> > example, according to our measurements, the following function uses
> > 0 bytes of memory: `Enum.to_list(1..10)`. On OTP 20, this always
> > uses between 350-380 byes depending on the platform. There are a few
> > other instances of these kinds of functions which we believe should
> > be using memory somewhere, but the results that we get back from our
> > measurements say they are not using any memory, and all of them are
> > around functions that in OTP 20 used very little memory. We are also
> > seeing somewhat frequently what appears to be _negative_ net memory
> > usage, which again seems really strange.
> >
> > So, is this some sort of optimization that we're missing, or is
> > there somewhere else (maybe in a heap fragment?) that these
> > structures might be stored? And if they are somewhere else, is it
> > possible to measure this memory usage?
> >
> > At the moment we're measuring memory usage by using `erlang:trace/3`
> > to listen to the garbage collection events, and calculate the used
> > memory from there, and adding any remaining used memory on the heap
> > at the end of the function (after the last GC run).
> >
> > Thanks again for any help y'all might be able to offer!
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED <mailto:erlang-questions@REDACTED>
> > http://erlang.org/mailman/listinfo/erlang-questions
> >
> >
> >
> > --
> > J.
> >
> >
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://erlang.org/mailman/listinfo/erlang-questions
> >
>
> --
> http://www.pragtob.info/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20180727/34aa9996/attachment.htm>
More information about the erlang-questions
mailing list