[erlang-questions] Memory usage in OTP 21

[Devon Estes]

Thanks so much for this! That's probably a good idea calling
`erlang:garbage_collect()` and relying on the trace information. I'll play
around with that to see how that affects things.

I'm seeing the behavior with the heap_block_size on this branch:
https://github.com/PragTob/benchee/tree/memory-measurement-issues

We have some samples in there, usually to show how to use benchee, but I've
added one to reproduce this issue (in `samples/wtf.exs`). If you run `mix
run samples/wtf.exs`, you'll see the process_info data printed to the
console, both before and after the function to be measured is run, and you
won't see any indication that a garbage collection event happened. If you
bump up the size of the range in the sample `samples/wtf.exs` to `1..5000`,
you'll see that 9 GC runs take place (7 minor, 2 major).

Also, if it helps, the code around memory measurement is all in
`lib/benchee/benchmark/measure/memory.ex`.

I'll try tomorrow to write this up in Erlang to reproduce it that way as
well.

On Tue, Jul 31, 2018 at 9:39 AM Lukas Larsson <lukas@REDACTED> wrote:

> Hello,
>
> On Fri, Jul 27, 2018 at 10:11 PM Devon Estes <devon.c.estes@REDACTED>
> wrote:
>
>> I hope this helps jog some ideas as to what might be going on here, and
>> thanks again for any help y'all can offer.
>>
>
> I think I have figured out what is going on. It seems to be the allocation
> of the term containing the garbage_collection_info that is messing with
> measurements. I'll try to change the code inside process_info to give a
> more accurate view of the current memory usage, however as long as the call
> to process_info happens in the process doing the call there will be some
> miss match.
>
> I would suggest that you instead do a call to erlang:garbage_collect()
> before and after the benchmark is run and rely on tracing to get the
> correct information. Alternatively you could call erlang:process_info on
> the process from the trace collector, though then you would need some type
> of synchronization in between the processes.
>
> I have not been able to reproduce the scenario you mentioned where the
> heap_block_size is increased without getting a trace message. What code did
> you run to trigger that? As always an Erlang example would be preferable,
> but I should manage with an Elixir one as well.
>
> Lukas
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20180731/a3000947/attachment.htm>