[erlang-questions] Memory usage in OTP 21

[Devon Estes]

And here's a gist with an Erlang reproduction:
https://gist.github.com/devonestes/abe4dc60484e5426d4890c4494430194. When I
run this escript as is, I see that the `recent_size` is different (which is
only supposed to change after a GC run, right?) but no trace messages
indicating that garbage collection took place. However, with this Erlang
example, the `heap_block_size` and `old_heap_block_size` remain the same.
When I bump up the size of the list I'm mapping over (ToMap) from 50 to
100, I see garbage collection trace messages indicating that a single minor
collection took place.

On Tue, Jul 31, 2018 at 8:08 PM Devon Estes <devon.c.estes@REDACTED> wrote:

> Thanks so much for this! That's probably a good idea calling
> `erlang:garbage_collect()` and relying on the trace information. I'll play
> around with that to see how that affects things.
>
> I'm seeing the behavior with the heap_block_size on this branch:
> https://github.com/PragTob/benchee/tree/memory-measurement-issues
>
> We have some samples in there, usually to show how to use benchee, but
> I've added one to reproduce this issue (in `samples/wtf.exs`). If you run
> `mix run samples/wtf.exs`, you'll see the process_info data printed to the
> console, both before and after the function to be measured is run, and you
> won't see any indication that a garbage collection event happened. If you
> bump up the size of the range in the sample `samples/wtf.exs` to `1..5000`,
> you'll see that 9 GC runs take place (7 minor, 2 major).
>
> Also, if it helps, the code around memory measurement is all in
> `lib/benchee/benchmark/measure/memory.ex`.
>
> I'll try tomorrow to write this up in Erlang to reproduce it that way as
> well.
>
> On Tue, Jul 31, 2018 at 9:39 AM Lukas Larsson <lukas@REDACTED> wrote:
>
>> Hello,
>>
>> On Fri, Jul 27, 2018 at 10:11 PM Devon Estes <devon.c.estes@REDACTED>
>> wrote:
>>
>>> I hope this helps jog some ideas as to what might be going on here, and
>>> thanks again for any help y'all can offer.
>>>
>>
>> I think I have figured out what is going on. It seems to be the
>> allocation of the term containing the garbage_collection_info that is
>> messing with measurements. I'll try to change the code inside process_info
>> to give a more accurate view of the current memory usage, however as long
>> as the call to process_info happens in the process doing the call there
>> will be some miss match.
>>
>> I would suggest that you instead do a call to erlang:garbage_collect()
>> before and after the benchmark is run and rely on tracing to get the
>> correct information. Alternatively you could call erlang:process_info on
>> the process from the trace collector, though then you would need some type
>> of synchronization in between the processes.
>>
>> I have not been able to reproduce the scenario you mentioned where the
>> heap_block_size is increased without getting a trace message. What code did
>> you run to trigger that? As always an Erlang example would be preferable,
>> but I should manage with an Elixir one as well.
>>
>> Lukas
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20180801/89477273/attachment.htm>