Much larger old_heap than heap

[Loïc Hoguin]

Hello,

I was able to dig deeper today. It's not that there was living data 
(first thing I checked of course). It's that the process was processing 
so much data that the old heap quickly gets full of no longer useful 
data. It sits long enough in memory to make it to old heap, but not very 
long in the grand scheme of things.

Setting fullsweep_after to 0 reduces the heap size by 2-10 times 
depending on the current state size.

Cheers,

On 18/03/2021 18:56, Björn-Egil Dahlberg wrote:
> Ehum?
> 
> total_heap_size = heap_size + old_heap_size. Meaning 1st gen heap + 2nd 
> gen heap. So total_heap_size /should/ be equal, or more probable, higher 
> than the heap_size.
> 
> The reason you don't see that it shrinks during a garbage collect is ofc 
> that there's still living data on the heap.
> 
> Den ons 17 mars 2021 kl 21:53 skrev Loïc Hoguin <lhoguin@REDACTED 
> <mailto:lhoguin@REDACTED>>:
> 
>     Hello,
> 
>     I am trying to understand why the total_heap_size of a few processes
>     is so much higher than heap_size. As can be seen in the following
>     snippet, the old_heap is responsible for the discrepancy:
> 
>      > erlang:process_info(QPid, garbage_collection_info).
>     {garbage_collection_info,[{old_heap_block_size,1439468},
>                                {heap_block_size,196650},
>                                {mbuf_size,289},
>                                {recent_size,11674},
>                                {stack_size,35},
>                                {old_heap_size,940791},
>                                {heap_size,86028},
>                                {bin_vheap_size,36483},
>                                {bin_vheap_block_size,46422},
>                                {bin_old_vheap_size,34148},
>                                {bin_old_vheap_block_size,46422}]}
> 
>     Why?
> 
>     How can I reduce this? Garbage collecting does nothing.
> 
>     Cheers,
> 
>     -- 
>     Loïc Hoguin
> 

-- 
Loïc Hoguin
https://ninenines.eu