[erlang-questions] Erlang - Question on memory usage reported

[Scott Lystig Fritchie]

Sorry for replying to a message a week and a half old...

Rajesh Bhat <rbhat@REDACTED> wrote:

rb> I am running an application that starts up with a memory footprint
rb> of around ~725MB when started (Approx 9000 processes on
rb> startup).. Application spawns processes as new requests are
rb> processed and grows to about 50K processes and memory usage of 5.1GB
rb> during the day. [...]

rb> At the end of the day, a cleanup job reviews the process state and
rb> kills the pids that are not required and cleans up the ets
rb> table. [...]

rb> [...] however running 'htop' on the system still indicates Erlang
rb> beam process with a resident memory usage of around 5.1GB. Any
rb> pointers that explains this behavior is greatly appreciated. Is
rb> there a way to release the memory?

Not easily, no.  When a UNIX process calls free(), there's no guarantee
that the memory free'd by that call will be returned to the OS now,
later, or ever.  It depends on how it's allocated.  If the allocator
used mmap(), then it can't munmap() the memory until (at least) all
other chunks in the mmap()'ed region are also freed.  If it's allocated
using sbrk() or related, you can't move the break back down (to
deallocate memory) if there are still live chunks of memory above the
break.

The Erlang VM uses its own set of allocators instead of using malloc
directly, so they add another layer of when-to-release-to-the-OS that I
can't comment directly on, sorry.

-Scott