Known BEAM memory leak ? Profiling tools ?

[Scott Lystig Fritchie]

>>>>> "jh" == Jon Holdsworth <jon@REDACTED> writes:

jh> To the erlang-questions people: Erlang running under Linux Redhat
jh> 6.1 on Intel and Solaris 2.7 on Sparc, the BEAM program has been
jh> seen to apparently leak memory after being left running for some
jh> time.  It is possible that this 'memory leak' is in our own code,
jh> but it has not revealed itself yet.

Your own code?  Have you written any linked-in drivers (static or
dynamic)?  Or is your code purely Erlang?  If the former, well, that's
another bucket of mud.  The latter...

... are you sure you don't have a "process leak"?  By that I mean a
number of processes which have escaped a supervision tree?  I ran into
an evil memory consumption problem that, on the surface, looked like
you describe: both "ps" and "top" said that the VM's UNIX process was
eating a ton of memory, and I couldn't figure out why.  I suspected a
memory leak with binary objects.

It finally hit me to fire up the process manager, via "pman:start()".
The silly thing timed out, causing virtual memory thrashing & wild
disk activity.  After a few tries, I got it to pop up a window before
the start function timed out.  Its process listing was *huge*.
Dumbstruck, I tried "length(processes())", expecting about 150 and
instead got I-forget-how-many thousand processes.  They weren't doing
anything other than sitting around consuming a few kilobytes of memory
each ... and blocking waiting to receive messages that would never
arrive.

Sure enough, we had a process leak.  Being much more careful about
process linkage made the problem vanish.  ... Many thanks to Lennart
Öhman for (much later) giving us a better and much more paranoid
perspective on process linking: "Pretend that your system will run for
10 years straight.  Even a tiny process leak will cause a Siberia
Problem(*) in that amount of time."

Our worries about binary objects (in R6B-0) were unfounded.  I
honestly don't recall why I didn't think about that kind of leak
earlier, but I do remember how surprised I was when I found it.  And
how chagrined I was to not think of it earlier.

-Scott

(*) When you need to fly/drive/hitchhike/walk all the way to an
equipment closet in Siberia just to press a stupid reset button.