[erlang-questions] Garbage Collection, BEAM memory and Erlang memory

[Roberto Ostinelli]

On Wed, Jan 28, 2015 at 5:19 PM, Fred Hebert <mononcqc@REDACTED> wrote:

> On 01/28, Roberto Ostinelli wrote:
> > Here's the erl_crash.dump analysis, done with the nice Fred's script:
> >
> > analyzing erl_crash.dump, generated on:  Wed Jan 28 13:59:36 2015
> >
> > Slogan: Received SIGUSR1
> >
> > File descriptors open:
> > ===
> >   UDP:  0
> >   TCP:  180071
> >   Files:  6
> >   ---
> >   Total:  180077
> >
>
> So uh, is 180,000 TCP ports the number you have or expect? Could you be
> running out of these and then massive erroring takes place or anything
> like that? What are the limits in place for your system?
>

That's exactly the number of long lived connections to the system, so yes
it's expected.
ulimit (hard and soft) are set to 1M.


> Do you see anything wrong there? I honestly don't.
> >
>
> Nothing looks obviously wrong. At this point my number one suspsicion
> would be some process(es) that suddenly get lots of messages (links,
> monitors, node monitors, general events) and fill their mailboxes. This
> then prompts for GCs, which suspends the process, copies the mailbox to
> the heap, and then runs again.
>
> This kind of stuff can sometimes spiral out of control.
>


So, here's the thing. I got a bigger box, a 30GB. I'm launching the same
test, and What I have is the following:
https://cldup.com/1M4qzvbLp_-3000x3000.png

Basically what happens is that the memory "ramp-up" happens in two phases.
The second phase was the one previously making my VM blowup.
With this bigger box, this phase continues but then stabilizes.

Not sure why this happens in this way, but with a bigger box, as you can
see, when 22.3 GB of memory are reached, everything is stable.
This was confusing, but it simply looks like there's some time needed
before the memory gets allocated. I'm assuming this has to do with
fullsweep_after. With it set to 10, we get to the 22.3GB illustrated; with
it set to 5, we get to 19.3GB (3GB less).

I guess this is it for now. I can only thank you for your time.

Any additional remarks?

Best,
r.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20150128/1ac7418f/attachment.htm>