[erlang-questions] Performance with large queues

[Jayson Vantuyl]

>> It's possible that the process mailbox is collecting messages that never, ever get consumed.
> I'm not sure it is, actually, in this case - the gen_server main loop
> should always be popping the first message off the queue, whatever it is.
In that case, I would assume no message build-up.

>> Alternatively, does this behavior happen if you restart the service every, say, four hours?
> Difficult to say, although in both cases it's taken more than 4 hours to
> reach the point of crashing the VM.  The tipping point (presuming that's
> what it is) has happened both times under high load, and even then it
> runs happily for many days at a time until that point.  I've done some
> prodding of the live systems and there's nothing obviously "building up"
> (in terms of memory use, ETS table size, and everything else I could
> think to check).  Further, two other systems with similar loads were
> running for far longer with no issues.
That was actually the idea.  Assume that your gen_server dies every four hours, instead of running continuously.  Mnesia doesn't.  So, if the problem still occurs (and it's related to accumulation), it would be clear that it wasn't something in your gen_server.  If the problem goes away, then it would be pretty clear that it was in your gen_server.

-- 
Jayson Vantuyl
kagato@REDACTED