[erlang-questions] Garbage collection tuning for long living processes

[Jachym Holecek]

# Jonas Boberg 2011-10-21:
> Jachym Holecek wrote:
> > Or just split memory-intense tasks into separate short-lived processes.
> > There's for instance no reason why session setup should be done by the
> > same Pid as regular request processing, and the trickier the protocol
> > is, the more likely it is that breaking processing into small pieces
> > is a good idea I'd say.
> 
> I agree that splitting the tasks into separate short-lived processes
> would probably be the best approach, from a resource usage point of
> view, since no garbage collection would have to occur at all after the
> connection setup (the process would simply die).
> Unfortunately, splitting the connections lifecycle into multiple
> processes isn't an option for us - we are working with a legacy code
> base where doing so would be very costly.
> 
> One (long-living) process would have to manage the TCP connection
> though, and the in/out data would have to pass through it. Or would
> you actually do a handover (gen_tcp:controlling_process) between
> processes handling the different phases of the connection?

Yes -- handover to next stage handling process and exit (there
was a case in some protocol stack where the early stage process
didn't actually exit after handover -- it would stay around
monitoring the socket and schedule reconnect attempt whenever
the socket died -- but it only used very small & constant amount
of memory).

What you describe above is also reasonable in different contexts,
have one process handle all incoming traffic, deal with framing
and header decoding and pass request to temporary worker process
This is affordable as long as you're dealing with a binary
protocol with fairly simple header -- you end up passing around
just a small tuple holding a handful of small integers or atoms
and a binary holding message payload which will be decoded later
on.

BR,
	-- Jachym