[erlang-questions] Garbage collection tuning for long living processes

[Jonas Boberg]

Hi,

On Fri, Oct 21, 2011 at 1:29 PM, Jachym Holecek <freza@REDACTED> wrote:
> Joel Reymont wrote
> erlang:garbage_collect() -> true

Yes, I could introduce such calls in strategic locations throughout
the code. The problem is figuring out where to put those calls,
without increasing CPU usage to much.
I guess I am looking for the VM to handle this for me, by being
'smarter' in this case. For example, shrink a big heap if less than a
percentage of it has been used for the last X time/number of
allocations etc.

Jachym Holecek wrote:
> Or just split memory-intense tasks into separate short-lived processes.
> There's for instance no reason why session setup should be done by the
> same Pid as regular request processing, and the trickier the protocol
> is, the more likely it is that breaking processing into small pieces
> is a good idea I'd say.

I agree that splitting the tasks into separate short-lived processes
would probably be the best approach, from a resource usage point of
view, since no garbage collection would have to occur at all after the
connection setup (the process would simply die).
Unfortunately, splitting the connections lifecycle into multiple
processes isn't an option for us - we are working with a legacy code
base where doing so would be very costly.

One (long-living) process would have to manage the TCP connection
though, and the in/out data would have to pass through it. Or would
you actually do a handover (gen_tcp:controlling_process) between
processes handling the different phases of the connection?

Regards
Jonas