[erlang-questions] Erlang and Memory management

[Ulf Wiger]

On 1 Jun 2011, at 01:01, Jack Moffitt wrote:

>> I quite agree with you, I also think a shared heap architecture is the way to go and doing an SMP version would be very interesting. My only gripe with erjang is that it cannot yet use SMPs properly. But I assume they will fix that. A shared heap architecture could change
> 
> Could you elaborate a bit on the reason why you'd prefer a shared
> heap? I was under the impression that individual heaps were one of the
> ways Erlang achieved soft real-time performance. Is CPU power enough
> improved (or garbage collectors) since Erlang made that decision that
> it's no longer worth it?

Actually, one of the innovations of the Erlang team (Robert and Joe [1], I believe) was to realise that when no cyclical data structures are allowed, and data structures are immutable, GC is much helped by the fact that all pointers point from newer data to older data, and GC can easily be made re-entrant. Thus, incremental GC fits Erlang very well, and real-time performance need not be sacrificed.

A problem with the old shared-heap implementation was that the incremental GC was never implemented (this, as it turned out, was good news for Scala [2]). A compromise was to introduce a hybrid-heap system, where only data that is actually shared between processes gets put on a shared heap.

As it turns out, the hybrid-heap system ought to be a good compromise in the long term as well, since a copying GC has some advantages  over (reference-counting) shared-heap GC. In particular, reference counting doesn't add much value for non-shared data, but it does increase memory usage.

I am certain Kostis and Robert will correct me if I've uttered any falsehoods. :)

BR,
Ulf W

[1] http://www.erlang.se/publications/memory1995.ps
     One Pass Real-Time Generational Mark-Sweep Garbage Collection. 
    Joe Armstrong and Robert Virding. 
    International Workshop on Memory Management. 
    Kinross, Scotland, September 27-29, 1995.

[2] Instead of joining Ericsson and implementing the shared-heap GC, Erik Stenmnan became project manager in Martin Odersky's Scala group and helped bring about Scala 1.0.

Ulf Wiger, CTO, Erlang Solutions, Ltd.
http://erlang-solutions.com