[erlang-questions] [Q] Copying memory between processes

[Ladislav Lenart]

Bjorn Gustavsson wrote:
> Ladislav Lenart <lenartlad@REDACTED> writes:
>> But as soon as this structure is sent to one other process
>> the system is suddenly out of resources. The only reason
>> this could happen (that I could think of) is: When the
>> structure is copied to the heap of the other process, the
>> identity is not preserved, that is each item references its
>> own list of previous items (though equal). This would explain
>> expontential growth of the memory.
> 
> Yes.
> 
>> Am I right? And if it so, what is the reason (performance)?
> 
> Yes. In typical applications, shared data structures are uncommon.
Thanks,

I have two comments then:
1. This fact (and Erlang memory management) as a whole should
    be covered somewhere in docs (maybe it is and I just missed
    it?). Erlang doc states that when you modify something only
    the top most structure is created but its other "cells" point
    to the original (unmodified) data. But this is true only within
    one process. Furthermore each process has its own heap which
    complicates matters a bit too.
2. When debugging the above problem, the VM was stuck most of
    the time by not doing anything (the scheduler seemed stuck)
    although consuming 100% CPU and almost all memory (88%). When
    it was let to run longer, the memory consumption lowered to
    50% but the VM was still "stuck". I got the VM message "Can
    not reallocate ~1.7GB to the process heap" (or something like
    that) once in a while (for some sizes of the structure).
    I would expect the VM to segfault when it runs out of memory,
    but not this...

Ladislav Lenart