[erlang-questions] Early Out of Memory

[Vance Shipley]

Yes, sorry, I forgot to say that there is 5GB of free memory during this
time as reported by vmstat.
 On Mar 1, 2013 6:46 PM, "Danil Zagoskin" <z@REDACTED> wrote:

> Vance, and what do you expect?
>
> Function consumes memory, VM allocates more memory.
> On each iteration function consumes small amount of memory, but this does
> not mean VM requests memory from OS on each iteration.
> VM has some pre-allocated memory, and when almost all of it is used, VM
> allocates next large block.
>
> So, when there is already allocated 15.5 GB of available 16 and VM wants
> to allocate 800MB more, the call fails, saying "cannot allocate 800M".
> That's what you see in console, but it does not say how much is already
> used.
>
>
> 2013/3/1 Vance Shipley <vances@REDACTED>
>
>> On Fri, Mar 01, 2013 at 10:33:35AM +0100, Patrik Nyblom wrote:
>> }  Looks like you're running a 32 bit VM, in which case you have at a
>> }  absolute maximum 4GB of memory. A heap allocation of 1.3 GB means
>> }  an old version of the heap of possibly the same size (copying GC) +
>> }  old heap generations and so on. A heap also needs to have
>> }  continuous virtual memory, so the chance of having a single process
>> }  with this much heap running on a 32bit VM is slim at best. If you
>> }  have a lot of physical memory in the machine, run a 64bit OS and a
>> }  64bit Erlang VM.
>>
>> Patrik,
>>
>> Fair enough, I didn't realize that I had forgotten to build R16B as
>> 64 bit on my MacBook Pro.  When I did the test happily used up all
>> 16GB of physical memory and 10GB of swap before I killed it.
>>
>> But can you explain this one?:
>>
>>    Erlang R16B (erts-5.10.1) [source] [64-bit halfword] [smp:8:8]
>> [async-threads:10] [kernel-poll:false]
>>
>>    Eshell V5.10.1  (abort with ^G)
>>    1> F = fun(F, Acc) -> F(F, [lists:seq(1,1000) | Acc]) end.
>>    #Fun<erl_eval.12.17052888>
>>    2> F(F, []).
>>
>>    Crash dump was written to: erl_crash.dump
>>    eheap_alloc: Cannot allocate 790960704 bytes of memory (of type
>> "old_heap").
>>    Aborted (core dumped)
>>
>> Above I am running an R16B 64-bit halfword emulator on a server with 6GB
>> of physical memory.
>>
>> What are the system limits with the halfword emulator?  I assume that I
>> can't have a binary over 4GB.  Can I have an ets table bigger than 4GB?
>>
>> --
>>         -Vance
>>
>>
>> On 03/01/2013 07:09 AM, Vance Shipley wrote:
>> }  In the pathological example below why is eheap_alloc failing to
>> }  allocate 1.4GB of memory on a system with many more gigabytes of
>> }  memory available?
>> }
>> }  Erlang R16B (erts-5.10.1) [source] [smp:8:8] [async-threads:10] [hipe]
>> [kernel-poll:false]
>> }
>> }  Eshell V5.10.1  (abort with ^G)
>> }  1> F = fun(F, Acc) -> F(F, [lists:seq(1,1000) | Acc]) end.
>> }  #Fun<erl_eval.12.17052888>
>> }  2> F(F, []).
>> }
>> }  Crash dump was written to: erl_crash.dump
>> }  eheap_alloc: Cannot allocate 1366780092 bytes of memory (of type
>> "heap").
>> }  Abort trap: 6
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>
>
>
> --
> ---------------------------------------------
> Данил Загоскин | +7 906 064 20 47 | z@REDACTED
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20130301/4acda68c/attachment.htm>