[erlang-questions] Early Out of Memory

[Danil Zagoskin]

Vance, and what do you expect?

Function consumes memory, VM allocates more memory.
On each iteration function consumes small amount of memory, but this does
not mean VM requests memory from OS on each iteration.
VM has some pre-allocated memory, and when almost all of it is used, VM
allocates next large block.

So, when there is already allocated 15.5 GB of available 16 and VM wants to
allocate 800MB more, the call fails, saying "cannot allocate 800M". That's
what you see in console, but it does not say how much is already used.


2013/3/1 Vance Shipley <vances@REDACTED>

> On Fri, Mar 01, 2013 at 10:33:35AM +0100, Patrik Nyblom wrote:
> }  Looks like you're running a 32 bit VM, in which case you have at a
> }  absolute maximum 4GB of memory. A heap allocation of 1.3 GB means
> }  an old version of the heap of possibly the same size (copying GC) +
> }  old heap generations and so on. A heap also needs to have
> }  continuous virtual memory, so the chance of having a single process
> }  with this much heap running on a 32bit VM is slim at best. If you
> }  have a lot of physical memory in the machine, run a 64bit OS and a
> }  64bit Erlang VM.
>
> Patrik,
>
> Fair enough, I didn't realize that I had forgotten to build R16B as
> 64 bit on my MacBook Pro.  When I did the test happily used up all
> 16GB of physical memory and 10GB of swap before I killed it.
>
> But can you explain this one?:
>
>    Erlang R16B (erts-5.10.1) [source] [64-bit halfword] [smp:8:8]
> [async-threads:10] [kernel-poll:false]
>
>    Eshell V5.10.1  (abort with ^G)
>    1> F = fun(F, Acc) -> F(F, [lists:seq(1,1000) | Acc]) end.
>    #Fun<erl_eval.12.17052888>
>    2> F(F, []).
>
>    Crash dump was written to: erl_crash.dump
>    eheap_alloc: Cannot allocate 790960704 bytes of memory (of type
> "old_heap").
>    Aborted (core dumped)
>
> Above I am running an R16B 64-bit halfword emulator on a server with 6GB
> of physical memory.
>
> What are the system limits with the halfword emulator?  I assume that I
> can't have a binary over 4GB.  Can I have an ets table bigger than 4GB?
>
> --
>         -Vance
>
>
> On 03/01/2013 07:09 AM, Vance Shipley wrote:
> }  In the pathological example below why is eheap_alloc failing to
> }  allocate 1.4GB of memory on a system with many more gigabytes of
> }  memory available?
> }
> }  Erlang R16B (erts-5.10.1) [source] [smp:8:8] [async-threads:10] [hipe]
> [kernel-poll:false]
> }
> }  Eshell V5.10.1  (abort with ^G)
> }  1> F = fun(F, Acc) -> F(F, [lists:seq(1,1000) | Acc]) end.
> }  #Fun<erl_eval.12.17052888>
> }  2> F(F, []).
> }
> }  Crash dump was written to: erl_crash.dump
> }  eheap_alloc: Cannot allocate 1366780092 bytes of memory (of type
> "heap").
> }  Abort trap: 6
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>



-- 
---------------------------------------------
Данил Загоскин | +7 906 064 20 47 | z@REDACTED
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20130301/7ac5560d/attachment.htm>