[erlang-questions] Unexpected limitations

[Stu Bailey]

Hi all,

I have an 8 core MacBook Pro Retina Display with 16GB of memory.   I am
running R16B with no configure options...but I did install
libatomic_ops-7.2 .     I tried a very simple list allocation test which
surprisingly failed with a list size of 100 million.    I first made a list
with 10 million integers.   The beam hung on 100 million.   Thinking it was
something with lists:seq, I just wrote a dumb test to build big lists of
single character atoms.  That also failed at 100 million.   Is there a
known limitation to the size of lists in Erlang?

bash-3.2$ uname -a
Darwin myhost.local 12.3.0 Darwin Kernel Version 12.3.0: Sun Jan  6
22:37:10 PST 2013; root:xnu-2050.22.13~1/RELEASE_X86_64 x86_64

bash-3.2$ erl
Erlang R16B (erts-5.10.1) [source] [smp:8:8] [async-threads:10] [hipe]
[kernel-poll:false]

My Erlang Environment
Eshell V5.10.1  (abort with ^G)
1> lists:seq(1,10000000).
[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,
 23,24,25,26,27,28,29|...]
2> lists:seq(1,100000000).
beam.smp(85491,0xb039d000) malloc: *** mmap(size=458227712) failed (error
code=12)

      *** error: can't allocate region

                                      *** set a breakpoint in
malloc_error_break to debug
        beam.smp(85491,0xb039d000) malloc: *** mmap(size=659554304) failed
(error code=12)

              *** error: can't allocate region

                                              *** set a breakpoint in
malloc_error_break to debug
                beam.smp(85491,0xb039d000) malloc: *** mmap(size=791674880)
failed (error code=12)

                      *** error: can't allocate region

                                                      *** set a breakpoint
in malloc_error_break to debug
                        beam.smp(85491,0xb039d000) malloc: ***
mmap(size=791674880) failed (error code=12)

                              *** error: can't allocate region

                                                              *** set a
breakpoint in malloc_error_break to debug
                                beam.smp(85491,0xb039d000) malloc: ***
mmap(size=791674880) failed (error code=12)

                                      *** error: can't allocate region

                                                                      ***
set a breakpoint in malloc_error_break to debug
                                        beam.smp(85491,0xb039d000) malloc:
*** mmap(size=790962176) failed (error code=12)

                                              *** error: can't allocate
region


  *** set a breakpoint in malloc_error_break to debug
                                                beam.smp(85491,0xb039d000)
malloc: *** mmap(size=790962176) failed (error code=12)

                                                      *** error: can't
allocate region
     *** set a breakpoint in malloc_error_break to debug
                                                        Killed: 9


================

-module(big_lists).

-compile([export_all]).


make_list(0,Acc)->
    Acc;
make_list(Size,Acc) ->
    make_list(Size-1, [a] ++ Acc).

===================

5> c(big_lists).
{ok,big_lists}
6> big_lists:make_list(2,[]).
[a,a]
7> big_lists:make_list(100000,[]).
[a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a|...]
8> big_lists:make_list(1000000,[]).
[a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a|...]
9> big_lists:make_list(10000000,[]).
[a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a|...]
10> big_lists:make_list(100000000,[]).
beam.smp(85534,0xb039d000) malloc: *** mmap(size=381681664) failed (error
code=12)

      *** error: can't allocate region

                                      *** set a breakpoint in
malloc_error_break to debug
        beam.smp(85534,0xb039d000) malloc: *** mmap(size=659554304) failed
(error code=12)

              *** error: can't allocate region

                                              *** set a breakpoint in
malloc_error_break to debug
                beam.smp(85534,0xb039d000) malloc: *** mmap(size=791674880)
failed (error code=12)

                      *** error: can't allocate region

                                                      *** set a breakpoint
in malloc_error_break to debug

[a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a|...]
beam.smp(85534,0xb039d000) malloc: *** mmap(size=950009856) failed (error
code=12)

      *** error: can't allocate region

                                      *** set a breakpoint in
malloc_error_break to debug
        beam.smp(85534,0xb039d000) malloc: *** mmap(size=950009856) failed
(error code=12)

              *** error: can't allocate region

                                              *** set a breakpoint in
malloc_error_break to debug
                beam.smp(85534,0xb039d000) malloc: *** mmap(size=949153792)
failed (error code=12)

                      *** error: can't allocate region

                                                      *** set a breakpoint
in malloc_error_break to debug
                        beam.smp(85534,0xb039d000) malloc: ***
mmap(size=949153792) failed (error code=12)

                              *** error: can't allocate region

                                                              *** set a
breakpoint in malloc_error_break to debug

Crash dump was written to: erl_crash.dump
eheap_alloc: Cannot allocate 949152844 bytes of memory (of type "heap").
Abort trap: 6
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20130326/b8bebc39/attachment.htm>