[erlang-bugs] Simple floating-point lists operation hungs Erlang system

[Björn-Egil Dahlberg]

It is not a deadlock, it seems to be a memory issue.

I got the same behavior. It seemed to freeze but not long after that:
 mmap(size=374341632) failed (error code=12)
can't allocate region.

At that point it had allocated 2GB of memory and I hit the virtual memory
limit. This was a 32bit emulator on Mac osx lion.

I changed vecTestLoop/3 to
...
    L2 = proc(L),
    erlang:garbage_collect(),   %% <---
    vecTestLoop( L2, Now, I ).

It ran better.

A vecTestList with 1000000 elements takes 33M words of memory. Four such
lists takes roughly (33M * 4 * 4= ) 0.5+ GB of memory. On top of this
garbage will be generated, well in this case almost everything is garbage
since your list is recreated at each iteration.

The copying garbage collector needs twice the heap memory, for that
process, during a collection. It is not hard to reach the virtual memory
limit in this case. You say that your memory consumption was about 1 GB, I
think it was a bit more.

// Björn-Egil



2011/11/25 Denis Nesterov <copypastor@REDACTED>

> Hello!
>
> I wrote simple benchmark code to measure floating-point Erlang
> performance. And seems that code gets Erlang system into a deadlock :-p
>
> The thing is that my code doing VERY simple thing - it first generates big
> list of pairs of Euclidian vectors, then in loop performs sequential
> mult-add operation on each element.
> When I start one such process per CPU core and list is big enough, the
> Erlang systems quickly stops responding.
>
>  The code is:
>
> -module ( bug ).
> -export ( [start/0] ).
>
> -record ( vec3, { x=0, y=0, z=0 } ).
>
>
> vec3ma( V0, V1, K ) when is_record( V0, vec3 ) and is_record( V1, vec3 )
> and is_number( K ) ->
> #vec3{ x = V0#vec3.x + V1#vec3.x * K, y = V0#vec3.y + V1#vec3.y * K, z =
> V0#vec3.z + V1#vec3.z * K }.
>
> %create test data
> makeVecTestList( L, 0 ) -> L;
> makeVecTestList( L, N ) ->
> E = { #vec3{ x = N * 10.0, y = 20.0, z = 30.0 }, #vec3{ x=10.0, y=20.0,
> z=-40.0 } },
> makeVecTestList( [ E|L ], N - 1 ).
>
> %Test list-processing code
> proc( L ) -> proc( L, [] ).
> proc( [], Acc ) -> Acc;
> proc( [ { P, V } | T ], Acc ) -> proc( T, [ { vec3ma( P, V, 0.11 ), V } |
> Acc ] ).
>
> %process loop
> vecTestLoop( L, T, I ) ->
> Now = now(),
> DT = timer:now_diff( Now, T ) * 1.0e-3,
> io:format( "Step~p: ~p ms\n", [I, DT] ),
> vecTestLoop( proc( L ), Now, I ).
>
> start() ->
> L = makeVecTestList( [], 1000000 ),
> io:format( "Running test with: ~p\n", [length(L)] ),
> spawn( fun() -> vecTestLoop( L, now(), 1 ) end ),
> spawn( fun() -> vecTestLoop( L, now(), 2 ) end ),
> spawn( fun() -> vecTestLoop( L, now(), 3 ) end ),
> spawn( fun() -> vecTestLoop( L, now(), 4 ) end ).
>
>
> I used next test-cases:
>
> AMD 64 2 core, 3.1 GHZ, Windows XP, Erlang R14B04:
> - Works well with one process and any list that fits into memory
> - Works well with two processes and list of 500000 elements
> - Hangs in about 30-60 seconds with list of 750000 elements
> - Hangs almost immediately with list of 1000000 elements
>
> Intel Quad 2.67 GHz, Windows 7 x64, Erlang R14B04:
> - Works well with one process and any list that fits into memory
> - Hangs in about 30-60 seconds with list of 1000000 elements
> - Hangs almost immediately with list of 1500000 elements
>
>  This looks very similar to deadlock - when normal iterations freeezes,
> the erl.exe cannot be stopped with Ctrl+C/Break, typing "q()." etc, and
> starts to load ONLY ONE CORE. Though while it still alive, it consumes all
> cores.
>
> Memory consumption by erl.exe was about or even under 1 GB in all cases.
>
>
> _______________________________________________
> erlang-bugs mailing list
> erlang-bugs@REDACTED
> http://erlang.org/mailman/listinfo/erlang-bugs
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-bugs/attachments/20111126/eb90c25c/attachment.htm>