[erlang-questions] Heap alloc error in lists:foldl/3

[Ladislav Lenart]

Hello.

%% if I use sum, instead of foldsum, everything is ok

This is because foldsum/2 creates a list of 10 000 000 integers (call to
lists:seq/2). And you create Np of such lists in total, each one in a different
process. A list in Erlang is a chain of cons pairs (as in Lisp). Each cons
occupies 2 words, where one word is either 32bits or 64bits, depending on your
HW architecture. Thus one such list takes

    10 000 000 * 8 = 80MB (on 32-bit architecture)

Hence it is fairly possible that your system simply ran out of memory.

The version using sum/2 does not exhibit this behaviour, because it runs in
constant space.


HTH,

Ladislav Lenart


On 6.11.2012 15:03, Arif Ishaq wrote:
> Hi,
>  
> I'm a newbie, playing with a toy programme to see how parallel processing works
> in Erlang.
>  
> The goal is to calculate the value of pi using Leibnitz series. The function
> pi/2 takes as arguments the number of processes, Np, and the number of terms
> handled by each process, Nt.
>  
> The machine is an HP workstation with Windows Vista Enterprise.
>  
>  
> Here's the code:
>  
> -module(pp).
> -compile(export_all).
>  
> %% calculate PI using Np parallel processes, each calculating Nt terms of the
> %% Liebnitz series
>  
> pi(Np, Nt) ->
>     Collector = spawn(?MODULE, collect, [Np, self()]),
>  
>     [spawn_opt(
>        fun() ->  Collector ! foldsum((N-1)*Nt, N*Nt -1) end      %% if I use
> sum, instead of foldsum, everything is ok
>       ) || N <- lists:seq(1,Np)],
>  
>     receive
>         Result ->
>             io:format("result is ~p~n", [4 * Result])
>     end.
>  
>  
> sum(Kfrom, Kto) ->
>     sum (Kfrom, Kto, 0).
>  
> sum(Kfrom, Kto, Sum) when Kfrom =:= Kto ->
>     Sum;
> sum(Kfrom, Kto, Sum) ->
>     sum(Kfrom+1, Kto, Sum + contribution(Kfrom)).
>  
>  
> foldsum(Kfrom, Kto) ->   
>     lists:foldl(
>       fun(K, Sum) -> Sum + contribution(K) end,
>       0,
>       lists:seq(Kfrom,Kto)).
>  
>  
> contribution(K) ->
>     Value = 1 / (2 * K + 1),
>     case K rem 2 of
>         0 ->
>             Value;
>         1 ->
>             -Value
>     end.
>  
>  
> collect (Np, For) ->
>     collect(Np, For, 0, 0).
>  
> collect (Np, For, Ncollected, Sum) when Np =:= Ncollected ->
>     For ! Sum;
> collect(Np, For, Ncollected, Sum) ->
>     receive
>         R ->
>             collect(Np, For, Ncollected + 1, Sum + R)
>     end.
>  
>  
>  
> If I try to calculate pi using 16 processes, each processing 1 million terms,
> the result is ok.
> If, on the other hand, I try with 10 million terms each, erlang crashes.
> Even after closing the werl window, the process keeps running and has to be
> killed brutally.
>  
>  
> Erlang R15B (erts-5.9) [smp:4:4] [async-threads:0]
>  
> Eshell V5.9  (abort with ^G)
> 1> c(pp).
> {ok,pp}
> 2> pp:pi(16,1000000).
> result is 3.1415925910897737
> ok
> 3> pp:pi(16,10000000).
>  
> Crash dump was written to: erl_crash.dump
> eheap_alloc: Cannot allocate 40121760 bytes of memory (of type "heap").
>  
>  
> Abnormal termination
>  
> If I try to read the crash dump with the crash dump viewer tool, I get an error
> saying the file is not an erlang crash dump:
> erl_crash.dump is not an Erlang crash dump
>  
>  
> Why is this happening? Shouldn't foldl be tail recursive? Is this a bug? Or am I
> doing something wrong?
>  
>  
> Thanks and best regards
> Arif Ishaq
>  
>  
> 
> 
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions