[erlang-questions] Need help understanding why the Process(Pid) consumers a lot of memory and did not garbage collect

[Sergej Jurecko]

You are probably running into issues addressed here:
https://blog.heroku.com/archives/2013/11/7/logplex-down-the-rabbit-hole


Sergej

On Jan 15, 2014, at 8:18 AM, 郎咸武 wrote:

> I will appreciate If  anyone  can reply to me .
> 
> 
> 2014/1/14 郎咸武 <langxianzhe@REDACTED>
> Hi All.
> In recently, I ran into a problem which the Process consumers a lot of memory. There is roughly from 300M to 450 each process.。
> The memory will be free if invoke erlang:garbage_collect/1。So, In my view , the moemry may is not be used, and do not garbage collect. Why? I can not get it. Please give some for me. 
> For further resolve the problem,  I do some test in the simplification program. The memery is can garbage collect too(pass an hour).
> Test Method：
> 1、The R variable is about 19489 number records
> （如[{"author_id","users-a9KsYTIRdrq3iuAywiLRE2BGd3Y"},{"timestamp","2013-07-17 10:47:36"},{"time_int",63541277256},{"title",[233,155,133,229,133,184,231,139,172,229,136,155,232,136,170,230,181,183,229,164,169,230,150,135,229,143,176,232,133,149,232,161,168]},{"id","info-137429256786232"}]）
> 2、Obtain the Process of Pid
>       (trends1@REDACTED)5> Pid = test4:get_pid()
> 3、The Pid state before send R data
> (trends1@REDACTED)7> erlang:process_info(Pid, memory).
> {memory,2656}
> (trends1@REDACTED)8> erlang:process_info(Pid).        
> [{current_function,{test4,loop,0}},
>  {initial_call,{test4,loop,0}},
>  {status,waiting},
>  {message_queue_len,0},
>  {messages,[]},
>  {links,[]},
>  {dictionary,[]},
>  {trap_exit,false},
>  {error_handler,error_handler},
>  {priority,normal},
>  {group_leader,<0.31.0>},
>  {total_heap_size,233},
>  {heap_size,233},
>  {stack_size,1},
>  {reductions,1},
>  {garbage_collection,[{min_bin_vheap_size,46368},
>                       {min_heap_size,233},
>                       {fullsweep_after,0},
>                       {minor_gcs,0}]},
>  {suspending,[]}]
> 
> 4、 to send  R Data
> (trends1@REDACTED)9> Pid ! {self(),R}
> (trends1@REDACTED)9> .
> {<0.52.0>,
>  {ok,[{1,
>        [[{"author_id","users-a9KsYTIRdrq3iuAywiLRE2BGd3Y"},
>          {"timestamp","2013-07-17 10......................
> 5、The Pid state after  send R data, The memory is increasing.
> (trends1@REDACTED)11> erlang:process_info(Pid, memory).
> {memory,51356672}
> (trends1@REDACTED)12> erlang:process_info(Pid).        
> [{current_function,{test4,loop,0}},
>  {initial_call,{test4,loop,0}},
>  {status,waiting},
>  {message_queue_len,0},
>  {messages,[]},
>  {links,[]},
>  {dictionary,[]},
>  {trap_exit,false},
>  {error_handler,error_handler},
>  {priority,normal},
>  {group_leader,<0.31.0>},
>  {total_heap_size,6419485},
>  {heap_size,6419485},
>  {stack_size,1},
>  {reductions,2002},
>  {garbage_collection,[{min_bin_vheap_size,46368},
>                       {min_heap_size,233},
>                       {fullsweep_after,0},
>                       {minor_gcs,0}]},
>  {suspending,[]}]
> 6、to send R data again.
> (trends1@REDACTED)13> Pid ! {self(),R}.                
> {<0.52.0>,
>  {ok,[{1,
> 7、The memory  continue to increas.
> (trends1@REDACTED)14> erlang:process_info(Pid, memory).
> {memory,125381272}
> (trends1@REDACTED)15> erlang:process_info(Pid).        
> [{current_function,{test4,loop,0}},
>  {initial_call,{test4,loop,0}},
>  {status,waiting},
>  {message_queue_len,0},
>  {messages,[]},
>  {links,[]},
>  {dictionary,[]},
>  {trap_exit,false},
>  {error_handler,error_handler},
>  {priority,normal},
>  {group_leader,<0.31.0>},
>  {total_heap_size,15672560},
>  {heap_size,15672560},
>  {stack_size,1},
>  {reductions,4003},
>  {garbage_collection,[{min_bin_vheap_size,46368},
>                       {min_heap_size,233},
>                       {fullsweep_after,0},
>                       {minor_gcs,0}]},
>  {suspending,[]}]
> 
> I am  wondering the cause why  it is can not collect, Please give me some idea. Thanks a lot.
> (trends1@REDACTED)23> erlang:process_info(Pid, memory).
> {memory,195907792}
> 
> 
> I'm running Erlang R14B04 on the unbuntu.
> Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false]和
> Linux jason-lxw 3.2.0-55-generic #85-Ubuntu SMP Wed Oct 2 12:29:27 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
> 
> [1]代码：
>   1 -module(test4).
>   2 -export([get_pid/0,
>   3          get_data/1,
>   4          get_pids/1,
>   5          loop/0]).
>   6 
>   7 get_pids(L) ->
>   8     get_pids(0, L).
>   9 
>  10 get_pids(0, L) ->
>  11     L;
>  12 get_pids(N, R) ->
>  13     NewR = [get_pid()|R],
>  14     get_pids(N-1, NewR).
>  15 
>  16 get_pid()->
>  17      spawn_opt(?MODULE, loop, [], [{fullsweep_after,0}]).
>  18      %spawn(?MODULE, loop, []).                                                                                                                  
>  19 
>  20 get_data(Pid) ->
>  21      Pid ! {self(), get}.
>  22 
>  23 loop() ->
>  24     receive
>  25         {From, Data} ->
>  26             From ! Data,
>  27             loop()
>  28     end.
> 
> 
> Cheers
> Jason 
> -- 
> 只为成功找方法，不为失败找理由
> 
> 
> 
> -- 
> 只为成功找方法，不为失败找理由
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20140115/51df9888/attachment.htm>