[erlang-questions] Need help understanding why the Process(Pid) consumers a lot of memory and did not garbage collect

[Ola Bäckström]

I just briefly read your post so I might have missed something, but here’s some comments.

It looks like you’re sending really large messages, I can’t read from your mail how large, but 19489 times 200 bytes (just an example) would be 2Mb.
That is rather unusual.

Could you test making the tail call a full module call:
        {From, Data} ->
            From ! Data,
            test4:loop()

Just to check if that have any influence on GC.

The other thing is that you should be careful to empty the process “mailbox” by using an other-clause in receive:
    receive
        {From, Data} ->
            From ! Data,
            loop()
        Other ->
             exit({got_unhandled_message, Other}
    end.

I’d suggest to use a gen_server so you don’t need to bother too much.

/Ola

From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of ???
Sent: den 15 januari 2014 08:18
To: Erlang (E-mail)
Subject: Re: [erlang-questions] Need help understanding why the Process(Pid) consumers a lot of memory and did not garbage collect

I will appreciate If  anyone  can reply to me .

2014/1/14 郎咸武 <langxianzhe@REDACTED<mailto:langxianzhe@REDACTED>>
Hi All.
In recently, I ran into a problem which the Process consumers a lot of memory. There is roughly from 300M to 450 each process.。
The memory will be free if invoke erlang:garbage_collect/1。So, In my view , the moemry may is not be used, and do not garbage collect. Why? I can not get it. Please give some for me.
For further resolve the problem,  I do some test in the simplification program. The memery is can garbage collect too(pass an hour).
Test Method：
1、The R variable is about 19489 number records
（如[{"author_id","users-a9KsYTIRdrq3iuAywiLRE2BGd3Y"},{"timestamp","2013-07-17 10:47:36"},{"time_int",63541277256},{"title",[233,155,133,229,133,184,231,139,172,229,136,155,232,136,170,230,181,183,229,164,169,230,150,135,229,143,176,232,133,149,232,161,168]},{"id","info-137429256786232"}]）
2、Obtain the Process of Pid
      (trends1@REDACTED<mailto:trends1@REDACTED>)5> Pid = test4:get_pid()
3、The Pid state before send R data
(trends1@REDACTED<mailto:trends1@REDACTED>)7> erlang:process_info(Pid, memory).
{memory,2656}
(trends1@REDACTED<mailto:trends1@REDACTED>)8> erlang:process_info(Pid).
[{current_function,{test4,loop,0}},
 {initial_call,{test4,loop,0}},
 {status,waiting},
 {message_queue_len,0},
 {messages,[]},
 {links,[]},
 {dictionary,[]},
 {trap_exit,false},
 {error_handler,error_handler},
 {priority,normal},
 {group_leader,<0.31.0>},
 {total_heap_size,233},
 {heap_size,233},
 {stack_size,1},
 {reductions,1},
 {garbage_collection,[{min_bin_vheap_size,46368},
                      {min_heap_size,233},
                      {fullsweep_after,0},
                      {minor_gcs,0}]},
 {suspending,[]}]

4、 to send  R Data
(trends1@REDACTED<mailto:trends1@REDACTED>)9> Pid ! {self(),R}
(trends1@REDACTED<mailto:trends1@REDACTED>)9> .
{<0.52.0>,
 {ok,[{1,
       [[{"author_id","users-a9KsYTIRdrq3iuAywiLRE2BGd3Y"},
         {"timestamp","2013-07-17 10......................
5、The Pid state after  send R data, The memory is increasing.
(trends1@REDACTED<mailto:trends1@REDACTED>)11> erlang:process_info(Pid, memory).
{memory,51356672}
(trends1@REDACTED<mailto:trends1@REDACTED>)12> erlang:process_info(Pid).
[{current_function,{test4,loop,0}},
 {initial_call,{test4,loop,0}},
 {status,waiting},
 {message_queue_len,0},
 {messages,[]},
 {links,[]},
 {dictionary,[]},
 {trap_exit,false},
 {error_handler,error_handler},
 {priority,normal},
 {group_leader,<0.31.0>},
 {total_heap_size,6419485},
 {heap_size,6419485},
 {stack_size,1},
 {reductions,2002},
 {garbage_collection,[{min_bin_vheap_size,46368},
                      {min_heap_size,233},
                      {fullsweep_after,0},
                      {minor_gcs,0}]},
 {suspending,[]}]
6、to send R data again.
(trends1@REDACTED<mailto:trends1@REDACTED>)13> Pid ! {self(),R}.
{<0.52.0>,
 {ok,[{1,
7、The memory  continue to increas.
(trends1@REDACTED<mailto:trends1@REDACTED>)14> erlang:process_info(Pid, memory).
{memory,125381272}
(trends1@REDACTED<mailto:trends1@REDACTED>)15> erlang:process_info(Pid).
[{current_function,{test4,loop,0}},
 {initial_call,{test4,loop,0}},
 {status,waiting},
 {message_queue_len,0},
 {messages,[]},
 {links,[]},
 {dictionary,[]},
 {trap_exit,false},
 {error_handler,error_handler},
 {priority,normal},
 {group_leader,<0.31.0>},
 {total_heap_size,15672560},
 {heap_size,15672560},
 {stack_size,1},
 {reductions,4003},
 {garbage_collection,[{min_bin_vheap_size,46368},
                      {min_heap_size,233},
                      {fullsweep_after,0},
                      {minor_gcs,0}]},
 {suspending,[]}]

I am  wondering the cause why  it is can not collect, Please give me some idea. Thanks a lot.
(trends1@REDACTED<mailto:trends1@REDACTED>)23> erlang:process_info(Pid, memory).
{memory,195907792}


I'm running Erlang R14B04 on the unbuntu.
Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false]和
Linux jason-lxw 3.2.0-55-generic #85-Ubuntu SMP Wed Oct 2 12:29:27 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

[1]代码：
  1 -module(test4).
  2 -export([get_pid/0,
  3          get_data/1,
  4          get_pids/1,
  5          loop/0]).
  6
  7 get_pids(L) ->
  8     get_pids(0, L).
  9
 10 get_pids(0, L) ->
 11     L;
 12 get_pids(N, R) ->
 13     NewR = [get_pid()|R],
 14     get_pids(N-1, NewR).
 15
 16 get_pid()->
 17      spawn_opt(?MODULE, loop, [], [{fullsweep_after,0}]).
 18      %spawn(?MODULE, loop, []).
 19
 20 get_data(Pid) ->
 21      Pid ! {self(), get}.
 22
 23 loop() ->
 24     receive
 25         {From, Data} ->
 26             From ! Data,
 27             loop()
 28     end.


Cheers
Jason
--
只为成功找方法，不为失败找理由



--
只为成功找方法，不为失败找理由
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20140115/919233be/attachment.htm>