[erlang-questions] SSL memory consumption

[Hynek Vychodil]

On Tue, Jan 4, 2011 at 4:11 PM, Kaiduan Xie <kaiduanx@REDACTED> wrote:
> Thank you Hynek. I had figured out the problem. The root cause is that
> every time a gen_fsm is created in way 2, it inherits the '$ancestors'
> from its parent, and its parent is also added to the '$ancestors' of
> the process's dictionary.
>
> '$ancestors' of 1st gen_fsm: [p0],
> '$ancestors' of 2nd gen_fsm: [p0, p1],
> '$ancestors' of 3rd gen_fsm: [p0, p1, p2],
>
>
> ....
> '$ancestors' of nth gen_fsm: [p0, p1, p2, ...., pn-1]
>
> After resetting the '$ancestors' before creating gen_fsm by calling
> put('$ancestors', Parent), the memory footprint is reduced
> dramatically. What is the usage of '$ancestors'?

It is OTP related stuff (proc_lib mainly).

>
> Best regards,
>
> /Kaiduan
>
> On Tue, Jan 4, 2011 at 5:21 AM, Hynek Vychodil <hynek@REDACTED> wrote:
>> On Thu, Dec 30, 2010 at 6:32 AM, Kaiduan Xie <kaiduanx@REDACTED> wrote:
>>> After tried two different ways to call ssl:transport_accept and
>>> ssl:ssl_accept, the following observation was found,
>>>
>>> 1. Calls ssl:transport_accept and ssl:ssl_accept in a server loop, and
>>> for each SSL connection, starts a gen_fsm to receive data. The code is
>>> listed as below,
>>>
>>> loop3(LSock) ->
>>>    {ok, Socket} = ssl:transport_accept(LSock),
>>>    ok = ssl:ssl_accept(Socket),
>>>    {ok, {PeerAddr, PeerPort}} = ssl:peername(Socket),
>>>    io:format("Accepted peer:~p:~p:~p:~p~n", [self(),PeerAddr, PeerPort, I]),
>>>    {ok, Pid} = gen_fsm:start(erlsip_transport_tls_connection,
>>> {Socket, PeerAddr, PeerPort}, []),
>>>    ssl:controlling_process(Socket, Pid),
>>>    loop3(LSock).
>>>
>>> 2. Calls ssl:transport_accept, and ssl:ssl_accept in a gen_fsm. Upon a
>>> SSL connection is accepted, starts another gen_fsm to accept new SSL
>>> connection. The code is listed as below.
>>>
>>> init({server, LSocket, Parent}) ->
>>>    {ok, wait_for_connection, #state{lsocket = LSocket,
>>>                                     parent = Parent}, 0};
>>> wait_for_connection(timeout, State) ->
>>>    io:format("Waiting for TLS connection ... ~p/~p~n", [self(), State#state.par
>>> ent]),
>>>    {ok, Socket} = ssl:transport_accept(State#state.lsocket),
>>>    ok = ssl:ssl_accept(Socket),
>>>    gen_fsm:start(erlsip_transport_tls_connection,
>>>                 {server, State#state.lsocket, State#state.parent},
>>>                  []),
>>>    {ok, {PeerAddr, PeerPort}} = ssl:peername(Socket),
>>>    io:format("Accepted peer:~p/~p:~p~n", [self(), PeerAddr, PeerPort]),
>>>    {next_state, wait_for_crlf_crlf, State#state{socket = Socket,
>>>                   peer_address = PeerAddr, peer_port = PeerPort}}.
>>>
>>> Memory consumption per connection in the second way is about 120 K
>>> with R14B01 while a connection takes about 30 K in first way.
>>>
>>> I am very happy with the result, but can someone explain the difference?
>>>
>>> Thanks,
>>>
>>> /Kaiduan
>>
>> I'm just guessing but it can be GC issue. In first case you do all
>> those calls inside one process so all used structures inside loop3 can
>> be GCed. In second case you do this sequence always in new process and
>> when there are not future reductions (there is not any communication -
>> fsm doesn't work) this memory is not freed. Try add explicit GC to
>> your init/1 in second case and measure again.
>>
>>> On Thu, Dec 23, 2010 at 9:31 PM, Kaiduan Xie <kaiduanx@REDACTED> wrote:
>>>> I just replaced R14A with R14B01, each SSL connection consumes about
>>>> 120 K memory, 50K less than that of R14A.
>>>>
>>>> Thank Tony, Jesper for the help, can we reduce more?
>>>>
>>>> /Kaiduan
>>>>
>>>> On Thu, Dec 23, 2010 at 8:38 PM, Jesper Louis Andersen
>>>> <jesper.louis.andersen@REDACTED> wrote:
>>>>> On Fri, Dec 24, 2010 at 02:33, Kaiduan Xie <kaiduanx@REDACTED> wrote:
>>>>>
>>>>>> What do you mean the old ssl and new ssl, Tony?
>>>>>
>>>>> SSL was replaced recently with a new implementation and you have the
>>>>> 'new' version here if you are on R14b
>>>>>
>>>>> --
>>>>> J.
>>>>>
>>>>
>>>
>>> ________________________________________________________________
>>> erlang-questions (at) erlang.org mailing list.
>>> See http://www.erlang.org/faq.html
>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
>>>
>>>
>>
>>
>>
>> --
>> --Hynek (Pichi) Vychodil
>>
>> Analyze your data in minutes. Share your insights instantly. Thrill
>> your boss.  Be a data hero!
>> Try GoodData now for free: www.gooddata.com
>>
>



-- 
--Hynek (Pichi) Vychodil

Analyze your data in minutes. Share your insights instantly. Thrill
your boss.  Be a data hero!
Try GoodData now for free: www.gooddata.com