[erlang-questions] ssl_connection process hibernation option

[Kaiduan Xie]

Ingela,

I also found ssl_connection_sup consumes a lot of memory. For the
system with 20K SSL connections, ssl_connection_sup eats 65,638,608
bytes (65M), so for each SSL connection, ssl_connection_sup itself
takes about 3K bytes. Do you have any idea to reduce this? This result
is taken from R14A on x86/SMP.

Thanks for help,

/Kaiduan

(erlsip_proxy@REDACTED)6> process_info(pid(0,66,0)).
[{registered_name,ssl_connection_sup},
 {current_function,{gen_server,loop,6}},
 {initial_call,{proc_lib,init_p,5}},
 {status,waiting},
 {message_queue_len,0},
 {messages,[]},
 {links,[<0.16515.0>,<0.133.1>,<0.4229.1>,<0.6277.1>,
         <0.7301.1>,<0.7813.1>,<0.8069.1>,<0.8197.1>,<0.8261.1>,
         <0.8293.1>,<0.8309.1>,<0.8317.1>,<0.8321.1>,<0.8325.1>,
         <0.8327.1>,<0.8323.1>,<0.8319.1>,<0.8313.1>,<0.8315.1>,
         <0.8311.1>|...]},
 {dictionary,[{'$ancestors',[ssl_sup,<0.62.0>]},
              {'$initial_call',{supervisor,ssl_connection_sup,1}}]},
 {trap_exit,true},
 {error_handler,error_handler},
 {priority,normal},
 {group_leader,<0.61.0>},
 {total_heap_size,8102320},
 {heap_size,1682835},
 {stack_size,9},
 {reductions,2046919},
 {garbage_collection,[{min_bin_vheap_size,46368},
                      {min_heap_size,233},
                      {fullsweep_after,65535},
                      {minor_gcs,6}]},
 {suspending,[]}]
(erlsip_proxy@REDACTED)7> process_info(pid(0,66,0),memory).
{memory,65638608}


On Tue, Feb 8, 2011 at 2:19 AM, Ingela Andin <ingela@REDACTED> wrote:
> Hi!
>
> I think hibernation should be optional and if you choose to use it, it
> could have configurtion parameters such as the
> one you suggest.
>
> Regards Ingela Erlang/OTP team - Ericsson AB
>
> 2011/2/7 Kaiduan Xie <kaiduanx@REDACTED>:
>> Ingela,
>>
>> May I have a suggestion to have the option to set the idle time for
>> SSL connection so that after a mount of time of idle, the SSL
>> connection will be put into hibernation?
>>
>> Thanks,
>>
>> /Kaiduan
>>
>> On Mon, Feb 7, 2011 at 3:32 AM, Ingela Andin <ingela@REDACTED> wrote:
>>> Hi!
>>>
>>> We (the OTP team) would consider it.
>>>
>>> Regards Ingela Erlang/OTP team - Ericsson AB
>>>
>>>
>>> 2011/2/4 Jeroen Koops <koops.j@REDACTED>:
>>>> Hi Ingela,
>>>> Thanks for your response. Suppose I come up with a patch for this, do you
>>>> think the OTP team would consider it for inclusion in an upcoming release?
>>>>
>>>> On Fri, Feb 4, 2011 at 4:51 PM, Ingela Andin <ingela.andin@REDACTED> wrote:
>>>>>
>>>>> Hi!
>>>>>
>>>>> We already put some effort into cleaning the state of the gen_fsm
>>>>> processes holding the connection,
>>>>> but some form of hibernation could be interesting too we will take
>>>>> this into consideration.
>>>>>
>>>>> Regards Ingela Erlang/OTP team - Ericsson AB
>>>>>
>>>>> 2011/2/4 Jeroen Koops <koops.j@REDACTED>:
>>>>> > I would very much appreciate this too.
>>>>> >
>>>>> > As a test, I inserted a proxy-module between the gen_fsm and
>>>>> > ssl_connection
>>>>> > callback, that changes all responses from the ssl_connection callback to
>>>>> > include the 'hibernate' option (it's not a hack -- it's AOP!) and the
>>>>> > savings are dramatic: before the change an SSL connection used around
>>>>> > 350
>>>>> > kB, after the change only around 600 bytes. If a system is maintaining
>>>>> > thousands of SSL connections, this really adds up.
>>>>> >
>>>>> > If we could somehow get hibernation built in to the ssl application,
>>>>> > that
>>>>> > would be great.
>>>>> >
>>>>> > On Fri, Feb 4, 2011 at 9:57 AM, Ferenc Holzhauser <
>>>>> > ferenc.holzhauser@REDACTED> wrote:
>>>>> >
>>>>> >> Hi,
>>>>> >>
>>>>> >> I'm using R14B01 new SSL to set up a lot of (not very active) client
>>>>> >> connections.
>>>>> >>
>>>>> >> I have noticed that ssl_connection processes use a lot of memory
>>>>> >> between
>>>>> >> GC's.
>>>>> >> A forced GC makes them small again until the next piece of received
>>>>> >> data.
>>>>> >>
>>>>> >> This adds up to a significant memory usage (around 350-400 kB/session
>>>>> >> on a
>>>>> >> x64 system in my case) limiting the number of possible sessions.
>>>>> >>
>>>>> >> An option to hibernate this gen_fsm would decrease the average memory
>>>>> >> usage
>>>>> >> in this scenario.
>>>>> >>
>>>>> >> Would it be possible to introduce this option?
>>>>> >>
>>>>> >> Thanks in advance,
>>>>> >> Ferenc
>>>>> >>
>>>>> >
>>>>
>>>>
>>>
>>> ________________________________________________________________
>>> erlang-questions (at) erlang.org mailing list.
>>> See http://www.erlang.org/faq.html
>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
>>>
>>>
>>
>> ________________________________________________________________
>> erlang-questions (at) erlang.org mailing list.
>> See http://www.erlang.org/faq.html
>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
>>
>>
>