[erlang-questions] 'ssl_otp_session_cache' ets table

Bogdan Andu bog495@REDACTED
Thu Nov 6 14:39:09 CET 2014


Hi,

i have tried to pass to execute th following forms of commands:

erl  -ssl session_lifetime 60

erl  -ssl session_lifetime '60'

erl  -ssl session_lifetime '[60]'


in order to set a ssl session_lifetime of 60 seconds.

in init of ssl_manager , after session_lifetime + 5 seconds a process is
created to validate or invalidate sessions.
So, after 65 seconds a foldl is called on sessions table to check sessions
after formula:
Now - TimeStamp < LifeTime

However, even after I stop the client, the table 'ssl_otp_session_cache' 's
size remains the same.


After that once every ?CLEAN_SESSION_DB which is 60 seconds , the table is
sweeped to delete any expired sessions,

but also nothing happens.


On Thu, Nov 6, 2014 at 12:28 PM, Ingela Andin <ingela.andin@REDACTED>
wrote:

> Hi!
>
>
> 2014-11-06 8:53 GMT+01:00 Bogdan Andu <bog495@REDACTED>:
>
>> Hi,
>>
>> on my production servers I have relayd (on OpenBSD) daemon as a reverse
>> proxy to some webservers
>> where one can fine tune some connection parameters, as well as some ssl
>> parameters.
>>
>> I give a snippet from a relayd.conf configuration file on one of my
>> production server:
>>
>> .............
>> #
>> # Relay and protocol for HTTP layer 7 loadbalancing and SSL acceleration
>> #
>> http protocol www_ssl_prot {
>>         header append "$REMOTE_ADDR" to "X-Forwarded-For"
>>         header append "$SERVER_ADDR:$SERVER_PORT" to "X-Forwarded-By"
>>         header change "Connection" to "close"
>>
>>         response header change "Server" to "Apache 0.1"
>>
>>         # Various TCP performance options
>>         tcp { nodelay, sack, socket buffer 65536, backlog 128 }
>>
>>         ssl { no sslv2, no sslv3, tlsv1, ciphers "HIGH" }
>>         ssl session cache disable
>> }
>>
>> the last directive tells relayd not to use ssl cache.
>>
>> This configuration is working for years and relayd was restarted once by
>> accident - my fault.
>>
>> SO, y question is:
>>
>> can we have this configurable in Erlang, in other words, we might be able
>> to start an erlang vm such as:
>>
>> erl -ssl session_cache 'disable' -name x@REDACTED ....
>>
>> The ssl option session_cache can be set to disabled by default and can
>> take values either disable or enabled.
>>
>>
>>
> You can already disable the reuse of the sessions using the server option *{reuse_sessions,
> boolean()}* which default to true.
> The thing we plan to do is to have a configurable limit on the table size
> when sessions are reused.
>
> Regards Ingela Erlang/OTP team Ericsson AB
>
>
>
>
>
>
>> Just to let you know...
>>
>> The statistics of a node running from yesterday when  I opened this
>> thread of discussion, using a single client:
>>
>> Node: 'n1@REDACTED' (Connected) (17/6.2) unix (openbsd 5.4.0) CPU:2 SMP
>> +A:10
>> Time: local time 09:48:48, up for 000:20:27:49, 0ms latency,
>> Processes: total 681 (RQ 0) at 610275 RpI using 11526.4k (11805.8k
>> allocated)
>> Memory: Sys 94835.5k, Atom 407.7k/419.5k, Bin 176.4k, Code 9934.7k, Ets
>> 80739.2k
>>
>> ...........
>>
>> So, can be this made configurable?
>>
>>
>>
>> On Wed, Nov 5, 2014 at 8:52 PM, Ingela Andin <ingela.andin@REDACTED>
>> wrote:
>>
>>> Hi!
>>>
>>> 2014-11-05 14:15 GMT+01:00 Tony Rogvall <tony@REDACTED>:
>>>
>>>> I think the ssl session times is the problem here, and the lack of a
>>>> maximum size.
>>>>
>>>> You can change the session time in the ssl environment:
>>>> session_lifetime
>>>> The default is set to 24 hours (in seconds) (if I read it correctly, in
>>>> ssl_manager.erl)
>>>>
>>>
>>> Default values are always hard. It is the maximum recommended time for a
>>> session to live
>>> according to the spec.
>>>
>>>
>>>> I guess that a session_cache_size could be a nice thing to have,
>>>> limiting the growth of the session cache.
>>>>
>>>>
>>> Yes I agree.
>>>
>>>
>>>> In other words you have to estimate the life time of your clients and
>>>> try to find a reasonable session_lifetime to match that, without
>>>> blowing up
>>>> the system.
>>>>
>>>> Maybe the ssl_session_cache_api could be used to implement a strategy
>>>> with a max size
>>>> cache. Retire session least recently used, while performing the update?
>>>> There is a time_stamp in the session that that could be used for this
>>>> purpose.
>>>>
>>>> OTP: Why is a fixed limit not implemented in the standard
>>>> ssl_session_cache?
>>>> Could this be a target for DOS attacks?
>>>>
>>>
>>>
>>> We are aware of the problem, and it is on our todo list. One reason it
>>> has not had top priority is that on the server side there are often other
>>> mechanisms
>>> like firewalls and webbserver settings that limits the problem.  And the
>>> reason why it was not implemented in the first place is that
>>> implementations by nature
>>> are iterative and at first you are faced with a lot of bigger problems
>>> to solve and then you need to iterate and fine tune and fix things that you
>>> now have a better understanding of.
>>>
>>> In current master there is a change to the session table that limits the
>>> growth on the client side if the client behaves inappropriate. It also
>>> splits the session table into
>>> a server and a client table which is a better implementation as the same
>>> Erlang node can be both a client and a server at the same time. So if
>>>  someone feels like contributing a max limit please base it on the master
>>>  branch, otherwise I suspect someone compiling about it did raise the
>>> priority level  a little.
>>>
>>>
>>> Regards Ingela Erlang/OTP team - Ericsson AB
>>>
>>>
>>>
>>>> /Tony
>>>>
>>>> > On 5 nov 2014, at 13:08, Bogdan Andu <bog495@REDACTED> wrote:
>>>> >
>>>> >
>>>> >
>>>> > Hi,
>>>> >
>>>> > I performed a series of test regarding the an Erlang SSL server.
>>>> >
>>>> > In this setup a major role is played by the table called
>>>> 'ssl_otp_session_cache', and of course the processes using it.
>>>> >
>>>> > The problem is that the size of table increases constantly and,
>>>> because an ets table does not automatically deallocate memory unless the
>>>> object are deleted from that table, the size of table remains the same even
>>>> if there no ssl connections to server.
>>>> >
>>>> > For example, with a single client running 'ad infinitum' the table
>>>> increases at a rate of 5 MBytes/hour. In 12 hours there are allocate around
>>>> 60 MB of memory only for this table.
>>>> >
>>>> > Some info about this:
>>>> > $ erl -sname console@REDACTED -remsh n1@REDACTED -setcookie operator
>>>> > Erlang/OTP 17 [erts-6.2] [source] [64-bit] [smp:2:2]
>>>> [async-threads:10] [kernel-poll:false]
>>>> >
>>>> > Eshell V6.2  (abort with ^G)
>>>> >
>>>> >
>>>> > (n1@REDACTED)1> ets:i().
>>>> >  id              name              type  size   mem      owner
>>>> >
>>>> ----------------------------------------------------------------------------
>>>> >  12              cookies           set   0      291      auth
>>>> >  4111            code              set   410    26132    code_server
>>>> >  8208            code_names        set   58     7459     code_server
>>>> >  12307           httpc_manager__session_cookie_db bag   0      291
>>>>   httpc_manager
>>>> >  16404           ssl_otp_cacertificate_db set   0      291
>>>> ssl_manager
>>>> >  20501           ssl_otp_ca_file_ref set   0      291      ssl_manager
>>>> >  24598           ssl_otp_pem_cache set   3      360      ssl_manager
>>>> >  28695           ssl_otp_session_cache ordered_set 138057 8421893
>>>> ssl_manager
>>>> >  32797           dets              duplicate_bag 2      308      dets
>>>> >  40990           ign_requests      set   0      291
>>>> inet_gethost_native
>>>> >  45087           ign_req_index     set   0      291
>>>> inet_gethost_native
>>>> >  2261635104      shell_records     ordered_set 0      81
>>>>  <0.30638.37>
>>>> >  ac_tab          ac_tab            set   33     2216
>>>>  application_controller
>>>> >  code_map        code_map          set   100    2791     <0.72.0>
>>>> >  config          config            set   12     892      <0.72.0>
>>>> >  dets_owners     dets_owners       set   1      298      dets
>>>> >  dets_registry   dets_registry     set   1      299      dets
>>>> >  file_io_servers file_io_servers   set   1      344      file_server_2
>>>> >  global_locks    global_locks      set   0      291
>>>> global_name_server
>>>> >  global_names    global_names      set   0      291
>>>> global_name_server
>>>> >  global_names_ext global_names_ext  set   0      291
>>>> global_name_server
>>>> >  global_pid_ids  global_pid_ids    bag   0      291
>>>> global_name_server
>>>> >  global_pid_names global_pid_names  bag   0      291
>>>> global_name_server
>>>> >  httpc_manager__handler_db httpc_manager__handler_db set   0
>>>> 291      httpc_manager
>>>> >  httpc_manager__session_db httpc_manager__session_db set   0
>>>> 291      httpc_manager
>>>> >  inet_cache      inet_cache        bag   0      291      inet_db
>>>> >  inet_db         inet_db           set   29     600      inet_db
>>>> >  inet_hosts_byaddr inet_hosts_byaddr bag   0      291      inet_db
>>>> >  inet_hosts_byname inet_hosts_byname bag   0      291      inet_db
>>>> >  inet_hosts_file_byaddr inet_hosts_file_byaddr bag   0      291
>>>> inet_db
>>>> >  inet_hosts_file_byname inet_hosts_file_byname bag   0      291
>>>> inet_db
>>>> >  models          models            set   3      28952    <0.72.0>
>>>> >  sys_dist        sys_dist          set   1      334      net_kernel
>>>> > ok
>>>> > (n1@REDACTED)5> ets:info(ssl_otp_session_cache).
>>>> > undefined
>>>> > (n1@REDACTED)7> (8421893*8)/1024.
>>>> > 65796.0390625
>>>> > (n1@REDACTED)8> memory().
>>>> > [{total,92699464},
>>>> >  {processes,8964000},
>>>> >  {processes_used,8963152},
>>>> >  {system,83735464},
>>>> >  {atom,429569},
>>>> >  {atom_used,421768},
>>>> >  {binary,199040},
>>>> >  {code,10411520},
>>>> >  {ets,69163032}]
>>>> >
>>>> > The memory allocated to table 'ssl_otp_session_cache' is roughly 64
>>>> MB in 12 hours.
>>>> >
>>>> > On an OpenBSD platform such process gets killed immediately it hits
>>>> some memory and/or CPU limits.
>>>> >
>>>> > To make this test on OpenBSD I had to put 'infinit' to memory,
>>>> otherwise the Erlang VM would be killed.
>>>> >
>>>> > How can one control , tweak or configure this table such that it does
>>>> not accumulate such data at such high rate.
>>>> >
>>>> > I seems the table being created private, and there is no way to
>>>> ets:delete_all_objects/1 from table manually.
>>>> >
>>>> > I know that this table caches some SSL data related to clients, but
>>>> the client has the same IP address,
>>>> > and I wonder why is neccesary to store a lot of SSL connection info
>>>> about the same client when only the ephemeral peer port
>>>> > differs?
>>>> >
>>>> > How the size of this table can be held in reasonable limits and the
>>>> rate it's size increases ?
>>>> >
>>>> > Please if somebody shed some light on these issues.
>>>> >
>>>> > Thank you,
>>>> >
>>>> > Bogdan
>>>> >
>>>> >
>>>> > _______________________________________________
>>>> > erlang-questions mailing list
>>>> > erlang-questions@REDACTED
>>>> > http://erlang.org/mailman/listinfo/erlang-questions
>>>>
>>>> "Installing applications can lead to corruption over time. Applications
>>>> gradually write over each other's libraries, partial upgrades occur, user
>>>> and system errors happen, and minute changes may be unnoticeable and
>>>> difficult to fix"
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> erlang-questions mailing list
>>>> erlang-questions@REDACTED
>>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>>
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20141106/9a5a9710/attachment.htm>


More information about the erlang-questions mailing list