[erlang-questions] ssl_session_cache: trouble + questions

[Ingela Andin]

Hi!

2015-12-09 18:02 GMT+01:00 Danil Zagoskin <z@REDACTED>:

> Hi Lucas!
>
> I read the ssl 7.1 sources.
>
> As for current maint:
> Improved:
>   - ssl_session:valid_session already uses the new time API:
> https://github.com/erlang/otp/blob/maint/lib/ssl/src/ssl_session.erl#L70
>   - Max cache size is by default 1000 now:
> https://github.com/erlang/otp/blob/maint/lib/ssl/src/ssl_manager.erl#L69
> Left as before:
>   - Cache table is still ordered_set:
> https://github.com/erlang/otp/blob/maint/lib/ssl/src/ssl_session_cache.erl#L36
>

There is a reason for using orderd_set see:
35ffd19df295f5ff73f9968b65dc8ad957c943e5


  - Multiple concurrent validators still possible:
> https://github.com/erlang/otp/blob/maint/lib/ssl/src/ssl_manager.erl#L385
>
>
There will only be one validator instance started for the reason of hitting
the threshold (max value), but you are correct that the poller validator
assumes that the old one already should be gone. I will make a backlog item
to make sure there is only one of those, in the meantime the other changes
will hopefully makes this less of a problem.


Regards Ingela Erlang/OTP Team - Ericsson AB



> Limiting session cache size and using monotonic_time as timestamp should
> prevent our kind of problems.
>
> Thank you!
>
>
> On Wed, Dec 9, 2015 at 3:39 PM, Lukas Larsson <lukas@REDACTED> wrote:
>
>> Hello!
>>
>> You did not mention what version of ssl that you are using? If you do not
>> have the latest version, please have a look at the release notes in ssl and
>> see if any of the fixes in there applies to you. There are fixes made in
>> ssl that relate to a very large session cache.
>>
>> Lukas
>>
>> On Tue, Dec 8, 2015 at 1:15 PM, Danil Zagoskin <z@REDACTED> wrote:
>>
>>> Hi!
>>>
>>> Recently our servers started to consume lots of SYS CPU.
>>> Inside a VM top processes (by reductions per second) were ssl session
>>> validators.
>>> Most popular current function in runnable processes was
>>> calendar:datetime_to_gregorian_seconds/2.
>>> Also gproc was very slow (it uses ETS).
>>>
>>> According to `ets:i().` the largest ETS table was:
>>>     49178           server_ssl_otp_session_cache ordered_set 5015080
>>> 305919839 ssl_manager
>>>
>>> We have worked around the problem by using lower session_lifetime.
>>>
>>> But reading the code I came to these questions:
>>>   - The cache is `ordered_set` type which has logarithmic access time.
>>> Does it have to be `ordered_set`, not just `set` (with constant access
>>> time)?
>>>   - There is no protection agains running multiple validators. This
>>> leads to many processes accessing single table and doing the same work.
>>> This seems to greatly increase SYS CPU usage and slowdown in other ETS
>>> tables. Should we skip new validator start if previous one is still running?
>>>   - ssl_session:valid_session is called for every session in cache and
>>> calls `calendar:datetime_to_gregorian_seconds({date(), time()})` itself.
>>> Should we use `erlang:monotonic_time(seconds)` everywhere instead? Or maybe
>>> we should pre-calculate minimal allowed timestamp to avoid extra
>>> arithmetics?
>>>
>>>
>>> --
>>> Danil Zagoskin | z@REDACTED
>>>
>>> _______________________________________________
>>> erlang-questions mailing list
>>> erlang-questions@REDACTED
>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>
>>>
>>
>
>
> --
> Danil Zagoskin | z@REDACTED
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20151210/ab74e3f0/attachment.htm>