[erlang-questions] Scaling SSL wrapped websockets - experiences

[Erik Søe Sørensen]

In my experience with many simultaneous SSL connections in Erlang, what
surprised me was that as default, Erlang is configured to have
reusable/continuable SSL sessions. This means that some ETS table
containing the session info can grow fairly large. That feature can be
disabled, however, which helps a great deal on memory usage.

- Erik

2017-07-03 20:10 GMT+02:00 Dmitry Kolesnikov <dmkolesnikov@REDACTED>:

> Hello,
>
> There was posts at Internet about 2M connection per node.
> http://www.phoenixframework.org/blog/the-road-to-2-
> million-websocket-connections
>
> I’ve scale it up to 1M connection on AWS long time ago. This numbers is
> “easy” to achieve out of box for system with low traffic. I had the
> following traffic pattern 20 connection per second (poisson arrival rate),
> connections stays idle but clients shows 3 second think time (poisson rate).
>
> The major concern was a memory all the time. The out of box deployment on
> AWS large instance was capable to handle 300K connections before OOM. I’ve
> use hibernate feature aggressively to reduce memory demand. Finally, I’ve
> used 2xlarge node to run the system.
>
> The latency was not an issue. The message scheduling was good while CPU
> utilization was below 70%. However, I’ve reduce logging to error-level and
> made the nodes responsible for message routing only. I’ve learn that you
> need to keep in balance CPU/Memory to get acceptable results but this
> depends on your traffic.
>
> Unfortunately, that set-up excluded SSL. I was not successful to scale SSL
> up to 1M due to heavier memory demand (I’ve user OTP 18 for that problem).
> Unfortunately, I do not recall exactly the number of concurrent SSL
> connection (cannot find the data sheets). It was high >>20K but <<100M for
> sure.
>
> Frankly speaking, I would design a system around ELB to terminate SSL
> traffic and use weighted load balancing feature to scale ELB connectivity
> then rush for high concurrency on single node due to operational simplicity.
>
> I hope, I shed some light on your problem.
>
> Best Regards,
> Dmitry
>
>
> > On Jul 2, 2017, at 12:56 PM, Paul Peregud <paulperegud@REDACTED> wrote:
> >
> > I'm looking at SSL + websocket + cowboy scaling with regards to number
> > of simultaneous connections. I'm interested in relationship between
> > number of connections and latency of message delivery. Is there any
> > data on performance of SSL implementation in BEAM?
> >
> > Best regards,
> > Paul Peregud
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://erlang.org/mailman/listinfo/erlang-questions
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20170703/894c832f/attachment.htm>