[erlang-questions] mochiweb https/ssl

[sasa]

Oops, I have accidentally replied to sender, rather than to the group.
Sorry :-(
Here goes again for the group...


Hello Chandru, 

thank you for taking the time to read my posts and respond!

On Fri, 2011-05-06 at 21:30 +0100, Chandru wrote:


>         The behavior was somewhat better - the load test of about 2000
>         requests would work correctly for about 30 seconds, then all
>         subsequent requests would fail. When using erlang ssl, such
>         load test
>         would fail immediately. 
> 
> 
> Which side do you think they failed? Server or client?


It's a very good question, since you use the word think :-)
I strongly suspect it is the server. There are couple of reasons, most
significant one being that the client works perfectly in the
configuration where stunnel is on the separate machine.



>         I have then moved stunnel proxy to a separate server, and then
>         it
>         worked perfectly. 
>         
>         Currently, I am suspecting that cpu load might be the problem.
>         When I
>         had stunnel on the same machine as erlang server, cpu load was
>         going
>         high. The same thing happened when I used erlang ssl. When
>         stunnel is
>         on a separate machine, the cpu load on both machines is fairly
>         low. 
> 
> 
> That is quite strange.  What is the spec of the machine? This might be
> a peculiarity of the virtual instances you get on EC2. Is there any
> chance you can repeat this test on dedicated hardware? When running
> both on the same machine, have you monitored which process eats more
> CPU? beam or stunnel?


It is the "large" EC2 instance with two cores and 7.5 GB of memory. The
same type of instance sustained 12 hours of http load testing with 20
000 clients and is running the production system serving about 700 http
users on average (about 1000 at the peak) since January this year.
Unfortunately I don't have dedicated hardware available. On monday I
will try to test it again using four cores instance on the server.

As for your last question, I didn't do exact measuring, but from what I
have noticed, the beam was doing some more work. Also, when they were on
the separate machines, cpu load of the beam server was 2x higher than
the stunnel one. Both combined were significantly smaller (about 10x)
then the cpu load which occurred when they were on the same machine.

The beam server code is very simple. Unfortunately I don't have it here
right now, as I am posting this from my home, but I can post the code on
Monday if necessary.


> I remember a similar situation in the past trying to run load tests on
> EC2. It was hard to get a predictable set of results when load testing
> because the performance for the same test used to fluctuate.


Yes, I also suspect it is hard to get the exact results on the cloud,
which is why I originally mentioned I am using it.
However, from the load tests I did couple of months ago, and from the
current batch, I feel I can get approximation which is good enough. The
tests seemed to  work or fail consistently so far.



>         I cannot explain why it works better with stunnel. Do I have
>         to
>         manually configure ssl sessions when using erlang? As I
>         mentioned
>         earlier, the client does long polling i.e. it sends an
>         request, and as
>         soon as the request is responded, the new one is made. Is it
>         possible
>         that ssl handhsake occurs for every request? 
> 
> 
> I doubt the handshake occurs for every request. It might have
> something to do with the fact that stunnel is written in C so is
> likely to benefit in terms of performance. You could try hipe
> compilation of the ssl application to see if that gives you any better
> performance.


Well it is the erlang client, and in fact, I'm using the ibrowse library
and not httpc. There is no explicit ssl connection, but rather an
infinite loop of https requests. At the client side, on what level is
the SSL session being maintained: OS, erlang  or the http(s) library?

I will try to hipe compile it, thank you for the suggestion. Are there
any instructions out there, which you could recommend, or should I
google it myself?


> 
> cheers
> Chandru
> 


Thank you again for taking the time to help me.

Best regards,
Sasa
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110507/d00e6ca8/attachment.htm>