[erlang-questions] Erlang Web Servers challenge

[Joe Armstrong]

On Wed, Jul 6, 2011 at 1:42 PM, Zabrane Mickael <zabrane3@REDACTED> wrote:
> What are you trying to show with this benchmark? That is, what is the
> goal of it?
>
> At the end of the day, I'll try to compare Erlang Web servers with others
> competitors (not written in Erlang).
>
> Serving a static file is something nginx and a couple of
> Haskell HTTP servers are very good at (see the Warp-project).
>
> Please, just follow and stick to the rules.
> Nginx or Varnish (https://www.varnish-cache.org/) or any HTTP accelerator
> are out of this challenge scope.
>
> This violates Rule 1 in Mark Nottingham's http benchmark advice:
> http://www.mnot.net/blog/2011/05/18/http_benchmark_rules
>
> I'm aware of that rule (Steve Vinoski
> post: http://steve.vinoski.net/blog/2011/05/09/erlang-web-server-benchmarking/).
> Thanks anyway.
> Why it's so hard to just play the game guys?

Because if you propose a new game then people will question and try to
change the rules.

That's how baseball got invented, and I know cricket is far better,
but that is a bit off-topic here.

In the particular case just about the last thing I want to know about
is how fast I can serve static content - there is an entire industry devoted
to exactly this question.

Erlang was designed for building fault-tolerant systems, it was *not* designed
for building fast systems. I would expect therefore that it would do badly
at things that non-fault tolerant systems are good at (like serving
static content).

(( I'm sure we could build a fast content delivery network in Erlang -
but this is not easy - the low hanging
fruit was taken year ago - and it's got nothing to do with Erlang, its
all about algorithms, caching, locality
peering agreements  etc.))

Like it or not, the results of such a measurement will be
misinterpreted and abused.
The fastest ever Erlang web server should be way slower than the
fastest non-fault
tolerant web server.

A result of the form "the fastest Erlang web server is way slower than
the fastest non fault-tolerant
server" will be widely quoted and blogged, and the results will be
taken out-of-context.

For me I like to "make things work" before I "make them fast" - so the
question of exactly what happens
under stress, just before things fail is extremely interesting.

I am very interested to know exactly how and when things fail when you
stress them.

Sending 100 bytes from a file seems to measure nothing interesting.
Since it's the same file
I expect it to be read *once* then cached in memory. It's so small
that I expect no IP fragmentation at all.

You won't really be measuring the performance of the web server, but
rather the perfomance of
a tiny sub-set of gen_tcp and file. If the authors of the various web
serveres havn't made some catestripohuic
mistales in their implamentaion I expect their code to be just a thin
layer of the standard librarie rountines and
I expect the results to be broadly similar.


I always like to ask the question:

     What would you do with the results of such a measurement?

Because unless you can give a good answer to this, there does not seem
much point in doing the experiment.

/joe


> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>