[erlang-questions] benchmarks game harsh criticism (was Learning Erlang from the scratch)

[Thomas Lindgren]

--- Joe Armstrong <erlang@REDACTED> wrote:

> I've been following various discussions about
> benchmarks, and must say
> that I am not impressed.
> 
> The general arguments seems to go like this:
> 
>     language A is good at X
>     language B is good at Y
>     language B is bad at X
> 
>    Therefore hack language B so that it is good at X
> 
>     This argument has appeared many times with
> different values of A B
> X and Y - usually by a proponent of
> A who because A solves X better than B assumes that
> A is in some sense
> better than B.
> 
>      The problem is that this argument is just plain
> daft.
> 
>      If A is good at solving X then use it to solve
> X. If B is good at
> solving Y then use it to solve Y.

On the other hand, we could also look at it from the
point of the language community. If A is useless for
doing the jobs we need to do, we can't use A. 

(It can be worse than that; if A is perceived to be
useless in one area, it runs the risk of being
perceived to be a toy, not suitable for serious use,
etc. Possibly quite unfairly, but there you are.)

Likewise, if A does not solve problems relevant to a
larger community of users, then A will be restricted
to use in its niche of competence.

>      Suppose we change B to make it good at solving
> X, what have we done? - make
> yet another variant of A.

Not if B solves X in a better way than does A.

> Scanning log files efficiently (for
> example) would
> be very important If it has to be done every few
> seconds - but if it
> is done one a year the performance is totally
> irrelevant.

Actually, that single run can still be time critical.

> - the
> only interesting question is "is this time within my
> time budget"

It can indeed be quite sufficient to be good enough.
And your other points about systems thinking, which I
have deleted, are well taken.

Best,
Thomas



      ____________________________________________________________________________________
Be a better sports nut!  Let your teams follow you 
with Yahoo Mobile. Try it now.  http://mobile.yahoo.com/sports;_ylt=At9_qDKvtAbMuh1G1SQtBI7ntAcJ