[erlang-questions] benchmarks game harsh criticism (was Learning Erlang from the scratch)

[Ulf Wiger (TN/EAB)]

Joe Armstrong skrev:
> 
>      If A is good at solving X then use it to solve X. If B is
 > good at solving Y then use it to solve Y.
> 
>      Suppose we change B to make it good at solving X, what have
 > we done? - make yet another variant of A.

To quote Todd Proebsting:

"Languages (or language technologies) that solve real problems
can succeed
- Even if slow
- Even with simple types
- Even without academic significance
- Even without rocket science
- If useful"

(http://research.microsoft.com/~toddpro/papers/disruptive.ppt)


>    So sure we could make Erlang a lot faster - for example to
 > make regexps really fast all we have to do is link the C code
 > for regexps into the Erlang kernel - but it would no longer be
 > safe.

Yes, but efficient string processing - esp efficient string
processing based using binaries - is very important.

As I understand it, regexps can be used as long as it's a
safe subset.



> I think its nonsense to assume that
> a single language will be good at everything.

Agreed.


> Benchmarks are useful here -
> but they hardly tell me something I don't know already.

Still, the benchmarks game can be used (coupled with common
sense) to identify problems that may not be that obvious
in existing products, but would be in other interesting ones.

One example is the infamous line-oriented file input problem.
This is not a problem in /our/ systems, because if we read data
from disk, it is seldom, and when it happens, it's usually
mnesia using disk_log chunking. Disk_log /is/ fast, notably
so.

When playing with the benchmark dealing with line-oriented
input, I experimented with the line-oriented socket option.
It was fast - so fast, in fact, that the Erlang program couldn't
keep up, even though it ran in the tightest loop possible.
To solve this, we wouldn't have to make the system unsafe. We'd
need to implement flow control on port input, much like that
which already exists in the inet driver. Erlang would be better
for it - not worse.

I think this is a good finding.

BR,
Ulf W