obscure erlang-related publication

[Ulf Wiger (AL/EAB)]

Richard A. O'Keefe wrote:
>
> Matthias Lang <matthias@REDACTED> wrote about the paper
> 	 > "A comparison of six languages for system level 
> 	 > description of telecom applications"
> 	 > Jantsch, Kumar, Sander et al.
> 	 > http://www.imit.kth.se/~axel/papers/2000/comparison.pdf
> to the effect that he wasn't really impressed.
> 
> Neither was I.

Fair enough.
I wasn't very clear about merely suggesting that the 
article be added to the publications list, or some 
other list that proposes to give an overview of 
available research related to Erlang.

This is what the text under erlang.se/publications/
says:

"This page contains articles, books, papers, Powerpoint 
presentations, and Master's Theses that relate or refer 
to Erlang/OTP."

It doesn't suggest that all articles have been subjected
to peer review and collectively approved by the erlang
community (they haven't), or that all articles measure
up to some standard of being "impressive" (they don't).

Having said this, I think it's a good thing that a
discussion about the quality of available articles is
called into question.

I have no personal stake in this particular article

> They are comparing languages for *hardware* and software 
> concurrent systems.
> 
>  - So they don't include Ada, which is a mature language with 
> excellent
>    tool support and good support for concurrency and 
> structuring.  (And
>    was in 2000.)
> 
>  - So they include C++, and then find to their great surprise that
>    it doesn't do concurrency.  They _don't_ consider any of the
>    parallel/distributed versions of C++.

A surprising number of people in industry are of the 
opinion (mainly based on hearsay, I believe) that C++
_is_ a good systems description language - or alternatively
that there are programming languages and system description
languages (= UML), and no programming languages are 
particularly better than C++.

(One could have extended the paper with some words about
why one would want to evaluate a programming language 
as a system description language, but perhaps that's 
fodder for an entirely different paper...?)

Many researchers, for better and for worse, have a 
tendency to focus their research on what they think
industry wants (UML, Java, C++, ...)

>  - So they include Haskell, but find to their great surprise that
>    it doesn't do concurrency.  They _don't_ consider 
> Concurrent Haskell.
>    (Which I am pretty sure was around in 2000.  Certainly Concurrent
>    Clean, which is pretty Haskell-like, was around then, and really
>    was concurrent, although recent versions aren't.)

In fairness, they do write that their purpose of the
evaluation is to "illustrate, how giving high or low 
importance to a particular aspect affects the relative 
performance of a language", and:

"One motivation for this selection was to cover different 
paradigms and aspects. Another practical reason was that 
these languages are well known by the authors."


>  - They DO try to ensure that they aren't just
>    reporting their prejudices by writing an
>    application of the kind they care about in 
>    the several languages, but then they
>    deliberately choose to say nothing about the
>    code they got or their experience of writing it.

I agree that this was disappointing, and I had difficulty finding a good
reason for it, other than that it would have meant more work before
publishing the paper.


> Their evaluation method basically amounts to making
> your preconceived ideas of what kind of solution you
> are looking for seem respectable by wrapping
> (arbitrary!) numbers around them.

Do you have a link to a better method?
The evaluation described in the paper is _far_ less
arbitrary than many evaluations I've witnessed 
elsewhere. I think the way these comparisons are
handled in industry is sorely lacking.


> The other thing I learned was that I have been a 
> complete fool to myself by waiting until I actually had some 
> results worth discussing before publishing ideas.
> Now I know how to get my publication count high...

As much as I would welcome more papers from you,
there is also great value in trusting that when 
certain people _do_ publish, it will be well worth
reading.  (:

BR,
Ulf W