[erlang-questions] Erlang web servers challenge

Richard O'Keefe ok@REDACTED
Mon Jul 11 02:13:34 CEST 2011

On 9/07/2011, at 2:44 AM, Ulf Wiger wrote:
> Having said this, I invite anyone who goes through that kind of exercise to share their results. Not only will it help you evaluate the experiment honestly; it will increase the store of experiments that can be copied and tailored to the specific challenges of the next project.

I had an unpleasant experimental experience of my own last year.
Let me first give you the lesson I learned, and then the background.

LESSON: Expect your experiment to surprise you,
	probably by showing the experiment was a waste of time.

Background: I'm sick of arguments about style.  To my mind, it is so
obvious that baStudlyCaps isAVeryStupidWayToWriteIndeed and I
cannotUnderstandWhyOtherPeopleDoNotSeeThat.  But they don't.  My zeroth,
the New Zealand Anglican Church has even brought out an electronic
version of the liturgy called WePray, in a desperate attempt to seem
hip.  (Since it is only available for an operating system sold by what
may be the largest software company to have been convicted to software
piracy, I wonder what their ethics committee were doing.  But I digress.)

So I devised a little language called Chatterton
which allowed me to mechanically produce several style variants of
some sample programs and ask some 3rd year software engineering students
to find some mistakes in them.

What I expected was one of three things:
 - no measurable effect
 - more readable code (i.e., NOT baStudlyCaps) being easier to fix
 - more familiar (i.e., JustLikeXingJava) being easier to fix.

What I *got* was students telling me they couldn't read code on
paper; they needed syntax-colouring IDEs (the listings all fitted
on a single sheet of paper and used black-and-white styling) and
ideally a debugger so they could find mistakes by stepping through
the code.  I also got students telling me that it was horribly
unreasonable of me to expect them to read a 30-page manual; NOBODY
could read that much.  And finding a definition of an identifier
in a 2-page listing is just beyond human capacity; it's impossible
to do that without the machine assistance of an IDE.

So the whole experiment produced no worthwhile data for reasons having
nothing to do with what I thought I was testing.

As other people have been saying, the "Erlang web servers challenge"
is at serious risk of producing no worthwhile data.

More information about the erlang-questions mailing list