[erlang-questions] Erlang and bbcode

[Gordon Guthrie]

If we go back to 1999 - so I was the Chief Technical Architect at
if.comand we spend $100m on technology.

We didn't have ANY tests at all. What we did have was a testing team of I
suppose 35 full time testers and varyingly dozens to hundreds on secondment.

If you were in the business of consuming software (unlike Ericsson) you
didn't have kit lying around.

If you split out your code base into components that are pure functions
(unit testable) and components that have side effects (system testable) you
can build good test case coverage into your dev process. But if you are an
n-tier architecture in imperative/OO languages with side-effects in almost
all modules you can only do system tests.

If you are a n-tier application (web server/Java mid-tier/mainframe
backend) no developer has access to a machine that is capable of running
'the system' - so automatic system testing is simply impossible - it is not
possible write them, let alone run them many times a day.

So what you end up with is a monolithic user testing process (at
if.com/Halifax it was 12 weeks end to end MINIMUM from developers desktop
to production) with 6 major testing gates and you beat the software into
submission.

If you look at us now at Vixo (formerly Hypernumbers) we have a BFN* of
tests and we can release from dev to production many times a day.

The reason for this is that:
* most of our code is side-effect free and is built backwards from tests -
you write a test suite and tinker about until the module passes them all
* everyone has full access to a machine which has near production
performance (ie a laptop)

We rent boxes for a couple of hundred dollars a month that we paid capital
costs of £1.5m the pair in '99

Testing may well have been invented years ago but it is still fairly recent
in bespoke commercial software development.

Gordon

* BFN is the correct mathematical term for a Big Number

On 13 July 2012 13:11, Ulf Wiger <ulf@REDACTED> wrote:

>
> On 13 Jul 2012, at 13:04, Joe Armstrong wrote:
>
> > Since they have no specification, their can't really be any tests.
> >
> > Or worse, in the absence of a specification, the tests *become* the
> > specification.
>
> Well, in general, I'd have to take exception to this description ;-),
> but in the case of a markup language, pushing through without
> a specification is obviously asking for trouble.
>
> With common commercial development, there is very
> seldom a specification in any useful sense of the word. What
> there is, is a customer's more or less vague idea of what they
> want, and the reality any implementation of their idea will
> eventually face.
>
> In such situations, the tests *do* become the specification,
> because a specification *cannot* be written! If you try to write
> one anyway, it will seriously delay the project, and be obsolete
> as soon as it's finished, if it ever had a chance of being correct
> (it probably didn't).
>
> In other words, don't blame the designer who creates a test
> suite in those cases, rather than brow-beating the poor customer
> until they come up with a workable specification. The practice
> of using cue cards to write user stories (and the acceptance
> criteria on the back) evolved very much because it was a
> workable way of discussing the requirements with users,
> whether or not they had any engineering or science background.
>
> http://www.allaboutagile.com/user-story-example/
>
> It seems ridiculously vague and limited, but in practice, it actually
> helps to catch *some* of the information needed to arrive at some
> mutually understood acceptance criteria. This wasn't created
> because people didn't understand what formal specifications,
> or thorough testing, looked like - it was created in order to achieve
> at least *some* level of improvement with *real* projects and
> real customers.
>
> Besides, when did you last see a good specification of how
> a Web GUI should behave - or a good way to test it? *
>
> Another example: When I gave an invited talk at NWPT'99 [1] in
> Uppsala, Sweden, a member of the audience (probably a PdD
> student), reacted to my description of the 3-month test period
> before release of the AXD 301, saying "That's ridiculous! If you
> had written it using a language that could be formally verified,
> you wouldn't have to test at all!".
>
> I mumbled something about not knowing if that would have been
> at all possible, and Thomas Arts jumped in and offered to argue
> the point with him in the break. It was a very vivid discussion, as I
> recall.
>
> [1]
> http://www.it.uu.se/research/publications/reports/1999-008/nwpt99/proceedings/
>
> Obviously at that scale (for all except for that PhD student), there
> is no chance of arriving at a coherent formal specification, since
> the standards we were supposed to comply with were not
> themselves formally specified**, and many of the customers didn't
> necessarily want *exactly* what was in the standard anyway.
> Furthermore, many of the non-functional requirements were not
> specified at all, except as a more or less vague list of "goodness
> criteria", trying to encapsulate what it means for a system to be
> robust.
>
> What we ended up with was a large pile of informal requirements,
> and a huge set of test cases (nearly a million lines of code).
> At that point, the test suites *were* indeed the real specification,
> albeit an extremely bulky and inaccessible one.
>
> BR,
> Ulf W
>
> * One of the best is probably Cucumber (http://cukes.info/), which
> is based on cue cards, and relies on quite an ambitious mapping
> to the underlying web stack (e.g. Rails). Without such a mapping,
> it is less useful, although it still has its sweet spots.
>
> ** …although some would disagree. They are certainly specified in
> a bureaucratic way, but that's not the same thing.
>
> Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc.
> http://feuerlabs.com
>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20120713/7bbf551b/attachment.htm>