[erlang-questions] Testing a large, heterogeneous system

[David Welton]

Hi,

I'm looking for ideas on testing a system we're developing.

Currently we have nothing, so anything is better than that.  We're
looking to get the most 'bang for our buck', so to speak, so tests
that cover, even if roughly, more code are going to be better in some
ways that something that's too specific to a small part of the system.

Erlang, in our project, sits in the middle: it offers a web interface
that uses lots of Javascript via Chicago Boss, manages some web
sockets, talks to a database (Postgres), interacts with several C
nodes, and also manages various external programs via erlexec and
open_port.  To add some complexity, specialized hardware is involved
and dealt with via the C programs/nodes.

I read the chapter on Common Test in Learn You Some Erlang, and that
looks like a pretty good platform.  What I'm struggling with is what
sort of strategy to use to test the whole thing.

Things I'm dubious about:

* How to handle the external bits?  It'd be nice if the tests could be
run on a developer's machine without requiring the hardware.  But I
could also see something rigged up to interact with the hardware being
valuable as well.  Forced to choose, I'd go with the former, I think.

* The interaction with the hardware stuff is kind of complex and
involves a lot of state, and messages going back and forth and input
from the user via the web, storing stuff in the database and so on.
The web portions of things... maybe could be tested in isolation to a
certain point, but the sequence of messages going back and forth is
tricky and probably the place where tests will give us the most
benefits.   I'm used to tests that try and test fairly small chunks of
a system, but I'm not sure how feasible that is in this case - the
setup and teardown might involve a lot of stuff just for a small
test...

Sorry if this is all a bit muddled - any advice is appreciated!

Thank you,
--
David N. Welton

http://www.welton.it/davidw/

http://www.dedasys.com/