[erlang-questions] Testing a large, heterogeneous system

[Peer Stritzinger]

On 2013-08-01 09:25:58 +0000, Jesper Louis Andersen said:
> 
> I am a fan of building virtualized environments rather than mocking. 
> That is, you take something like
> 
> http://www.vagrantup.com/
> 
> and then you build up the parts which your system consists of and make 
> sure you can deploy code inside your virtual environment. For Postgres, 
> look at pg_virtuelenv. Most of the C-nodes can perhaps be pushed into 
> the virtual environment. The specialized hardware is usually nasty and 
> you may have to mock that in order to get correct operation.
> 
> But I usually hate mocking with a passion and much prefer to run 
> virtualized setups. It also forces you to think about 
> ease-of-development in general as it forces you to have an environment 
> on a box.

For embedded stuff its often hard to emulate everything (certainly nice 
to have if its not too much work to set up).  In these case I test the 
drivers that actually talk to the hardware devices on the actual 
hardware.  Also using QuickCheck either with its C interface or if the 
driver is talked to from Erlang anyway through rpc from a non embedded 
node.

For having the whole system in a box I write models for the hardware 
side and outside world in Erlang.  Usually hooking in just above the 
driver I tested like mentioned above.

These models have double use: I can use them to run tests without the 
special hardware just making it talk to the "world model" often on a 
separate node.  When using QC I can also let it interrogate the state 
of the "world model"  against the expected state.  The world model can 
also be used as directly as state based QC model for testing the system 
from the outside (wherever this is).

A extra use of these "world model" is that I can assemble complicated 
simulations of several system e.g. a whole factory plant using multiple 
stations each with its Erlang running embedded system.  This is nice to 
have for larger system tests and also for building demos, measurement, 
and as development enviroment to fiddle with the higher level aspects.

They can be considered Mocking but I don't see these models as such.  
They are not 100% of course so that they have in common with mocking.  
But when mocking one usually tries not to implement too much details.  
Weras with the simulation approach one tries to capture most of the 
essential system behaviour.

BTW: Erlang is a great language for building simulations a neat convenience.