Pools, pipes and filters or Stream Algebra

[Jay Nelson]

How can I accomplish a task?  This is what I really want to
do rather than use an application.  Around 1990 people were
asking me if they should buy a PC.  I usually said no because
they had no reason for one and no task that they wanted to
accomplish.  They just thought they should have one.  I started
thinking about why the average user couldn't do anything with
a computer, and why it was hopeless for most to try to write
a program.  I concluded that it was because everything was
abstract and there were no constraints on how things could be
constructed.  The most powerful and flexible tool is useless to
all but the trained professional, but a screwdriver can be used
by nearly everyone.

I embarked on a mental exercise designing a Rube-Goldbergian
programming language that consisted of physical devices and
an analog language replete with flashlights, spinning slotted disks,
paper punches and light sensors.  A hopeless exercise, but it
was driven by the belief that observable behavior and feedback
is easier for the average person to understand than abstract
concepts.  My goal at the time was a simple quest:  how would
an average person use their computer to check if they had won
the Lotto each week, by writing their own program.  It is a much
more difficult task than expected and I came away convinced that
if such a simple task was as impossible as it seemed, computers
were useless to the populace (and actually programmers can do
great harm to the populace by misleading them with programs
they don't understand -- look at all the spyware and viruses that
now circulate and the potential for propaganda unlike anything
seen on TV since the impressionable computer user believes that
he is in control).

Lately I've been worried about load balancing, distributed computing,
process management, code reuse and dynamic UIs.  I want to look
at the problems holistically rather than individually.  Iterative and OO
languages subliminally make you reason about individual objects or
single situations and prevent you from seeing how the system works
as a dynamic, adaptive organism.  I want to think about the life of
the system, the range of data it will ever see, the ebb and tide of
data flows, the whirlpools of local data feedback and the dispersal of
results all at once and try to perceive where the real issues will occur.
I also want to do this analysis and modelling without having to implement
the underlying detailed functionality.

Earlier discussions talked about piping processes together and building
dynamic, cooperative sets of processes.  My current view of what was
traditionally called an 'application' is a pool of data that resides in the
computer and streams that affect that pool.  The user is a pump that
supplies new data to the pool.  Filters are imposed between the user
and the trusted pool that represents the current internal state to keep
it from being contaminated.  The user's requests may cause the display
to reconfigure so that the view he sees is appropriate for the task he is
currently concerned about.  The screen is a pool that can siphon from
the data model pool or can receive pumped data from the data model.
Filters are what eliminate errors, remove redundant or unwanted data,
reduce the volume based on system constraints or user constraints or
otherwise transform the data before arriving at a pool.  Splitters, repeaters
and replicators can be used to handle multiple pools from a single
stream or to replenish pools that leak.  Multiple streams can likewise
be coalesced with a composite filter to generate a single pool.

Ultimately I would be interested in a Stream Algebra that encompasses
streams and the operations that may occur on them, axioms and theories
of computation equivalence using the algebra and an implementation of
the operators inherent in such an algebra.  Given such an algebra it seems
that graphical tools for developing a system would be much easier to use
than that which OO promised to deliver.  I envision something that a
plumber would feel very comfortable working with on screen.

jay