Communicating processes vs. APIs

[Jay Nelson]

Joe wrote:

 > We might like to ask what a unix pipe:
 >
 > cat <file1 | x | y | z > file2
 >
 > Might look like in Erlang
 >
 > Perhaps we should be thinking more in terms of abstractions that
 > allow us to glue things together with pipes etc.

My thought was that a firewall router is the model of what all this
abstraction is about.  The purpose is to allow messages in, but
to filter out the bad stuff; present a unified front end for
simplicity yet allow arbitrary server complexity behind the scenes
that is opaque to the caller.

This translates into the following abstract pieces:

1) Stream accepter - the only way in
2) Filter - removes the bad stuff
3) Router - determines who is interested; acts as middleman
4) Handler - implements the behaviour the client expects

The router is the only thing that receives from the outside
and sends to the outside.  Filters may exist on both sides
of the router.

I've always complained about error handling code because
it gets in the way of the algorithms.  I've wanted a way
to write parallel code: one is the error-free thread of the
algorithm logic, and the other is only the error handling part
of the code.  Finally, erlang offers a way although the partitioning
is different: use intermediate processes to clean up the data
before it arrives at the algorithm process.  The final process
can safely assume only valid data, or die if one of the upstream
processes has a bug.  Likewise, protocol transformation can
occur a step at a time.  This approach leads to maximal
reuse of software, something that discrete objects does
not allow: the reassembly of logical pieces in a different
coherent whole.

The best thing of all this discussion is that we can design,
discuss and argue about the protocol semantics without
considering what algorithms are being implemented.
The protocol can be later tweaked without modifying the
algorithms.  Using filter processes gives the same capability
when dealing with streams that have errors in them -- the
error handling can be dealt with wholly independent of the
algorithm or its efficiency in dealing with good data.

Marrying this back to the UI thread, the user generates
the stream and the first filter notifies the user of obvious
errors without doing a round trip through the system.
Errors which require more correlation of information progress
deeper in the chain of processes before being sent back
to the user, with only those that properly penetrate the
router receiving full responses.

jay