Erlang to driver communication: why not a non-serialized option?

[Scott Lystig Fritchie]

Well, there's an easy answer to that: to maintain the same interface
(on the Erlang side) when communicating with a C driver or a
driver in an external process (communicating via a pipe).  However, in
the last couple of major releases, the fiction that the two types of
drivers are the same has eroded.

(Aside: I use "C driver" to mean a driver that is either
 linked into the VM at compile time or a driver dynamically loaded by
 erl_ddll.  I'll call the original kind of driver, in an external UNIX
 process, a "pipe driver".  Are there better words to name them?)

The first erosion was the addition of "outputv" to the ErlDrvEntry
structure used by C drivers.  Code on the Erlang side still
sent an I/O list to the port, but the driver side can access the data
via and ErlIOVec structure.  That way is more efficient because the
data doesn't need to be fully copied into a single contiguous buffer
before the driver can access it.  (I've noticed that small amounts of
data will be copied into a single buffer, though.)

The second (and much larger) erosion was the introduction of
driver_output_term() and driver_send_term().  These functions break
the serialization barrier by making it possible to send an Erlang term
to an Erlang process instead of a mere sequence of bytes.(*)

The third erosion is something that snuck in with R8 (?),
erlang:port_call/3.  It allows both sending and receiving arbitrary
/erlang terms to and from the driver.  Cool! ... except that it, too,
maintains the "C drivers and pipe drivers are the same" fiction.

To make a long quesion short, what reasons are there for avoiding
giving C drivers direct access to the C structure that represents the
term?  I'm interested for a couple of reasons:

1. Efficiency.  It seems like every Erlang project I've worked on
involves moving lots of data through drivers: TCP, UDP, then UNIX
domain sockets and shared memory, and now Berkeley DB.  All that extra
byte copying is damn annoying.

2. Ease of driver implementation.  Accessing a terms' C data structure
may not seem "easy" in the general case (like dealing with an
arbitrary length & depth I/O list), but you can force the Erlang side
to restrict what kinds of terms it passes to the driver.  And dealing
with those C structures may very well be easier than the *tedious*
work of serializing the arguments to a driver call on the Erlang side
and unserializing them in the driver side ... and having to do the
same thing again for the reply.  All that byte copying is damn
annoying.

Perhaps I should've picked a project with an API that has fewer than
40+ functions I wanted to expose to Erlang.  Perhaps I picked the
perfect driver to expose this problem, to whine if anyone has a
solution already, and if not, storm off and go write a solution
myself.  :-)

-Scott

(*) Despite this "advance" in communicating in the driver -> Erlang
direction, there are still vestiges of serial communication.  For
example, the prim_inet.erl module uses an interesting trick to make
"receive" filtering easier:

	  1. Send a serialized request to the tcp_inet driver asking
	  it to perform an accept(2) on an existing socket.

	  2. Erlang gets a small serialized message back from the
	  driver that acknowledges that the request was kosher.  The
	  message includes an encoded 16 bit integer, call it Ref,
	  that will be included in the "real" response to the accept
	  call.

	  3. Erlang then simply has to wait for the real answer:

            receive 
                {inet_async, S, Ref, Status} ->
                    case Status of
                        ok -> accept_opts(L, S);
                        Error -> close(S), Error
                    end
            end;

Ref isn't an Erlang reference but just an integer, but it's doing the
same thing that references are chiefly used for: tagging messages for
selective receipt.