File descriptor passing, was Re: Advantages of a large number of threads....

[Shawn Pearce]

Cute trick Scott!

I just had to handroll a few of those input scanners that come for
free with the inets driver.  :-(  Its too bad I couldn't do the same
thing with my serial driver.  I doubt I'll see much ASN.1 on the serial
port, but then again...  never know!

I've always been leary of the fd passing system in UNIX, but if it works,
it works.  I think its been frowned upon over time, and may be something
that kernel developers would love to see die.  I know its been used
in the past for servers to push the fd off from a network server to the
actual work slaves.  I think Oracle used it at one point in time, and
may still use it actually.

Calls like inet:peername/1 may get very confused, neigh, crash something
(Erlang caller?) when given a UNIX domain socket.  So I'd strongly
suggest the caller never know you pulled this magic trick.  Otherwise,
so long as you steer clear of those calls, I can't see why it would not
work.  If the socket was a true TCP socket though, it should work just
fine.

I'd love to know how the performance of a single accepter/dispatcher
node and multiple work nodes on a multiple CPU machine compares to
say Apache.  Bind all of the nodes into a cluster and use process
monitoring to automatically migrate the accepter/dispatcher on a crash,
and away you go!  :)

Scott Lystig Fritchie <fritchie@REDACTED> wrote:
> Hi there.  I put my code where my mouth was.  I just checked in to the
> Jungerl a driver that can pass file descriptors to and from Erlang
> nodes running on the same machine.  It has been lightly tested under
> Linux & FreeBSD and appears to work.  Let me know if you have problems
> with it ... or fix it yourself!  :-)
> 
> I discovered that it's possible to create a UNIX domain socket with a
> custom driver, then extract its file descriptor, then use
> gen_tcp:fdopen/2 to create a socket that has more nifty properties,
> like all of the options supported by inet:setopts/2.  ... Well, I
> haven't tested that assertion very thoroughly, but it's a quite
> promising & lazy approach.  After all, it would be really nice to be
> able to use all those nifty features (like SUN RPC or ASN.1 packet
> type encoding) by using someone else's code (namely the inets driver)!

-- 
Shawn.