massive tcp servers

[Joe Armstrong]

> If you wrote your own TCP (or simpler custom protocol) in userspace
> then maybe you could pull it off with just one box. You could keep the
> TCP control structures in a database on disk and map the active ones
> into a large-but-bounded-size pool in memory.
> 
> I have only limited experience with hacking TCP/IP, but my impression
> is that writing your own TCP is probably no harder than e.g. writing a
> good HTTP/1.1 implementation. I recommend the first two volumes of
> Douglas Comer's networking series, in turn recommended to me by Tobbe
> from when he wrote a TCP in Erlang.

  I suspect  it might  be pretty  easy - Adam  Dunkels wrote  a TCP/IP
"thingy" in  PHP - not a  full TCP but  just enough to answer  an HTTP
request see

  http://www.sics.se/~adam/phpstack/

  I talked to Adam he said it  took him "3 hours" - including the time
to learn  PHP - now Adam  has admittedly implemented TCP  many times -
but staring at his code might provide some inspiration.

  If you  wrote "enough" TCP to  just handle a HTTP  request you might
come up with a pretty high performance, highly concurrent web server -
who knows.

  IMHO the  tricky bit  is making  a tunnel so  that a  regular Erlang
program can see the raw IP datagrams - the rest is just plain coding.

  Luke sees to have some code that does this bit - (am I right Luke).

  Cheers

/Joe


> 
> On Linux I think the most practical option would be to use a 'tun'
> interface (linux/Documentation/networking/tuntap.txt), which allows
> you to write a user-space network interface that operates at IP-level
> (rather than ethernet-level). That way you can use Linux's IPv4
> implementation and only worry about TCP yourself.
> 
> -Luke
>