massive tcp servers

[Hal Snyder]

Shawn Pearce <spearce@REDACTED> writes:

> Dustin Sallings <dustin@REDACTED> wrote:
>> 
>> 	Does anyone have any experience with massively large tcp servers?  
>> 	I'm doing something like a chat server where there will be many, many 
>> connections simultaneously along with some mechanism for addressing 
>> those connections.  Does anyone have any idea what might be required to 
>> have something on the order of 10,000,000 concurrent connections in a 
>> cluster.  Obviously I want as small of a cluster as possible for 
>> hardware costs.  I also get the impression that large clusters may not 
>> be that easy to scale.
>
> http://www.sics.se/~joe/apachevsyaws.html
>
> Erlang will easily take 80,000 connections on some OSes and still
> keep a pretty good throughput.
>
> 10 million connections may take quite a few machines; I'd expect
> you would want to be hitting around 100k-200k TCP connections per
> physical computer, with about 10k-80k TCP connections per Erlang node.
> 100k/computer = 100 computers.

Jonathan Lemon's kqueue paper discusses of large numbers of TCP
connections, testing HTTP sessions with and without kernel event
delivery.

  ... The unmodified thttpd server runs out of cpu when the number of
  idle connections is around 600, while the modified server still has
  approximately 48% idle time with 10,000 idle connections.

http://www.cs.princeton.edu/courses/archive/fall03/cs518/papers/kqueue.pdf