20k messages in 4s but want to go faster!

[Joel Reymont]

I can't seem to send 20k messages in much less than 4 seconds and  
would appreciate any suggestions on how to improve my broadcasting  
speed.

The messages are binary, 69 bytes, including the trailing 0 (Flash).  
There's no size prefix. Flow control is {active, once}, e.g.

     Opts = [binary,
             {packet, 0},
             {reuseaddr, true},
             {backlog, 1024},
             {active, false}],

The publishing routing looks like this, where Send is a closure with  
gen_tcp:send/2 and a captured socket.

     F = fun(_, {_, Send}, _) -> Send(Msg) end,
     F1 = fun() -> dict:fold(F, ignore, State#state.subscribers) end,
     spawn_link(F1),

I'm running this on small EC2 instances and the actual fold takes <  
200ms. I'm hesitant to attribute the difference to the TCP stack.

I also tried this variant but only shaved 100-200ms from the 4s.

     F = fun(_, {_, Send}, _) -> spawn(fun() -> Send(Msg) end) end,
     dict:fold(F, ignore, State#state.subscribers),

I tried distributing my test harness over 4 EC2 instances but this  
saved me a second or so, for the 4 second total I'm trying to beat now.

What else can I do?

	Thanks, Joel

---
Mac hacker with a performance bent
http://www.linkedin.com/in/joelreymont