[erlang-questions] 20k messages in 4s but want to go faster!

[Rapsey]

Well I have a similar problem with my streaming server. I suspect the main
issue with using gen_tcp is that every gen_tcp:send call will involve a
memory allocation and memcpy.
My server needs to be able to fill up at least a gigabyte connection and
this involves a very large number of gen_tcp:send calls. The only way I
could achieve that number is by writing my own driver for data output.
This means bypassing gen_tcp completely and writing the socket handling
stuff by hand. Basically whenever I need to send the data, I loop through
the array of sockets (and other information) and send from one single buffer
(the sockets are non-blocking).
It works great and is fast as hell. I suspect it could fill up at least
2-4gigabits.


Sergej

On Fri, Jul 10, 2009 at 8:24 PM, Joel Reymont <joelr1@REDACTED> wrote:

> I can't seem to send 20k messages in much less than 4 seconds and would
> appreciate any suggestions on how to improve my broadcasting speed.
>
> The messages are binary, 69 bytes, including the trailing 0 (Flash).
> There's no size prefix. Flow control is {active, once}, e.g.
>
>    Opts = [binary,
>            {packet, 0},
>            {reuseaddr, true},
>            {backlog, 1024},
>            {active, false}],
>
> The publishing routing looks like this, where Send is a closure with
> gen_tcp:send/2 and a captured socket.
>
>    F = fun(_, {_, Send}, _) -> Send(Msg) end,
>    F1 = fun() -> dict:fold(F, ignore, State#state.subscribers) end,
>    spawn_link(F1),
>
> I'm running this on small EC2 instances and the actual fold takes < 200ms.
> I'm hesitant to attribute the difference to the TCP stack.
>
> I also tried this variant but only shaved 100-200ms from the 4s.
>
>    F = fun(_, {_, Send}, _) -> spawn(fun() -> Send(Msg) end) end,
>    dict:fold(F, ignore, State#state.subscribers),
>
> I tried distributing my test harness over 4 EC2 instances but this saved me
> a second or so, for the 4 second total I'm trying to beat now.
>
> What else can I do?
>
>        Thanks, Joel
>
> ---
> Mac hacker with a performance bent
> http://www.linkedin.com/in/joelreymont
>
>
> ________________________________________________________________
> erlang-questions mailing list. See http://www.erlang.org/faq.html
> erlang-questions (at) erlang.org
>
>