[erlang-questions] Comet server broadcasting messages to 40k clients

[Linan Wang]

how about spawn process for each send_chunk?           ...
           TStart = now(),
           lists:foreach(fun(S) ->
               spawn(fun() -> send_chunk(S,
io_lib:format("(~b usecs) Hello world!<br>", [timer:now_diff(now(),
TStart)])) end),
                Sockets),

(sorry to bob, clicked the wrong button :))

On Wed, Jan 21, 2009 at 1:38 PM, Adriano Bonat <adrianob@REDACTED> wrote:

> Hi Bernard,
>
> On Wed, Jan 21, 2009 at 2:52 AM, Bernard Duggan <bernie@REDACTED> wrote:
>
> > My first suggestion would be to check that you're using kernel polling
> > (do you start erl with "+K true"?  You should see the string
> > "[kernel-poll:true]" in the version line).  If not, that's probably a
> > significant part of it, since for every packet that comes in the VM will
> > be iterating over all 40k sockets to figure out which one it came from
> > before the message even arrives at your code.
>
> Yes, I ran the tests using kernel polling.
>
> > Also, be aware that using the -- operator on a big list can be a very
> > expensive operation (there was a thread about it last week, in fact), so
> > using it to remove a single socket from a list that big could be a
> > significant cost.
>
> That's one of the Joe's advices on his book :)
> But the latency for the message is not caused by that point...
>
> > Finally, I'd suggest that, if you haven't already tried it, you spawn a
> > separate thread (process, in erlang parlance) to handle each connection
> > rather than trying to deal with them all in one thread.  While I don't
> > think this is going to improve your performance here, it makes your code
> > simpler and easier to follow.  Remember, erlang processes are not like C
> > threads - they're cheap, plentiful and carry a much lower switching
> > overhead.  This would involve a little more work for your case since you
> > want to broadcast to everyone based on a signal from anyone, but it is
> > more in keeping with "The Erlang Way(tm)".  (Actual followers of The
> > Erlang Way are of course free to correct my views...).
>
> Yes, that would go with the Erlang's way, but I wanted a way that
> works faster. Before I implemented my own http server, I was using
> mochiweb, and it uses a thread per connection and has the same
> performance problem as using a single thread holding several
> connections.
>
> Any other ideas are welcome, and thanks for you reply :)
>
> Regards.
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>



-- 
Best regards

Linan Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20090121/225c29b1/attachment.htm>