[erlang-questions] Non-blocking send in erlang: how to save memory when send one content to many people

[Rapsey]

On Thu, Dec 15, 2011 at 11:48 AM, Zabrane Mickael <zabrane3@REDACTED>wrote:

> > On thread initialization the pipes get created and read pipe is placed
> in epoll/kqueue. Write pipe is from erlang to NIF, and read pipe is for NIF
> to read that data. I have a simple struct:
> > typedef struct kqmsg
> > {
> >     char what;
> >     int fd;
> >     ErlNifPid pid;
> >     void* data;
> > }kqmsg;
> > So I just fill up this struct with whatever info is required and do:
> write(pipe_write,&msg,sizeof(struct kqmsg)
> >
> > As for sockets, I do not use prim_inet:getfd, sockets are completely
> separate from gen_tcp. The NIF thread keeps the socket FD until it reads
> the first buffer from it. Once this happens it creates a socket resource,
> then sends the binary and socket with enif_send to the Erlang process that
> is in charge of deciding what to do with it.
>
>
> Does this approach "really" increase performances of your server?
>

In the live streaming use case absolutely. HTTP is half-duplex. Once the
server receives all the headers, it does not need to listen on that socket
anymore. This means there is no need to keep the socket in the NIF listen
thread and all you are doing is periodic write's on the FD from the stream
buffer.  This is where the real optimization is. Maintaining one circular
buffer and just looping sockets over it and doing writes.
On my servers traffic goes through haproxy on port 80. Once there is a good
amount of users on the server, haproxy actually uses more CPU than my
streaming server.


Sergej
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20111215/9c2c43cb/attachment.htm>