[erlang-questions] Non-blocking send in erlang: how to save memory when send one content to many people

[Jon Watte]

We regularly have a hundred thousand sockets open, and an Erlang process
per socket. We don't do high throughput I/O to each of those clients, but
Erlang processes are *not* operating system level threads.
Also, binaries on the heap are shared across processes, if they are bigger
than some limitation -- I think it's 64 bytes.

Thus, you COULD have some process that pre-loads files into binaries. Then,
when a  socket connects and asks for data, you ask for that binary from
that process, and then write that binary to the socket. One Erlang process
per socket.
This would be a super simple implementation and probably very robust. I
don't know if it will scale to the I/O throughput that you request, but it
certainly would have a chance to do so.

Sincerely,

jw


--
Americans might object: there is no way we would sacrifice our living
standards for the benefit of people in the rest of the world. Nevertheless,
whether we get there willingly or not, we shall soon have lower consumption
rates, because our present rates are unsustainable.



On Wed, Dec 14, 2011 at 3:35 PM, Max Lapshin <max.lapshin@REDACTED> wrote:

> On Thu, Dec 15, 2011 at 2:23 AM, Lukas Larsson
> <lukas@REDACTED> wrote:
> > You might be interested in the new (R15B) file:sendfile function, if
> having
> > the data on disc is an option.
> >
>
> I think, that something like vmsplice may be an option. I see in
> sources of file, that prim_inet:getfd is no here (yes, I know that it
> is undocumented =)
> and that is why I may use some weird tricks with optimizing.
>
> sendfile is a bad option because:
> 1) I need to serve from memory
> 2) I need to serve several thousands of connections. Threads are not
> the way to go.
>
> However, I think that I will need to patch efile_drv for using
> something like vmsplice
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20111214/ddd71412/attachment.htm>