[erlang-questions] why is mmap so darn difficult

[Tim Watson]

Max,

On 12 Jun 2013, at 09:42, Max Lapshin wrote:
> I thought that video streaming, when you need to read about half of
> gigabyte per second from disk, repack it and write to network, is the
> good place for mmap.
> It was a mistake. If you get slow mmap disk read, than whole erlang
> sticks and you get PHP-like performance when whole thread is blocked
> for one client.
> 

Isn't that because the pread call you used was in a NIF, and therefore could block the scheulder(s) no? My thought's about the approach were initially somewhat different. The write operations were just file:write, whereas there would be no pread, but rather data would be delivered to the socket using sendfile/5, which would utilise the async thread pool if it's enabled.

> Perhaps in embedded mmap would be better when you want to save memory
> and have 10 requests per day, but on high load mmap doesn't help.

I'd like to handle writing hundreds of thousands of input events per second and reading at various different speeds, but supporting (reads of) up to tens of thousands per second.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20130612/5062fdf7/attachment.htm>