[erlang-questions] Support for polling HDD in linux 4.4

[Lukas Larsson]

Hello,

On Sun, Apr 24, 2016 at 9:12 AM, Max Lapshin <max.lapshin@REDACTED> wrote:

> There is a news on LWN: http://lwn.net/Articles/663879/
> and Ubuntu 16.04 with kernel that seems to support it.
>
> We in erlyvideo have to maintain our own "thread pool" inside erlang in
> top of erlang thread pool because it is very important to have different
> pools for different devices.
>

May I ask why you need this? Is it because you don't want work on different
devices that accidentally are hashed to the same thread to slow down each
other?


> All this is very complicated and could be more elegant if disk had async
> API.
>

One of the main reasons that the async file threads exists is because the
VM does not know if it is reading from an NFS or not. NFSs are notoriously
unreliable and setting fds to non-blocking when writing/reading to it is
not a guarantee that the write/read will be non-blocking, at least it
wasn't when I last checked a couple of years ago.


> Have anyone looked at this polling API?  Maybe it is time to implement it
> in erlang?
>

Our current idea is to move the entire file driver to use dirty I/O
schedulers. This will get rid of the hashing problem of the current thread
pool as the dirty schedulers are load balanced. This is still quite a way
off, as we first have to work out some of the oddities of dirty schedulers
and also add support for ports to run on them.

We may also change our minds and do it some other way if that seems better
at that time, for instance the file polling you mention.

Lukas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20160424/b12b5c93/attachment.htm>