[erlang-questions] Support for polling HDD in linux 4.4

[Benoit Chesneau]

Related but not totally I was wondering if it couldn't be possible to have
custom dirty schedulers so we could start a vm with a custom thread pools
to separate usages. For example we could have custom dirty schedulers to
handle different loads and still using the dirty nif API.

Is there any paper somewhere describing the internals of the dirty nif
schedulers?

- benoît



On Sun, Apr 24, 2016 at 10:59 AM Lukas Larsson <garazdawi@REDACTED> wrote:

> Hello,
>
> On Sun, Apr 24, 2016 at 9:12 AM, Max Lapshin <max.lapshin@REDACTED>
> wrote:
>
>> There is a news on LWN: http://lwn.net/Articles/663879/
>> and Ubuntu 16.04 with kernel that seems to support it.
>>
>> We in erlyvideo have to maintain our own "thread pool" inside erlang in
>> top of erlang thread pool because it is very important to have different
>> pools for different devices.
>>
>
> May I ask why you need this? Is it because you don't want work on
> different devices that accidentally are hashed to the same thread to slow
> down each other?
>
>
>> All this is very complicated and could be more elegant if disk had async
>> API.
>>
>
> One of the main reasons that the async file threads exists is because the
> VM does not know if it is reading from an NFS or not. NFSs are notoriously
> unreliable and setting fds to non-blocking when writing/reading to it is
> not a guarantee that the write/read will be non-blocking, at least it
> wasn't when I last checked a couple of years ago.
>
>
>> Have anyone looked at this polling API?  Maybe it is time to implement it
>> in erlang?
>>
>
> Our current idea is to move the entire file driver to use dirty I/O
> schedulers. This will get rid of the hashing problem of the current thread
> pool as the dirty schedulers are load balanced. This is still quite a way
> off, as we first have to work out some of the oddities of dirty schedulers
> and also add support for ports to run on them.
>
> We may also change our minds and do it some other way if that seems better
> at that time, for instance the file polling you mention.
>
> Lukas
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20160424/4315cfc6/attachment.htm>