[erlang-questions] Dirty NIF - classifying as CPU or I/O bound

[Jesper Louis Andersen]

On Sun, Oct 14, 2018 at 5:29 PM Steve Vinoski <vinoski@REDACTED> wrote:

>
>
> On Sun, Oct 14, 2018 at 11:08 AM Roger Lipscombe <roger@REDACTED>
> wrote:
>
>>
>> "If you should classify CPU bound jobs as I/O bound jobs, dirty I/O
>> schedulers might starve ordinary schedulers."
>>
>
> According to git, Rickard Green wrote this, so I'd take it as advice you
> shouldn't ignore.
>
>
If we think about the two kinds of jobs:

CPU bound jobs will hog a CPU for their whole operation. Thus, we want them
to live on their own threads next to the scheduler threads, so they can't
stop the scheduler threads.

IO bound jobs tend to use the CPU very little but hog the IO resources
quite a lot (in particular the disk). Because of this, one might want to
run many of these threads, far more than there are CPU cores in the
machine, since they'll just block immediately and never use the CPU
resources.

However, if you accidentally or deliberately assign CPU bound jobs to the
IO threads, you might have few scheduler threads and many IO threads in the
system. In this case, the kernel might decide to give fair priority to all
the threads, in which case the scheduler threads will suffer because of
their fewer numbers.

Rickard and Sverker will know if there are differences in the strategies of
the two kinds of job types, but the above would happen even if they were
treated the same.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20181014/40b07d62/attachment.htm>