[erlang-questions] Dirty NIF - classifying as CPU or I/O bound

[Rickard Green]

On Sun, Oct 14, 2018 at 5:37 PM Jesper Louis Andersen <
jesper.louis.andersen@REDACTED> wrote:

> On Sun, Oct 14, 2018 at 5:29 PM Steve Vinoski <vinoski@REDACTED> wrote:
>
>>
>>
>> On Sun, Oct 14, 2018 at 11:08 AM Roger Lipscombe <roger@REDACTED>
>> wrote:
>>
>>>
>>> "If you should classify CPU bound jobs as I/O bound jobs, dirty I/O
>>> schedulers might starve ordinary schedulers."
>>>
>>
>> According to git, Rickard Green wrote this, so I'd take it as advice you
>> shouldn't ignore.
>>
>>
> If we think about the two kinds of jobs:
>
> CPU bound jobs will hog a CPU for their whole operation. Thus, we want
> them to live on their own threads next to the scheduler threads, so they
> can't stop the scheduler threads.
>
> IO bound jobs tend to use the CPU very little but hog the IO resources
> quite a lot (in particular the disk). Because of this, one might want to
> run many of these threads, far more than there are CPU cores in the
> machine, since they'll just block immediately and never use the CPU
> resources.
>
> However, if you accidentally or deliberately assign CPU bound jobs to the
> IO threads, you might have few scheduler threads and many IO threads in the
> system. In this case, the kernel might decide to give fair priority to all
> the threads, in which case the scheduler threads will suffer because of
> their fewer numbers.
>
> Rickard and Sverker will know if there are differences in the strategies
> of the two kinds of job types, but the above would happen even if they were
> treated the same.
>
>
>
The above is true. That is, if you are not sure of the jobs characteristics
you want to classify it as CPU bound. If you are wrong, and it is actually
IO bound and blocks for a very long time it might cause problems for dirty
CPU bound jobs, but those problems are typically not as severe as the
problems experienced the other way around.

Regards,
Rickard
-- 
Rickard Green, Erlang/OTP, Ericsson AB
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20181015/a7245d86/attachment.htm>