[erlang-questions] Dirty NIF - classifying as CPU or I/O bound

[Roger Lipscombe]

On 14 October 2018 at 15:46, Jesper Louis Andersen <
jesper.louis.andersen@REDACTED> wrote:

> On Sun, Oct 14, 2018 at 2:42 PM Roger Lipscombe <roger@REDACTED>
> wrote:
>
>> If I *don't know* whether the job is going to be CPU bound or I/O bound
>> (it executes arbitrary code provided by a third party), am I safest to just
>> classify the dirty job as CPU-bound? Or is this warning hinting at a
>> disaster of biblical proportions[1] if I even *think* about fudging the
>> classification?
>>
>>
> Either classification risks being wrong, so you can't really do any of
> them safely. The two classifications exist because IO resources and CPU
> resources tend to orthogonally consumed: If we have many IO bound jobs, we
> can still run CPU bound jobs and vice versa. But if you don't know what
> kind of job you are looking at a priori, you have no way to classify it
> correctly.
>

Thanks Jesper, I guess my question is rooted in this statement in the docs:

"If you should classify CPU bound jobs as I/O bound jobs, dirty I/O
schedulers might starve ordinary schedulers."

This, to me, implies that I should probably classify unknown jobs as CPU
bound, rather than I/O bound, because the documentation only mentions bad
things happening one way round.

Based on my limited knowledge of how dirty schedulers works, my instinct
tells me that classifying jobs as CPU bound when they're I/O bound will
probably just be less efficient, whereas classifying jobs as I/O bound when
they're CPU bound will result in trying to run too many jobs at once. But
I'm just guessing.

Cheers,
Roger.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20181014/b4439593/attachment.htm>