[erlang-questions] CPU affinity and enif_thread_create

[Jachym Holecek]

# Max Lapshin 2011-08-21:
> I've created libx264 encoder in nif and started new thread for it.
> libx264 was started with option threads 4 and it really created 4 threads.
> 
> Now I have problems. All created threads are bound to one CPU core and
> it is not enough. 100% busy and I have to drop frames.
> 
> I have installed linux kernel with BFS (brainfuck scheduler), but it
> didn't change anything.
>
> So I have called sched_setaffinity by hands and libx264 threads spread
> across cores.
> 
> And I have questions: does erlang VM makes some calls, that bind
> future created threads to current core?

Nothing explicit in ethr_thr_create() (this ends up being called from the NIF
API you're probably using) if I'm looking correctly, but it could perhaps be
accidental?

  * Your thread is created from a scheduler thread.
  * Scheduler thread tries to be bound to some specific CPU.
  * On Linux affinity is hereditary[*].
  * So your worker thread gets bound to the same CPU.

Of course, OTP people will know for sure.

BR,
	-- Jachym

[*] Linux's pthread_setaffinity_np(3) says:

    A new thread created by pthread_create(3) inherits a copy of its
    creator's CPU affinity mask.

The manpage further says this is implemented in terms sched_setaffinity(2)
which does mention inheritance for fork()'d processes, but doesn't mention
threads, so this hypothesis might be wrong -- it's sched_setaffinity()
that's being used by Erlang, not its also nonstandard yet slightly more
portable pthread cousing (for purely historical reasons I assume?).