Any pending improvements to async thread pool?

[Scott Lystig Fritchie]

Hi there.  I've a question for the OTP team: are there any
improvements to the async thread pool that might be appearing in a
future Erlang release?

Last night I was bitten by a "feature" that I've known about for ages
but had forgotten about.(*)  If I have a driver D that utilizes the
async thread pool, and if that driver gives an async thread a task T
that takes many seconds to finish, then it's quite possible to block
task T' by another driver (also using the async thread pool) until
task T has finished.

As an example, say I've got a driver that executes the C library's
sleep(200) call using an async thread.(**)  Then, I try launching the
debugger (or "pman" or "appmon" or whatever).  The application won't
run for 200 seconds...

... because the "efile_drv" driver is also using the async thread
pool.  The task of operating on one of the beam files is assigned by
efile_drv to the same thread that's executing the sleep(200).  The
result is a lot of wasted time.

The current scheme for mapping tasks to a particular thread is, IMHO,
broken.  A short-term solution is to manage my own async worker
thread(s) rather than using the async thread pool already provided.

However, given that "efile_drv" can end up blocking *itself* by trying
to operate on slow media (CD-ROM, NFS, Flash-based storage, or even a
really busy local on-disk file system(***)), I think it ought to be
fixed.  If there are other idle threads available, they should be
usable.

-Scott

(*) I mentioned this in my Erlang Driver Toolkit paper: section 7.3,
last paragraph.

(**) That's the first driver using async threads I wrote.  :-)
However, any computation that takes a long time is sufficient.

(***) Back when I was running USENET News servers, I had file systems
so busy that a single open(), link, or unlink() system call sometimes
took 3/4 of a second.  Each.  I don't recommend workloads that high.