[erlang-questions] : driver_entry stop() and driver_async() interaction

[Dave Smith]

One other note...I eventually wrote my own thread pool/queue for load
balancing purposes. async_* can cause the load to be very spiky if you
can not predict the amount of time a queued operation might take
(which is usually the point of using threads...). I think it would be
_great_ if the VM/driver interface provided a more traditional thread
pool/queue API in addition to the existing async functionality. I know
of several Erlang drivers that have had to write their own and it can
be a very challenging problem to get right. :)

D.

On Thu, Dec 18, 2008 at 7:36 AM, Dave Smith <dizzyd@REDACTED> wrote:
>>>>> Is it the responsibility of the code in the stop() callback to call
>>>>> driver_async_cancel() on each outstanding async work item, or will this
>>>>> be done automatically by the emulator before the call to stop()?
>
> Yes, it is up to the driver to cancel outstanding items. In wrestling
> with this recently, I found that the best approach was to cancel all
> the items and then queue up a sentinel callback that triggers a
> condition variable. I then wait for that cv to get signaled and know
> at that point that the driver is clear to shutdown. This approach
> seemed to work quite well on a 8 proc box with reasonable load --
> YMMV. :)
>
>>>>> If this is the responsibility of the code in stop(), is it guaranteed
>>>>> that no async work item will be executing or scheduled during the call
>>>>> to the stop() callback?
>>>>>
>>>>> If no guarantee is made, is holding the PDL necessary and sufficient to
>>>>> guarantee this?
>
> I researched this, and I believe that the answer is "no". Scheduling
> the async work item does increment the PDL ref count when the item is
> _inserted_ into the queue. Once the item is in the queue, all bets are
> off -- the only guarantee you have is that the PDL won't disappear
> (i.e. you can safely lock/unlock it). The PDL says nothing about
> whether or not the async work item is executing during stop(). It
> seems to be the responsibility of the driver author to sort out these
> intricate timing issues.
>
> erts/emulator/beam/erl_async.c is where all the code of interest resides.
>
> It's entirely possible that I have made gross errors in my reading of
> the emulator code and would happily receive instruction from one of
> the core VM team. :)
>
> D.
>