[erlang-questions] : : driver_entry stop() and driver_async() interaction

[Raimo Niskanen]

On Thu, Dec 18, 2008 at 08:08:06AM -0700, Dave Smith wrote:
> One other note...I eventually wrote my own thread pool/queue for load
> balancing purposes. async_* can cause the load to be very spiky if you
> can not predict the amount of time a queued operation might take
> (which is usually the point of using threads...). I think it would be
> _great_ if the VM/driver interface provided a more traditional thread
> pool/queue API in addition to the existing async functionality. I know
> of several Erlang drivers that have had to write their own and it can
> be a very challenging problem to get right. :)

Aggreed. It has been on the Future Plans list forever.
Using async thread for e.g prime number calculations
will today block random file operations...

> 
> D.
> 
> On Thu, Dec 18, 2008 at 7:36 AM, Dave Smith <dizzyd@REDACTED> wrote:
> >>>>> Is it the responsibility of the code in the stop() callback to call
> >>>>> driver_async_cancel() on each outstanding async work item, or will this
> >>>>> be done automatically by the emulator before the call to stop()?
> >
> > Yes, it is up to the driver to cancel outstanding items. In wrestling
> > with this recently, I found that the best approach was to cancel all
> > the items and then queue up a sentinel callback that triggers a
> > condition variable. I then wait for that cv to get signaled and know
> > at that point that the driver is clear to shutdown. This approach
> > seemed to work quite well on a 8 proc box with reasonable load --
> > YMMV. :)
> >
> >>>>> If this is the responsibility of the code in stop(), is it guaranteed
> >>>>> that no async work item will be executing or scheduled during the call
> >>>>> to the stop() callback?
> >>>>>
> >>>>> If no guarantee is made, is holding the PDL necessary and sufficient to
> >>>>> guarantee this?
> >
> > I researched this, and I believe that the answer is "no". Scheduling
> > the async work item does increment the PDL ref count when the item is
> > _inserted_ into the queue. Once the item is in the queue, all bets are
> > off -- the only guarantee you have is that the PDL won't disappear
> > (i.e. you can safely lock/unlock it). The PDL says nothing about
> > whether or not the async work item is executing during stop(). It
> > seems to be the responsibility of the driver author to sort out these
> > intricate timing issues.
> >
> > erts/emulator/beam/erl_async.c is where all the code of interest resides.
> >
> > It's entirely possible that I have made gross errors in my reading of
> > the emulator code and would happily receive instruction from one of
> > the core VM team. :)
> >
> > D.
> >
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions

-- 

/ Raimo Niskanen, Erlang/OTP, Ericsson AB