[erlang-questions] : : : : : driver_entry stop() and driver_async() interaction

[Chris Newcombe]

> Sounds a bit hackish, but why not; drivers tend to be hackish.

Thanks!


> An even hackier hack would be to use the I/O queue for this,
> and that even works today.

I've never used the I/O queue so I don't know much about it.  IIRC
(from looking at the docs a couple of years ago), I got the impression
that it is an 'earlier' driver API.  But maybe I just haven't
encountered the right use-case.

EDTK uses binaries via outputv and driver_send_term, to minimize
copying.  As the BerkeleyDB driver is often used to transfer a _lot_
of data, this is important.


> Was not this (later in the same mail from me) enough?
> :
>> Raimo wrote:
>> > While we are at it we plan to introduce a driver flag
>> > indicating that the driver wants to be called on
>> > flush() if there are pending async jobs (as well as
>> > today if the IO queue is not empty).

Ah, very sorry -- yes I somehow missed this part of your e-mail.

So all I need (for EDTK) would be the API increment and decrement the
'async operation count'.
(I'd really prefer to avoid (mis)using the I/O queue for that)

many thanks,

Chris



On Fri, Jan 9, 2009 at 12:42 AM, Raimo Niskanen
<raimo+erlang-questions@REDACTED> wrote:
> On Fri, Dec 19, 2008 at 07:08:11AM -0800, Chris Newcombe wrote:
>> > We plan while fixing the bug with async_free() mentioned below
>> > to introduce a reference counter of async jobs so it will be
>> > guaranteed that stop() is not called until all async jobs are done.
>>
>> Excellent.  But please could you provide APIs to increment and
>> decrement that reference count, so that drivers that implement their
>> own private thread-pools can achieve the same behaviour?
>
> Sounds a bit hackish, but why not; drivers tend to be hackish.
> An even hackier hack would be to use the I/O queue for this,
> and that even works today.
>
>>
>> Also, I hate to cause feature creep, but if you do implement the above
>> reference count/delayed 'stop', please could you also add an optional
>> "prepare to stop" callback, which the VM calls when the Erlang code
>> closes the port (e.g. the port owner process dies, or something
>> explicitly closes the port)?  Some drivers often have fairly
>> long-running operations (seconds/minutes) running in their own
>> threads, and would like timely notification that they should force
>> such jobs to terminate, to allow "stop" to be called as soon as
>> possible (the berkeley_db driver has to do this).  i.e. It's not
>> always the case that running async jobs should be left to complete in
>> their own sweet time.  (Note that this is different from 'cancelling'
>> pending-but-not-yet-started jobs.)
>
> Was not this (later in the same mail from me) enough?
> :
>> Raimo wrote:
>> > While we are at it we plan to introduce a driver flag
>> > indicating that the driver wants to be called on
>> > flush() if there are pending async jobs (as well as
>> > today if the IO queue is not empty).
> :
> Today the flush() callback is called when the port gets killed e.g
> gets an exit signal from its linked owner when it dies, if there
> is data in the I/O queue. Then the stop() callback is called when the
> I/O queue is emptied.
>
>>
>> Rationale:  It does actually matter a lot to some drivers that "stop"
> :
>
> --
>
> / Raimo Niskanen, Erlang/OTP, Ericsson AB
>