[erlang-questions] linked in driver port performance

[Vinubalaji Gopal]

Hi Michael,
  I do use ERL_DRV_FLAG_USE_PORT_LOCKING in my driver entry.
Here is the structure of my program (pseudo code which mimics the original
code):

static int worker_call(ErlDrvData handle, unsigned int command, char *buf,
                   int len, char **rbuf, int rlen, unsigned int *flags) {
// decode the pid and ref using ei_decode_pid and ei_decode_ref

// decode other stuff

// get a worker from a pool
// *worker  = worker_pool->get();
// load the worker with data
 driver_async(port, NULL, do_work, worker, NULL);
}

static void do_work(void *worker_data) {
// get worker with worker data
// do the work
}

static void ready_async(ErlDrvData handle, ErlDrvThreadData worker_data) {
//get worker from the data
// output the results of the worker back to the
//give back the worker to the pool
worker_pool->release(worker)
}

The Erlang side: I had to make the gen server have the following structure
to accept calls in a parallel fashion - not sure if this is the most
efficient way - but got this idea from a program generated by dryverl.  It
still doesn't perform that well :(

handle_call({do_work, Param1, Param2}, {FromPid, FromTag}, State)
      when is_pid(FromPid), is_reference(FromTag) ->
   erlang:port_call(State#state.port, Type, {FromPid, FromTag, Param1,
Param2}),
    {noreply, State}.


// get the real reply asynchronously from the driver and send it back to the
real caller
handle_info({worker_reply, Port, EncOutputMainTerm, Output1, Output2},
State)
              when Port == State#state.port ->
   {FromPid, FromTag} = binary_to_term(EncOutputMainTerm),
    Reply = {Output1, Output2}
    FromPid ! {FromTag, Reply},
    {noreply, State}.

Thanks for the explanation on the linked in driver permanent issue.


On Tue, Jul 13, 2010 at 11:15 PM, Michael Truog <mjtruog@REDACTED> wrote:

> Ok, I didn't realize that you had found the same linked in driver
> permanent issue on trapexit.  I at least offered my explanation of what
> you seem to be experiencing.
>
> After looking at NIFs, they actually look like they are using port level
> locking by default (as described at
> http://erlang.org/doc/man/erl_nif.html by the "Threads and concurrency"
> section).  It would be much easier to create a NIF, but there is no
> interface for async threads through NIFs (though you can send a process
> a message with enif_send, which might help facilitate asynchronous
> behavior).  If you stick with a port driver, you would need to use
> ERL_DRV_FLAG_USE_PORT_LOCKING as previously mentioned.  Either way you
> want the C code to be quick, so you don't block the Erlang VM scheduler
> thread.
>
> - Michael
>
> On 07/13/2010 10:10 PM, Michael Truog wrote:
> > I think you are having problems with the driver becoming permanent when
> > you make the first asynchronous call (there is code that locks the port
> > driver so the dll/so can not be unloaded (so it can't be reloaded)).
> > The code is trying to protect the code from the possibility that a
> > thread finishes but the port driver has already been unloaded, which is
> > a valid concern (but could be handled in more complex ways).  The
> > problem has been mentioned before here
> >
> http://www.trapexit.org/forum/viewtopic.php?t=14295&sid=5de3c47e32136682080511d8d95d458e
> > .
> >
> > ERL_DRV_FLAG_USE_PORT_LOCKING is the flag to make sure different
> > scheduler threads can call the port driver instances concurrently (the
> > entry in the driver_flags struct member
> > http://www.erlang.org/doc/man/driver_entry.html#driver_flags) as
> > discussed here http://www.erlang.org/doc/man/erl_driver.html#smp_support
> > .  I assume NIFs use the port driver default of "driver level locking"
> > (driver_flags == 0, i.e., not ERL_DRV_FLAG_USE_PORT_LOCKING), but I am
> > not sure.
> >
> > - Michael
> >
> > On 07/13/2010 08:28 PM, Vinubalaji Gopal wrote:
> >
> >> Hi all,
> >>   I have been digging hard to write a port linked in driver which would
> >> perform better and most of my searches resulted in dead threads (like
> >>
> http://www.trapexit.org/forum/viewtopic.php?t=14295&sid=5de3c47e32136682080511d8d95d458e
> )
> >> and most of my experiments did not give me any good results . I tried to
> use
> >> async threads but I am not able to call these async threads
> asynchronously.
> >> If I try to make multiple erlang processes open the same port it just
> fails
> >> saying that the driver has become permanent. If I try to open a single
> port
> >> driver  - I have to do that in a single gen_server and it blocks when I
> >> call  port_command - so all the messages are queued and processed
> >> sequentially. Is there a way to write a linked in driver which will
> perform
> >> better and/or better links to help (I have read the erl_driver
> >> documentation, erl_ddll documentation numerous times).
> >>
> >>
> >>
> >>
> >>
> >
> > ________________________________________________________________
> > erlang-questions (at) erlang.org mailing list.
> > See http://www.erlang.org/faq.html
> > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
> >
> >
> >
>
>


-- 
Vinu

In a world without fences who needs Gates?