[erlang-questions] driver_create_port overhead and driver_caller

QDev q3.development@REDACTED
Fri Jun 14 12:19:07 CEST 2013


Is there any advice from OTP team on best way to handle long running task in driver that needs to do ongoing communication with peer Erlang process? Can Erlang process calls outputv and use mutex to talk to async thread (or manual spawned thread) be best? Is locking in outpuv not risk to schedulers if outputv take too long, like with NIF calls? Is there other way - would it be better in outputv to put binary/ErlIOVec directly in driver queue with driver_enqv or driver_enq_bin but then port data lock has to be use to synchronise reads/writes to driver queue anyway. That means more contention if many concurrent threads are working no? So I was thinking better to use mutex per long running thread instead.

Also, is it considered necessary to pool worker thread manually? In previous releases, using async thread pool could have detrimental effect on I/O performance - is this still the case? Because if using async thread pool is fine now, maybe easier to do that instead of pooling threads manually, though problem of signalling is still there.

Thanks
QD

On 13 Jun 2013, at 17:06, QDev wrote:

> I have linked in driver which deals with embedded library, can make calls some in outputv, some using driver_async, but now have to support some other long running operations requiring thread affinity and (more importantly) need to receive multiple commands/actions from Erlang client code associated with long running transaction in library API. Currently considering to spawn (or pool) some new thread for these (long running tasks) and using ErlDrvMutex and simple queue/array data structure to communicate with it from outputv, but there is also affinity between Erlang callers process id and long running task thread (since caller originates with erlang:send/2 direct to port, which is accessed via ets table), because replies are sent via driver_send_term so need to also store client's process id in ErlDrvTermData and use driver_monitor_process to ensure that if client dies, work is rolled back. IIRC driver_caller not thread safe, so need to only use this in outputv is that still
>  correct?
> 
> Now, just getting used to driver API changes in R16 (haven't written driver since R13B) and have seen driver_create_port. This seems to create new port running driver code, so wonder if this is better approach for these long running tasks, and what are benefits/advantage of doing this?
> 
> Thanks
> QD
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions




More information about the erlang-questions mailing list