[erlang-questions] What API should be used in port-drivers to guarantee thread-safety of driver_send_term and driver_output_term?

[Zhemzhitsky Sergey]

Glad to see, that in case of detached threads driver_send_term is still safe to use.

Thanks a bunch for the help.

Best Regards,
Sergey


-----Original Message-----
From: Rickard Green [mailto:rickard@REDACTED] 
Sent: Thursday, September 06, 2012 10:49 AM
To: Zhemzhitsky Sergey
Cc: Erlang-Questions Questions
Subject: Re: [erlang-questions] What API should be used in port-drivers to guarantee thread-safety of driver_send_term and driver_output_term?

No, that is safe to do.

Regards,
Rickard

On 09/06/2012 08:25 AM, Zhemzhitsky Sergey wrote:
> Hello Rickard,
>
> Thanks a lot for the idea with pipes.
>
>>> driver_send_term() is thread safe in the smp case, driver_output_term() is *not*.
> Does it mean that it's dangerous to use driver_send_term() from an arbitrary detached thread and driver_output_term from one of the driver callbacks at the same time in the case when smp is enabled?
>
> Best Regards,
> Sergey
>
> -----Original Message-----
> From: erlang-questions-bounces@REDACTED 
> [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Rickard 
> Green
> Sent: Thursday, September 06, 2012 12:32 AM
> To: Erlang-Questions Questions
> Subject: Re: [erlang-questions] What API should be used in port-drivers to guarantee thread-safety of driver_send_term and driver_output_term?
>
> As Lukas said, this is not something you can solve in your driver. It would require changes in the emulator.
>
> driver_send_term() can of course be made thread safe also in the 
> non-smp emulator. Initially we also planned on doing so, but after 
> looking closer at it we dropped that idea. This since it would pull in 
> too much stuff from the smp emulator. One way to do it could be to 
> replace "erl -smp disable" with "erl -smp enable +S1" :-)
>
> The way to send information from threads to Erlang processes in the non-smp case is something like the following using a thread safe queue and a pipe. Create a pipe and call driver_select(port, fd, ERL_DRV_READ, 1) to show your interest in the read end of the pipe. When a thread has enqueued an element on the queue it should also write a byte on the write end of the pipe (note that it is important that this is done *after* the enqueue operation). This will trigger a call to the ready_input(drv_data, fd) callback by the scheduler thread. In the ready_input() callback you read away the byte written on the pipe and then dequeue the element, create a message and send it to the process using driver_[send|output]_term(). In the windows case you need to use an event instead of a pipe.
>
> Also, note that even though driver_send_term() is thread safe in the smp case, driver_output_term() is *not*.
>
> Regards,
> Rickard Green, Erlang/OTP, Ericsson AB
>
> On Sep 5, 2012, at 6:37 PM, Lukas Larsson wrote:
>
>> Hello,
>>
>> I do not think[1] this is a problem which you can solve in your 
>> driver. It is probably the internals of the Erlang VM which are not 
>> threadsafe, i.e. if there are no locks around message queues to 
>> processes in the non-smp emulator. So while sending data to a 
>> process, that same process could be reading data from the queue, and 
>> voila you get a segfault. So in order to fix this I think you would 
>> have to patch the emulator it self.
>>
>> Lukas
>>
>> [1]: Note that I haven't actually checked the relevant code, I have 
>> divined this from what I know of the emulator.
>>
>> On Wed, Sep 5, 2012 at 5:18 PM, Zhemzhitsky Sergey 
>> <Sergey_Zhemzhitsky@REDACTED> wrote:
>>> Hi erlang gurus,
>>>
>>>
>>>
>>> We’re developing erlang port driver that will send different terms 
>>> to some processes from the separate threads using driver_send_term.
>>>
>>> Each native thread started in the port driver (by means of
>>> erl_drv_thread_create) will be associated with only one erlang 
>>> process and will send messages only to it.
>>>
>>>
>>>
>>> As a rule the documentation for driver_send_term says that it’s not 
>>> thread-safe when smp is disabled, so we have tested the driver with 
>>> disabled smp support and found that from time to time it leads to segfaults.
>>>
>>>
>>>
>>> So, the question is what is the proper way to make calls to 
>>> driver_send_term (and driver_output_term) thread-safe?
>>>
>>> Will driver_pdl_lock, driver_pdl_unlock help?
>>>
>>> Will wrapping all the calls to driver_send_term (and
>>> driver_output_term) with erl_drv_mutex_lock and erl_drv_mutex_unlock help?
>>>
>>>
>>>
>>>
>>>
>>> Best Regards,
>>>
>>> Sergey
>>>
>>>
>>>
>>> _______________________________________________________
>>>
>>>
>>>
>>> The information contained in this message may be privileged and conf 
>>> idential and protected from disclosure. If you are not the original 
>>> intended recipient, you are hereby notified that any review, 
>>> retransmission, dissemination, or other use of, or taking of any 
>>> action in reliance upon, this information is prohibited. If you have 
>>> received this communication in error, please notify the sender 
>>> immediately by replying to this message and delete it from your 
>>> computer. Thank you for your cooperation. Troika Dialog, Russia.
>>>
>>> If you need assistance please contact our Contact Center (+7495) 258
>>> 0500 or go to www.troika.ru/eng/Contacts/system.wbp
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> erlang-questions mailing list
>>> erlang-questions@REDACTED
>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>


--
Rickard Green, Erlang/OTP, Ericsson AB.