[erlang-questions] What API should be used in port-drivers to guarantee thread-safety of driver_send_term and driver_output_term?

[Rickard Green]

I suspect that we put different meanings into the expression "detached 
thread". However, just so that there are no misunderstandings I want to 
point out that you cannot create detached threads, i.e. threads that 
cannot be joined, using the erl_driver api. This since unloading of a 
driver cannot be made safe if the driver doesn't join all of its threads 
before it is unloaded.

Regards,
Rickard

On 09/06/2012 09:13 AM, Zhemzhitsky Sergey wrote:
> Glad to see, that in case of detached threads driver_send_term is still safe to use.
>
> Thanks a bunch for the help.
>
> Best Regards,
> Sergey
>
>
> -----Original Message-----
> From: Rickard Green [mailto:rickard@REDACTED]
> Sent: Thursday, September 06, 2012 10:49 AM
> To: Zhemzhitsky Sergey
> Cc: Erlang-Questions Questions
> Subject: Re: [erlang-questions] What API should be used in port-drivers to guarantee thread-safety of driver_send_term and driver_output_term?
>
> No, that is safe to do.
>
> Regards,
> Rickard
>
> On 09/06/2012 08:25 AM, Zhemzhitsky Sergey wrote:
>> Hello Rickard,
>>
>> Thanks a lot for the idea with pipes.
>>
>>>> driver_send_term() is thread safe in the smp case, driver_output_term() is *not*.
>> Does it mean that it's dangerous to use driver_send_term() from an arbitrary detached thread and driver_output_term from one of the driver callbacks at the same time in the case when smp is enabled?
>>
>> Best Regards,
>> Sergey
>>
>> -----Original Message-----
>> From: erlang-questions-bounces@REDACTED
>> [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Rickard
>> Green
>> Sent: Thursday, September 06, 2012 12:32 AM
>> To: Erlang-Questions Questions
>> Subject: Re: [erlang-questions] What API should be used in port-drivers to guarantee thread-safety of driver_send_term and driver_output_term?
>>
>> As Lukas said, this is not something you can solve in your driver. It would require changes in the emulator.
>>
>> driver_send_term() can of course be made thread safe also in the
>> non-smp emulator. Initially we also planned on doing so, but after
>> looking closer at it we dropped that idea. This since it would pull in
>> too much stuff from the smp emulator. One way to do it could be to
>> replace "erl -smp disable" with "erl -smp enable +S1" :-)
>>
>> The way to send information from threads to Erlang processes in the non-smp case is something like the following using a thread safe queue and a pipe. Create a pipe and call driver_select(port, fd, ERL_DRV_READ, 1) to show your interest in the read end of the pipe. When a thread has enqueued an element on the queue it should also write a byte on the write end of the pipe (note that it is important that this is done *after* the enqueue operation). This will trigger a call to the ready_input(drv_data, fd) callback by the scheduler thread. In the ready_input() callback you read away the byte written on the pipe and then dequeue the element, create a message and send it to the process using driver_[send|output]_term(). In the windows case you need to use an event instead of a pipe.
>>
>> Also, note that even though driver_send_term() is thread safe in the smp case, driver_output_term() is *not*.
>>
>> Regards,
>> Rickard Green, Erlang/OTP, Ericsson AB
>>
>> On Sep 5, 2012, at 6:37 PM, Lukas Larsson wrote:
>>
>>> Hello,
>>>
>>> I do not think[1] this is a problem which you can solve in your
>>> driver. It is probably the internals of the Erlang VM which are not
>>> threadsafe, i.e. if there are no locks around message queues to
>>> processes in the non-smp emulator. So while sending data to a
>>> process, that same process could be reading data from the queue, and
>>> voila you get a segfault. So in order to fix this I think you would
>>> have to patch the emulator it self.
>>>
>>> Lukas
>>>
>>> [1]: Note that I haven't actually checked the relevant code, I have
>>> divined this from what I know of the emulator.
>>>
>>> On Wed, Sep 5, 2012 at 5:18 PM, Zhemzhitsky Sergey
>>> <Sergey_Zhemzhitsky@REDACTED> wrote:
>>>> Hi erlang gurus,
>>>>
>>>>
>>>>
>>>> We’re developing erlang port driver that will send different terms
>>>> to some processes from the separate threads using driver_send_term.
>>>>
>>>> Each native thread started in the port driver (by means of
>>>> erl_drv_thread_create) will be associated with only one erlang
>>>> process and will send messages only to it.
>>>>
>>>>
>>>>
>>>> As a rule the documentation for driver_send_term says that it’s not
>>>> thread-safe when smp is disabled, so we have tested the driver with
>>>> disabled smp support and found that from time to time it leads to segfaults.
>>>>
>>>>
>>>>
>>>> So, the question is what is the proper way to make calls to
>>>> driver_send_term (and driver_output_term) thread-safe?
>>>>
>>>> Will driver_pdl_lock, driver_pdl_unlock help?
>>>>
>>>> Will wrapping all the calls to driver_send_term (and
>>>> driver_output_term) with erl_drv_mutex_lock and erl_drv_mutex_unlock help?
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Best Regards,
>>>>
>>>> Sergey
>>>>
>>>>
>>>>
>>>> _______________________________________________________
>>>>
>>>>
>>>>
>>>> The information contained in this message may be privileged and conf
>>>> idential and protected from disclosure. If you are not the original
>>>> intended recipient, you are hereby notified that any review,
>>>> retransmission, dissemination, or other use of, or taking of any
>>>> action in reliance upon, this information is prohibited. If you have
>>>> received this communication in error, please notify the sender
>>>> immediately by replying to this message and delete it from your
>>>> computer. Thank you for your cooperation. Troika Dialog, Russia.
>>>>
>>>> If you need assistance please contact our Contact Center (+7495) 258
>>>> 0500 or go to www.troika.ru/eng/Contacts/system.wbp
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> erlang-questions mailing list
>>>> erlang-questions@REDACTED
>>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>>
>>> _______________________________________________
>>> erlang-questions mailing list
>>> erlang-questions@REDACTED
>>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>
>
> --
> Rickard Green, Erlang/OTP, Ericsson AB.
>


-- 
Rickard Green, Erlang/OTP, Ericsson AB.