[erlang-questions] NIFs and hooking file descriptors to Erlang-manged poll

[Daniel Goertzen]

What is the measure of control that the VM has over drivers that it would
not have over native processes?  I've always interpreted native processes
as being simpler and more capable than drivers but with exactly the same
hazards (executing for too long, corrupting/crashing, etc).  I've worked
with NIFs but not drivers; perhaps there's some obvious aspect of drivers
that I've missed.

On Thu, Jan 29, 2015 at 2:41 PM, Björn-Egil Dahlberg <
wallentin.dahlberg@REDACTED> wrote:

> Yes I know. You are well versed in port crafting. =)
>
> I don't really like the idea of having third party libs directly involved
> in Erlangs I/O system, especially through callbacks we have no control
> over. I don't really like the idea of native processes either but that's
> beside the point.
> It's little different with drivers since we have some measure of control
> there.
>
> You pretty much spelled out how we envision NIF usage and reactiveness
> handling with thread pools, your own select and messaging erlang processes.
> That's the way to do it. We want to isolate it and mitigate problems of
> things we have no control over.
> It's not hard to write a thread-pool-select-message-dispatch-thingy, just
> some tedious boiler plate, but perhaps the thread pool handling could be
> done simpler or at least get a helping hand from some NIF framework.
> Though, I don't see such a framework being implemented by the OTP team any
> time soon either .. we have far more pressing stuff to deal with.
>
> // Björn-Egil
>
>
> 2015-01-29 20:32 GMT+01:00 Serge Aleynikov <serge@REDACTED>:
>
>> I understand your reasoning, though NIF (being the "most recent"
>> successor of C API) is a much cleaner API then the marshaling part of
>> drivers.  Having written a fair number of ports/drivers/NIFs I should say
>> that from coding perspective, IMHO, NIFs are the easiest/cleanest to use
>> when it comes to bringing existing C libraries with Erlang.  Consequently,
>> I would think that adding FD event awareness to that API (possibly prior to
>> having full-fledged native processes) would only make it more convenient to
>> use as it would cover more possible use cases.
>>
>> Serge
>>
>> On Thu, Jan 29, 2015 at 12:05 PM, Björn-Egil Dahlberg <egil@REDACTED>
>> wrote:
>>
>>>  On 2015-01-29 15:20, Serge Aleynikov wrote:
>>>
>>>  Is there a way to hook a file descriptor from within a NIF function to
>>> the select/epoll/poll loop managed by the emulator with a callback invoked
>>> in the "NIF-land" on activity detected on the file descriptor?
>>>
>>>  I am looking for similar functionality available to NIF functions that
>>> is available when writing drivers using driver_select() call (*).  More
>>> specifically, I have an eventfd file descriptor that I'd like to be
>>> notified about without allocating a separate OS thread to poll on that FD
>>> and return results by sending a message to a given Erlang Pid.
>>>
>>>
>>> I think you have discovered why we have drivers in Erlang. This is
>>> exactly the use case for drivers. Normally you don't do much in a driver.
>>> You react to an event and transfer the binary to erlang space and to what
>>> should be done in Erlang. Don't try to program Erlang in C (like
>>> erl_interface), do it in Erlang.
>>>
>>> And no, we don't have native processes (in which case drivers would be
>>> obsolete). And no, it's not an oversight in the NIF API, not really. You
>>> could see native processes as an extension to NIFs but it is something
>>> completely different.
>>>
>>> In some specific cases you could transfer a fd (socket) to Erlangs I/O
>>> system, i.e. select/poll. See gen_tcp:connect/3,4. I know of projects that
>>> open fd:s in NIFs which then get transferred to gen_tcp in this manner.
>>>
>>> // Björn-Egil
>>>
>>>
>>>  Thanks,
>>>
>>>  Serge
>>>
>>>  (*) http://www.erlang.org/doc/man/erl_driver.html#driver_select
>>>
>>>
>>> _______________________________________________
>>> erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions
>>>
>>>
>>>
>>> _______________________________________________
>>> erlang-questions mailing list
>>> erlang-questions@REDACTED
>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>
>>>
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20150131/497de71f/attachment.htm>