[erlang-questions] NIFs and hooking file descriptors to Erlang-manged poll

[Björn-Egil Dahlberg]

Yes I know. You are well versed in port crafting. =)

I don't really like the idea of having third party libs directly involved
in Erlangs I/O system, especially through callbacks we have no control
over. I don't really like the idea of native processes either but that's
beside the point.
It's little different with drivers since we have some measure of control
there.

You pretty much spelled out how we envision NIF usage and reactiveness
handling with thread pools, your own select and messaging erlang processes.
That's the way to do it. We want to isolate it and mitigate problems of
things we have no control over.
It's not hard to write a thread-pool-select-message-dispatch-thingy, just
some tedious boiler plate, but perhaps the thread pool handling could be
done simpler or at least get a helping hand from some NIF framework.
Though, I don't see such a framework being implemented by the OTP team any
time soon either .. we have far more pressing stuff to deal with.

// Björn-Egil


2015-01-29 20:32 GMT+01:00 Serge Aleynikov <serge@REDACTED>:

> I understand your reasoning, though NIF (being the "most recent" successor
> of C API) is a much cleaner API then the marshaling part of drivers.
> Having written a fair number of ports/drivers/NIFs I should say that from
> coding perspective, IMHO, NIFs are the easiest/cleanest to use when it
> comes to bringing existing C libraries with Erlang.  Consequently, I would
> think that adding FD event awareness to that API (possibly prior to having
> full-fledged native processes) would only make it more convenient to use as
> it would cover more possible use cases.
>
> Serge
>
> On Thu, Jan 29, 2015 at 12:05 PM, Björn-Egil Dahlberg <egil@REDACTED>
> wrote:
>
>>  On 2015-01-29 15:20, Serge Aleynikov wrote:
>>
>>  Is there a way to hook a file descriptor from within a NIF function to
>> the select/epoll/poll loop managed by the emulator with a callback invoked
>> in the "NIF-land" on activity detected on the file descriptor?
>>
>>  I am looking for similar functionality available to NIF functions that
>> is available when writing drivers using driver_select() call (*).  More
>> specifically, I have an eventfd file descriptor that I'd like to be
>> notified about without allocating a separate OS thread to poll on that FD
>> and return results by sending a message to a given Erlang Pid.
>>
>>
>> I think you have discovered why we have drivers in Erlang. This is
>> exactly the use case for drivers. Normally you don't do much in a driver.
>> You react to an event and transfer the binary to erlang space and to what
>> should be done in Erlang. Don't try to program Erlang in C (like
>> erl_interface), do it in Erlang.
>>
>> And no, we don't have native processes (in which case drivers would be
>> obsolete). And no, it's not an oversight in the NIF API, not really. You
>> could see native processes as an extension to NIFs but it is something
>> completely different.
>>
>> In some specific cases you could transfer a fd (socket) to Erlangs I/O
>> system, i.e. select/poll. See gen_tcp:connect/3,4. I know of projects that
>> open fd:s in NIFs which then get transferred to gen_tcp in this manner.
>>
>> // Björn-Egil
>>
>>
>>  Thanks,
>>
>>  Serge
>>
>>  (*) http://www.erlang.org/doc/man/erl_driver.html#driver_select
>>
>>
>> _______________________________________________
>> erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions
>>
>>
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20150129/08ea2afd/attachment.htm>