[erlang-questions] RFC: Erlang FFI
Fri Aug 10 12:30:49 CEST 2007
Scott Lystig Fritchie <fritchie@REDACTED> wrote:
> Most of the Erlang<->C/C++ efforts have been SWIG-like efforts to
> present as smooth an interface to the non-C side as possible. In
> Erlang, IMHO(*), such integration has been even more important. At
> least half of Erlang's advantage is its ability to survive faults(*),
> and creating arbitrary interfaces into unknown shared libraries is a
> good way to invalidate much/all of those survival mechanisms.
> Such an interface to libffi makes it possible to create Erlang
> interfaces to *any* shared library, without recompiling or relinking
> or other steps. To create the same "smooth" interface, the glue ends
> up being on the Erlang side of the divide. That philosophy is
> opposite from most FFI efforts in the Erlang world.
> There are 2 main FFI camps, it seems to me(*):
> 1. I don't care about fault tolerance, I *must* access to function
> F in library L, and to hell with the consequences of buggy
> 2. I really care about fault tolerance ... because if I didn't, I
> wouldn't bother with Erlang at all.
I would like to point out something that I forgot to emphasize in my
previous email: the expected uses of the FFI BIFs I'm proposing.
Just like ddll:load/3, erlang:port_control/3 and friends, the FFI BIFs
(ffi:call/3, ddll:load_library/3, ffi:call/2, ffi:ecall/3) are *not*
intended to be used by "standard" Erlang developers. They are
actually targeted at two kinds of, let's say, Erlang power developers:
1. library bindings developers;
2. developers that need to interface Erlang code with
C/Fortran/Assembly/whatever (e.g. for performance reasons, thus
making even the call speed a relevant factor).
This is directly related to your fault-tolerance and stability
worries. The FFI BIFs could be used to crash the Erlang VM more or
less like a linked-in driver could . But unaware developers that
just want to use libfoo from Erlang are usually *not* directly exposed
to the gore details of Erlang drivers: they don't see
erlang:port_control/3 (or ffi:call/3), but a documented Erlang libfoo
API that takes care of validating the parameters and performing the
underlying machinery. If a developer decides to bypass this API layer
and access libfoo by fiddling with erlang:port_control/3 (or
ffi:call/3), then he/she is definitely on his/her own.
The main difference between linked-in drivers and FFI BIFs, IMHO, is
that ffi:call/3 and friends make it a lot easier to develop library
bindings and to interface external code to Erlang, both for
quick'n'dirty hacks and well-refined products. The FFI approach is
also more powerful, expecially with the proposed ffi:ecall/3
I agree when you write that, when safety and fault-tolerance are
fundamental and "untrusted" code could not be allowed, the ideal
solution would be to run the untrusted FFI code in a separate process.
But it may require quite some work. From this point of view, I agree
with Denis: instead of developing more-or-less transparent pipe driver
with complicated serialization (as pointed out by Vlad), one may run
one or more Erlang nodes with the unsafe stuff, and issue remote calls
through a simplified API layer. This could be the cheaper and
possibly more flexible way to obtain both FFI and greater
fault-tolerance (at the price of speed, of course). And this approach
could even be used with "untrusted" linked-in drivers.
 Well, one may program an erlang:port_control/3 C callback that
reacts gracefully to every possible kind of garbage being received
as binary data. But the same defensive programming could be done
for C functions expected to be called through ffi:call/3. Being
able to call arbitrary C functions through FFI BIFs doesn't
prevent a library binding developer to create and document a set
of "safe" C functions that could be called from Erlang, with the
warning that other calls may have unexpected consequences.
 Let's see an "extreme" scenario: something like ffi:ecall/3
with some glue code based on libffi closures would allow (at least
in theory) to pass an Erlang fun() to the C side to be used as a
callback, even from code that is completely Erlang-unaware (as
an example: write a differential equation in Erlang and let the
GNU Scientific Library integrate it). I didn't find a way (yet)
to apply/3 an Erlang fun() from C code, but I didn't do much
investigation in the Erlang VM internals. The possibility to
call C-from-Erlang and Erlang-from-C, however, would increase
the Erlang language interoperability to the levels of Python,
Alceste Scalas <alceste@REDACTED>
CRS4 - http://www.crs4.it/
This message was sent using IMP, the Internet Messaging Program.
More information about the erlang-questions