[erlang-questions] Erlang FFI: 1st discussion summary
Alceste Scalas
alceste@REDACTED
Thu Sep 13 16:46:48 CEST 2007
Hello,
in order to make it easier to follow the Erlang FFI (Foreign
Function Interface) proposal, here's a summary of the discussion so
far, and a set of possible solutions for the issues being raised. I
could send other summaries like this, if/when needed.
The initial Erlang FFI proposal is still available in [0]. It was
accepted as EEP-0007 [1], but it has been frozen until the
discussion on this mailing list settles. A link to the first thread
of the discussion itself is in [2].
Here's an index of the issues and suggestions that have appeared so
far (I'm reordering them for ease of answer):
1. Raimo Niskanen proposed a type-tagged interface for the FFI
calls: call arguments and return values could be tuples in
the form {type_tag(), Value} [3];
2. Claes Wikström and Vlad Dumitrescu wondered how C pointers
(to buffers or strings) returned by FFI calls could be turned
into Erlang binaries;
3. when C buffers are turned into Erlang binaries, binary
matching could be used e.g. to extract C struct fields. But
this could require knowing the sizes of the C types on the
system in use.
Here are the proposals for addressing these issues (based on the
ideas gathered so far).
========================
1. Type-tagged FFI calls
========================
Type tags are extremely useful and can increase call safety, but
they can also kill FFI performance (see the final note in [4]). A
good compromise would be to leave the lower-level FFI BIFs without
tags, and implement type tags handling in Erlang (i.e. in a ffi.erl
module). Raimo agreed with this idea.
The old untagged ffi:call/3 and ffi:call/2 BIFs could be kept with a
different name (proposal: ffi:raw_call/3 and ffi:raw_call/2). The
higher-level, type-tagged interface for FFI calls could be:
ffi:call(Port, {ReturnType, Function}, [TaggedVal]) ->
{ReturnType,term()}
ReturnType = type_tag()
Function = string() | atom()
TaggedVal = {type_tag(), Val}
Val = term()
type_tag() = uchar|schar|...|pointer|size_t|ssize_t
It checks whether the required C function was preloaded with
erl_ddll:load_library/3. Then, two alternatives arise:
a. if the C function was preloaded, its signature is compared
with the type tags. If they match, a raw FFI call is
performed (with ffi:raw_call/2); otherwise, a badarg
exception is raised;
b. if the C function was *not* preloaded, the type tags will be
ignored and a raw FFI call will be performed (with
ffi:raw_call/3).
In both cases, the raw FFI call return value will be returned as a
{ReturnType, RawReturnValue} tuple.
--------------------------------------------------
1.1. Getting information about preloaded functions
--------------------------------------------------
The proposed high-level ffi:call/3 would need information about
functions and FFI signatures preloaded with
erl_ddll:load_library/3. This information could be useful for
developers, too (e.g. for debugging pourposes). For these
reasons, the erl_ddll:info/2 BIF could be extended with a
'preloads' argument, that would return a list of preloaded
functions, signatures etc. This information could be obtained
via erl_ddll:info/1 and erl_ddll:info/0 as well.
======================================================
2. Creating Erlang binaries from C strings and buffers
======================================================
The first proposal on this issue [5] can be revised considering type
tagging. A new 'cstring' type atom/tag can be introduced, in order
to distinguish NULL-terminated C strings from generic 'pointer's to
byte buffers. Two functions could be used for turning them into
Erlang binaries:
ffi:cstring_to_binary(TaggedCString) -> binary()
TaggedCString = {cstring, CStringPtr}
CStringPtr = integer()
Return a new binary with a copy of the given NULL-terminated
C string (including the trailing \0);
ffi:buffer_to_binary(TaggedPointer, Size) -> binary()
TaggedPointer = {pointer, Ptr}
Ptr = integer()
Return a new binary filled with a copy of Size bytes read
from the given C pointer.
These two functions would have, as seen in the previous section,
their type-untagged equivalents: ffi:raw_cstring_to_binary/1 and
ffi:raw_buffer_to_binary/2.
==================================
3. Determining the size of C types
==================================
The sizes of C types could be determined in run-time with a new
ffi:sizeof/1 BIF (initially proposed in [6]):
* ffi:sizeof(CType) -> integer()
CType = type_tag()
Return the number of bytes used by CType on the current
platform.
Type size information should, in general, *not* be hardcoded,
because it may change when running the same BEAM files on different
architectures. The BIF above is the recommended way for getting
type sizes when writing portable code.
However, when the FFI-based code is *not* expected to be portable
without recompilation, the size of C types remains constant and
could be determined when the Erlang/OTP sources are compiled. Thus,
this information could be stored in a .hrl file. Developers could
-include_lib("kernel/include/ffi_hardcodes.hrl") [7] and obtain a
set of faster and easier-to-use macros, for each supported FFI type:
FFI_HARDCODED_SIZEOF_<TYPE>
The type size in bytes
FFI_HARDCODED_<TYPE>_BITS
The type size in bits
The size in bits is precomputed in order to simplify binary
matching, since expressions like (?FFI_HARDCODED_SIZEOF_LONG * 8)
are not allowed in patterns.
===========================
4. Other minor enhancements
===========================
The following enhancements have never been discussed so far, but
they are very small and extremely trivial to implement:
* a new erl_ddll:load_library/2 function could be added, that
can be used instead of calling erl_ddll:load_library/3 with an
empty list of options (i.e. when no preloads are requested);
* when used with a library instead of a linked-in driver,
erlang:open_port/2 calls are quite noisy (the 'spawn' and the
list of options are redundant). A new erlang:open_port/1
function could be added:
erlang:open_port(Library) -> port()
Library = string()
Under the hoods, it could just call something like the
existing erlang:open_port({spawn, Library}, [binary]).
=============
5. That's all
=============
Please tell your opinion about the proposals above, and complain if
something is missing. Everything will be implemented depending on
your feedback. Thanks!
=====
Notes
=====
[0] Home page of the FFI for Erlang/OTP
http://muvara.org/crs4/erlang/ffi
[1] EEP-0007: Foreign Function Interface (FFI)
http://www.erlang.org/eeps/eep-0007.html
[2] Thread about the Erlang FFI on the erlang-questions mailing list
http://erlang.org/pipermail/erlang-questions/2007-September/029121.html
[3] Proposal for type-tagged FFI calls
http://erlang.org/pipermail/erlang-questions/2007-September/029174.html
[4] FFI tagging vs. call performance
http://erlang.org/pipermail/erlang-questions/2007-September/029179.html
[5] First proposal for turning C strings/buffers into binaries
http://erlang.org/pipermail/erlang-questions/2007-September/029141.html
[6] First proposal about a ffi:sizeof/1 BIF:
http://erlang.org/pipermail/erlang-questions/2007-September/029146.html
[7] From the implementation point of view, the ffi_hardcodes.hrl
header file could be autogenerated by GNU Autoconf from
ffi_hardcodes.hrl.in.
Regards,
alceste
--
Alceste Scalas <alceste@REDACTED>
CRS4 - http://www.crs4.it/
More information about the erlang-questions
mailing list