[erlang-questions] best practice for calling C from Erlang

[James Churchman]

(also maybe auto generating thrift (or avro) interfaces could work.. tho i have never used the C++ thrift bindings or even built a thrift interface, just used it to talk to nosql systems..)
On 14 Dec 2010, at 00:58, James Churchman wrote:

> Yeah il look into the BERT format, erlang Port Program was what i was thinking (and yes,.. should have mentioned you can do the reverse call D 2.0 from c :-)
> The reason is that i need to apply potentially thousands of diffs per sec and im not sure erlang will be quick enough to process all of them even with hipe.. but il benchmark a bit before converting that module to D 2.0 (its about 300 lines of erlang code) and see just at what rate erlang starts to not be quick enough.. 
> 
> having looked into nifs a bit, i really cant get the thinking behind them, they allow mutation of erlang data types, don't copy messages but pass the memory directly, can crash the entire VM, block unrelated processes, change the realtime nature of erlang etc…. the only sensible use of nifs, unless you trust the reliability of the c code very highly, seems to be to run them on a separate erlang VM and message that. Would erlang not better have a c interface that allows direct C/++ function calls with a nice way of dealign with erlagn types like Nifs but copies memory & spawns separate OS thread pool for the code? that sounds more useful to me
> 
> yeah was thinking that it should be possible to dynamically get the type form the serration format used, but maybes it is best to get the full spec and not do it dynamically.. highly nested types sounds like it could get tricky so il take you word on it :-)
> 
> On 14 Dec 2010, at 00:27, Jachym Holecek wrote:
> 
>> # James Churchman 2010-12-13:
>>> been wondering myself what the best way to interface D 2.0 is to erlang…. most of the
>>> usual C ones should would as you can call C (some C++) from D2.0
>>> 
>>> i am not sure what the best communication / serialisation mechanism is,
>>> but given a suitable one..
>> 
>> I don't know D, but BERT format is what other Erlang interfaces use (Java,
>> Python, erl_interface for C) -- it's pretty simple and convenient, if you're
>> OK with talking to Erlang node over sockets/pipes.
>> 
>> If you can call D from C too, you could probably wrap you D code into
>> a builtin port driver or a NIF -- but personally I wouldn't bother unless
>> someone beat me with a benchmark. The BERT + pipes solution is easy to
>> debug and keeps things nicely isolated.
>> 
>>> for auto generating if you don't already have a list of all function
>>> names then you could use ctags (available in macports etc… ) the
>>> "etags" (eg " etags my_c_file.c ") command will give you just the
>>> function names and return values, but no arity..
>>> 
>>> this might be ok as you could autoproduce a lookup table of function
>>> pointers in c, and assume the arity is given correct from the erlang
>>> code (eg from the length of an input array, tuple etc..) and call the
>>> function pointer with the arty of the supplied message from erlang
>>> just using a case for aritys upto say 10, otherwise ctags (eg " ctags
>>> my_c_file.c ") will give a much more verbose output and could be used
>>> to produce the exact code without function pointers, or even auto
>>> produce thrift / avro etc.. mappings .
>>> 
>>> i know that ctags this is used by text editors and plugins to allow
>>> the browsing of source.. so hopefully will give suitable quality
>>> output
>> 
>> Yep, for just a list of function names ctags/etags would probably work fine,
>> but for both static interface and a more dynamic one like you describe
>> (function calls built at runtime based on user-supplied specs, presumably
>> using libffi as backend [*]) you really need full function signature -- and
>> not just that, sometimes you'd like to dissect nested/chained C structures
>> and unions, so you'd need to supply specs for that too (but more likely hand
>> written glue code).
>> 
>> Clang output gives you all that information for free, with the slight caveat
>> that type aliases that were done with C macros will be visible to you only
>> in their expanded form (and I think some simple typedefs disappeared that
>> way too, but I might be wrong on that, it's late night ;-). So autogenerated
>> code will still be type-correct, but may end up ugly and possibly specific
>> to some version of C code you're interfacing to.
>> 
>> HTH,
>> 	-- Jachym
>> 
>> [*] There used to be an EEP (along with implementation) that work this way...
>