[erlang-questions] Proper way to run C code without crashing the VM
Jesper Louis Andersen
Sat Aug 19 17:05:12 CEST 2017
I did an experiment a couple of years ago:
1. Run an OCaml program behind a port from Erlang
2. *Pipeline* requests to the port. That is, don't wait for the roundtrip
of each message, just shove as many requests you can to the port and have
it handle them. The OCaml program just reflects whatever it got back (an
3. Use one CPU core for Erlang, once for OCaml.
This ran about 1 million req/s on a typical modern laptop. That is, the
amortized message overhead is about 1us. It suggests that the NIF usecase
is that you have many small calls which are latency bound (my above test is
a throughput one due to the amortization). If you have large datastructures
there may also be a reason for a NIF in some cases.
Measurement is key. I think there are many cases where you don't really
need a NIF and a port would do fine.
On Fri, Aug 18, 2017 at 11:13 AM Joe Armstrong <erlang@REDACTED> wrote:
> Re NIF's safety etc. Here's my take
> NIFs should be used for one reason only - performance.
> If performance is not a problem you should use an external port
> program, or an external c-node.
> Also the NIF should be called many times per second, not just once now
> and then - if the NIF is called infrequently then you should use and
> external port program.
> The only reason for using a NIF is to eliminate the round-trip times
> for talking to an external process, and speeding time to convert the
> internal data structures used by the programs since the NIFs have
> direct access to the stacks and heaps of the Erlang processes
> Using a NIF is breaking the guarantee - if you open the package and mess
> around with the bits inside then don't blame me if you break something
> If you can solve your problem in pure Erlang, then do so and don't use
> a NIF.
> There are a large number of reasons not to use NIFs
> - Your code might break the system in ways you don't understand
> - Your code will be single-platform and non-portable
> - Your code will not be 'future proof'
> - Understanding your code will require knowledge of Erlang +
> <another language> + a build procedure
> I have always believed that large systems should be made from small
> communicating components communicating via well-defined protocols -
> linking things together in memory makes the individual components
> larger and more unmanageable - we should be trying to do exactly the
> opposite, making things smaller and isolating them.
> The other problem with NIFs is that they make alternative
> implementations pf Erlang far more difficult. An Erlang implementation
> has four main components:
> 1) A compiler from Erlang to beam code
> 2) A beam instruction emulator
> 3) A load of BIFs (or NIF's if you like)
> 4) Huge libraries written in Erlang
> Each of these represent massive effort.
> Porting Erlang to (say) the JVM or .NET virtual machine would be
> relatively easy were it not for 3). Changing the compiler to a new
> instruction set and changing 1) and 2) is not that difficult but any
> changes the calling sequences would really mess up all the code for
> the BIFs and NIFs.
> Never forget that NIFs are an optimization and nothing else
> (since the same effect can be achieved in an external process) -
> "Premature Optimization is the root of all evil"
> [The best form of optimization is patience - in 20 years Erlang has become
> at least million times faster - of this probably a factor 10 comes from
> smarter compilation - and 100,000 from faster clock speeds and better
> (These are just guess but in 20 years Clock speeds have gone from Mhz to
> and memories increased from MBytes to GBytes)
> Pure Erlang code written 30 years ago ofter runs fine and is way faster
> than it was but the BIFs of 30 years ago are long dead and will not
> even compile.
> Patience is a virue]
> On Wed, Aug 16, 2017 at 3:53 PM, code wiget <codewiget95@REDACTED> wrote:
> > Hello,
> > I have been reading about NIF’s here:
> > http://erlang.org/doc/tutorial/nif.html but it seems like they are
> > incredibly unsafe: “it is also the least safe, because a crash in a NIF
> > brings the emulator down too”. The problem here is huge, ideally I can’t
> > ever have my Erlang VM close.
> > What solution do you Erlangers use when you have to run low level code?
> > goal is to send a sort of “GET” request with a UUID and I will receive
> > that UUID along with some binary values. My C code works with this, but I
> > can’t jeopardize the entire server crashing.
> > Thank you for your help!
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://erlang.org/mailman/listinfo/erlang-questions
> erlang-questions mailing list
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the erlang-questions