[erlang-questions] FFI: handling refcounted binaries

[Alceste Scalas]

Il giorno ven, 14/09/2007 alle 11.29 +0200, Martin Bjorklund ha scritto:
> >     ffi:cstring_to_binary(TaggedCString) -> binary()
> > 
> >     ffi:buffer_to_binary(TaggedPointer, Size) -> binary()
> > 
> > These two functions would have, as seen in the previous section,
> > their type-untagged equivalents: ffi:raw_cstring_to_binary/1 and
> > ffi:raw_buffer_to_binary/2.
> 
> In this case, I don't think the tagging gives you anything.

Tagging would make it more difficult to do a strcpy() over an arbitrary
pointer (thus avoiding a possible access violations).


> Another thing that would be nice would be to be able to get to a
> ref-counted binary in the C-code; i.e. let the C-code inc the ref
> count if it wants to.  This would save a copy when the binary is very
> big.

This would go slightly beyond the current FFI scope (that does *not*
allow to pass or return Erlang terms).  However, after a deeper review,
it seems to me that the ErlDrvBinary API still provides everything
necessary for this task.

Here is an implementation hypothesis, that should be checked by some
Erlang/OTP core developer.


=========================================
Handling refcounted binaries with the FFI
=========================================

Let's suppose that the 'binary' FFI type is added.

If you just want to *return* a binary (filled with data) from the C
side, it is easy.  You could just create an ErlDrvBinary in the usual
way, (erl_driver.h, driver_alloc_binary(), driver_binary_inc_refc()
etc.).  The FFI would turn it into a "normal" Erlang binary [1].

If you want to *receive* a handle to an Erlang binary in the form of an
ErlDrvBinary, and increment/decrement its refcount in order to keep it
for later use, then:

  a. if the Erlang binary is still refcounted, it is
     straightforward [2];

  b. however, a binary created from Erlang code may *not* be
     refcounted [3].  In this case, the FFI could force the
     creation of a refcounted binary, copy the contents
     of the original one, and pass an ErlDrvBinary to the C
     side [2].  There, the standard ErlDrvBinary API could be
     used.

This implementation *should* be totally compatible with the current
binary handling in Erlang linked-in drivers.  But this sentence should
be certified by Erlang/OTP core developers :-)

Regards,

alceste

Notes:

  [1] The FFI could just use the ErlDrvBinary2Binary macro
      in erts/emulator/beam/global.h

  [2] The FFI could just use the Binary2ErlDrvBinary macro
      in erts/emulator/beam/global.h

  [3] In the current Erlang VM implementation, if a binary is small
      enough it will live in the process heap, and will be copied
      around instead of refcounted.  The limit for these small
      binaries is 64 bytes, and is set by the ERL_ONHEAP_BIN_LIMIT
      in erts/emulator/beam/erl_binary.h.
      An Erlang-generated binary could also be a ErlSubBin, i.e. a
      portion of an existing binary (heap or refcounted).

-- 
Alceste Scalas <alceste@REDACTED>
CRS4 - http://www.crs4.it/