Port driver communication witout copy of binaries

[Raimo Niskanen]

What you really want to do can not be done (as far as I know)
but you might get it done with some tricks...

To avoid copying your driver must implement the
->outputv() entry point and you must send it I/O lists
being lists of binaries (might even be an improper list,
that is a binary in the tail). You will have to map
your tuples into that.

If you send [1,<<2,3,4>>,5,6|<<7,8>>] to the driver,
void (*outputv)(ErlDrvData drv_data, ErlIOVec *ev) will get:

ev->iov[0].iov_len = 1;
ev->iov[0].iov_base -> {1};
ev->binv[0] = NULL;
ev->iov[1].iov_len = 3;
ev->iov[1].iov_base -> ev->binv[1]->orig_bytes;
ev->binv[1]->orig_size = 3;
ev->binv[1]->orig_bytes = {2,3,4};
ev->iov[2].iov_len = 2;
ev->iov[2].iov_base -> {5,6};
ev->binv[2] = NULL;
ev->iov[3].iov_len = 2;
ev->iov[3].iov_base -> ev->binv[3]->orig_bytes;
ev->binv[3]->orig_size = 2;
ev->binv[3]->orig_bytes = {7,8};

approximately, excuse my syntax :-)

Binaries will be binaries and intermediate bytes
will be loose vectors. If your driver wants to
hang on to the data, it will have to use the
reference count in the binary to avoid premature freeing.

To send data back without copying your driver will
have to use driver_outputv() and it arrives to erlang as
a header list of integers followed by a list of
binaries. Conversion to tuple format will have to
be done in erlang.

Keep on dreaming...



Have a look at efile_drv.c in the sources...



rlenglet@REDACTED (Romain Lenglet) writes:

> Hi,
> 
> 
> I have the following need: I want to wrap C functions in Erlang. 
> Those functions get big binaries as input parameters, and return 
> big binaries, among other kinds of data.
> For efficiency, I would like to avoid to copy those binaries 
> around when communicating. Therefore, I am forced to implement a 
> C port driver, since this is the only available mechanism that 
> does not create a separate system process (and hence does not 
> require inter-process data copy when communicating).
> 
> If I needed only to send one binary in every message, that would 
> be OK, e.g.:
> 
> % in Erlang:
> Binary = <<...>>,
> port_command(Port, Binary),
> 
> // in the C port implem:
> void myoutput(ErlDrvData drv_data, char *buf, int len) {
> ...
> }
> 
> I guess that the Binary is not copied, and its data in the Erlang 
> heap is directly pointed by the *buf argument.
> By the way, is that true??? Sending binaries that way is what is 
> done in prim_inet for sending IP data, so I guess that no copy 
> is done here.
> 
> 
> However, I want to send and receive more complex data, which must 
> be manipulated by the driver, typically a tuple of simple terms 
> and binaries (which may be large), e.g. the tuple:
> Tuple = {ContextHandle, QopReq, Message}
> %%  ContextHandle = small binary()
> %%  QopReq = integer() | atom()
> %%  Message = large binary()
> 
> Such a tuple cannot be passed to port_command(Port, Tuple), since 
> it is not an IO list. And if I encode it into a binary, by 
> calling encode(Tuple), I guess that the binaries in the tuple 
> will get copied in the process (can anybody confirm this?).
> 
> 
> I have the same problem in the Driver -> Erlang direction, e.g. 
> to send the tuple:
> Tuple = {MajorStatus, MinorStatus, ConfState, QopState, 
> OutputMessage}
> %%      MajorStatus = integer()
> %%      MinorStatus = integer()
> %%      ConfState = bool()
> %%      QopState = integer()
> %%      OutputMessage = large binary()
> I hope that using the driver_output_term() C function and the 
> ErlDrvTermData construction technique, the binary data in the 
> tuple above will not be copied. Can anybody confirm this?
> 
> 
> 
> Is there any clean solution to my problem? Or am I doomed to 
> write my own BIFs and use my custom erts? Or to send data in 
> multiple messages, in sequence?
> 
> I dream of a way to extend the BIFs list at runtime, by loading 
> native libraries dynamically...
> 
> -- 
> Romain LENGLET

-- 

/ Raimo Niskanen, Erlang/OTP, Ericsson AB