Port driver communication witout copy of binaries

[Romain Lenglet]

Hi,


I have the following need: I want to wrap C functions in Erlang. 
Those functions get big binaries as input parameters, and return 
big binaries, among other kinds of data.
For efficiency, I would like to avoid to copy those binaries 
around when communicating. Therefore, I am forced to implement a 
C port driver, since this is the only available mechanism that 
does not create a separate system process (and hence does not 
require inter-process data copy when communicating).

If I needed only to send one binary in every message, that would 
be OK, e.g.:

% in Erlang:
Binary = <<...>>,
port_command(Port, Binary),

// in the C port implem:
void myoutput(ErlDrvData drv_data, char *buf, int len) {
...
}

I guess that the Binary is not copied, and its data in the Erlang 
heap is directly pointed by the *buf argument.
By the way, is that true??? Sending binaries that way is what is 
done in prim_inet for sending IP data, so I guess that no copy 
is done here.


However, I want to send and receive more complex data, which must 
be manipulated by the driver, typically a tuple of simple terms 
and binaries (which may be large), e.g. the tuple:
Tuple = {ContextHandle, QopReq, Message}
%%  ContextHandle = small binary()
%%  QopReq = integer() | atom()
%%  Message = large binary()

Such a tuple cannot be passed to port_command(Port, Tuple), since 
it is not an IO list. And if I encode it into a binary, by 
calling encode(Tuple), I guess that the binaries in the tuple 
will get copied in the process (can anybody confirm this?).


I have the same problem in the Driver -> Erlang direction, e.g. 
to send the tuple:
Tuple = {MajorStatus, MinorStatus, ConfState, QopState, 
OutputMessage}
%%      MajorStatus = integer()
%%      MinorStatus = integer()
%%      ConfState = bool()
%%      QopState = integer()
%%      OutputMessage = large binary()
I hope that using the driver_output_term() C function and the 
ErlDrvTermData construction technique, the binary data in the 
tuple above will not be copied. Can anybody confirm this?



Is there any clean solution to my problem? Or am I doomed to 
write my own BIFs and use my custom erts? Or to send data in 
multiple messages, in sequence?

I dream of a way to extend the BIFs list at runtime, by loading 
native libraries dynamically...

-- 
Romain LENGLET