[erlang-questions] Re: Processing huge binaries in NIFs

[Sverker Eriksson]

Konstantin Sorokin wrote:
> Hello!
>
> I want to process huge binaries in NIFs and in order not to screw up VM's
> scheduler I want to move time consuming jobs into a separate thread. Do I
> really need to make an internal copy of the binary before passing it to a
> thread for processing or it'll be OK just to set pointers to raw data in a
> task structure and then submit that task to the worker thread ?
>   

You don't have to copy the binary.

Solution 1.

Use enif_alloc_env() and enif_make_copy() to create a process 
independent binary term that you then can pass to your worker thread. 
enif_make_copy() will not copy the data if the binary is big, but rather 
increase a reference counter. Then use enif_free_env() to release your 
logical copy of the binary.

Solution 2.

Implement enif_keep_binary() which is on the ToDo list for the NIF API. 
enif_keep_binary() would just increase the reference counter of the 
ErlNifBinary without the hassle of having to create environment and 
Erlang term.

/Sverker, Erlang/OTP