[erlang-questions] Are binaries relocatable?

[Sverker Eriksson]

Looks like that should work if you're doing it right.

* Do you update shared data (like your reference counters) in a thread 
safe manner?

* Another way is to store the copied binary term and then use 
enif_make_copy(env, entry->content.bin.term)
to return the binary. You can also combine it with enif_make_sub_binary 
if you want to return part of the binary.

* Use debug built VM to ease trouble shooting:

cd $ERL_TOP/erts/emulator && make TYPE=debug smp plain
$ERL_TOP/bin/cerl -debug


/Sverker, Erlang/OTP


On 08/30/2014 07:07 AM, Hynek Vychodil wrote:
> Thanks for your answer very much. If I understand it correctly this way it
> should be safe:
>
>       ErlNifBinary bin;
>       if (!enif_is_binary(env, argv[0]))
>           return enif_make_badarg(env);
>
>       ErlNifEnv *priv_env = enif_alloc_env();
>       ERL_NIF_TERM term = enif_make_copy(priv_env, argv[0]);
>       ErlNifBinary bin;
>       enif_inspect_binary(priv_env, term, &bin);
>       entry->content.bin = (struct my_bin){.env = priv_env, .size =
> bin.size, .data = bin.data};
>
> And now mine entry->content.bin.data should be safe but unfortunately it
> still eventually hangs whole VM after few thousands cycles of referencing
> this data
>
>          CacheEntryRes_t *res = enif_alloc_resource(dc_entry_type,
> sizeof(CacheEntryRes_t));
>          unless (res) return insufficient_memory(env);
>          res->entry = entry;
>          ERL_NIF_TERM result = enif_make_resource_binary(env, res,
>                  entry->content.bin.data, entry->content.bin.size);
>          enif_release_resource(res);
>          return result;
>
> I do mine own reference count and release by
>
>         enif_free_env(entry->content.bin.env);
>
> It hangs only if I replace enif_alloc() + memcpy() and enif_free() version
> with above one :-(
>
> Thanks
>       Hynek
>
>
> On Fri, Aug 29, 2014 at 3:02 PM, Sverker Eriksson <
> sverker.eriksson@REDACTED> wrote:
>
>> On 08/28/2014 03:09 PM, Hynek Vychodil wrote:
>>
>>> Hi all,
>>> I would like to know if binaries are relocatable. If I have binary
>>> parameter
>>>
>>>       ErlNifBinary bin;
>>>       if (!enif_inspect_binary(env, argv[0], &bin))
>>>           return enif_make_badarg(env);
>>>
>>> I store binary in mine own environment
>>>
>>>       ErlNifEnv *priv_env = enif_alloc_env();
>>>       enif_make_copy(priv_env, argv[0]);
>>>
>>> Can I then rely on bin.data pointing into same memory until I free
>>> enif_free_env(priv_env) ?
>>>
>> No.
>> Your bin.data is refering to the original term argv[0] in 'env' which is
>> only valid until the NIF returns.
>> You must do enif_inspect_binary on the copy in order to get a safe pointer
>> that is valid until you do enif_free_env(priv_env).
>>
>>
>>
>>     I'm asking because I would like to avoid copying
>>> and share data or even return it using enif_make_resource_binary to catch
>>> and
>>> count references.
>>>
>> Whether a copied binary share data with its original is implementation
>> dependent. Today big binaries (>64 bytes) are shared
>> and small binaries are copied between environments.
>>
>>
>>   I have code which makes copy of binary data but when I switch to reference
>>> bin.data it hangs whole VM. I would like to know if I make some mistake in
>>> mine code and it should work or it is wrong idea at all.
>>>
>>>
>>>   There is a known bug when calling enif_make_copy after
>> enif_inspect_binary
>> that can cause the binary to get reallocated and thus the bin.data pointer
>> to get invalid
>> before the NIF returns. To detect this bug, you can call
>> enif_inspect_binary(env,argv[0],&bin2)
>> after enif_make_copy and check if bin2.data != bin.data.
>>
>> This bug has been lingering as I thought it was an exotic scenario
>> (apparently not) and I have
>> been a bit unsure how to fix it without ruin performance.
>>
>>
>> /Sverker, Erlang/OTP
>>
>>