[erlang-bugs] erl nif utf8 bug

[Paul Davis]

The definition of iolist() is [char(), binary(), iolist()] which your
example violates which is demonstrated by snippet I pasted using
iolist_to_binary/1.

If you check the docs to io:format/2 you'll notice that it accepts
arbitrary term()'s not just iolist()'s which is why that works.

On Sat, Feb 25, 2012 at 6:36 PM, Heinz N. Gies <heinz@REDACTED> wrote:
> Hi Paul,
> thanks for the answer, of cause you are right. Yet I still don't think enif_inspect_iolist_as_binary should throw an error here since the iolist [910275] is a perfectly valid io list.
>
> 8> io:format("~ts~n", [[910275]]).
> �� (funky utf8 char)
> ok
>
>
>
> Regards,
> Heinz
> --
> Heinz N. Gies
> heinz@REDACTED
> http://licenser.net
>
> On Feb 26, 2012, at 24:17, Paul Davis wrote:
>
>> Couple things wrong here. Firstly, enif_inspect_iolist_as_binary is
>> akin to iolist_to_binary. Specifically, it doesn't do any conversions
>> from Unicode code points to binary. To see that in action:
>>
>> Eshell V5.8.5  (abort with ^G)
>> 1> iolist_to_binary([910275]).
>> ** exception error: bad argument
>>     in function  iolist_to_binary/1
>>        called as iolist_to_binary([910275])
>>
>> The actual error here is that you're not checking return codes.
>> enif_inspect_iolist_as_binary is returning an error that you're
>> ignoring. Quick diff is:
>>
>> diff --git a/c_src/mymodule.c b/c_src/mymodule.c
>> index af311d2..23c9bfc 100644
>> --- a/c_src/mymodule.c
>> +++ b/c_src/mymodule.c
>> @@ -33,7 +33,9 @@ static ERL_NIF_TERM mymodule_nif_bin_size(ErlNifEnv*
>> env, int                                            const ERL_NIF_TERM
>> argv[])
>> {
>>   ErlNifBinary bin;
>> -  enif_inspect_iolist_as_binary(env, argv[1], &bin);
>> +  if(!enif_inspect_iolist_as_binary(env, argv[1], &bin)) {
>> +    return enif_make_badarg(env);
>> +  }
>>
>>   return enif_make_int(env, bin.size);
>> }
>>
>> On Sat, Feb 25, 2012 at 4:57 PM, Heinz N. Gies <heinz@REDACTED> wrote:
>>> I noticed that the nif functions in erlang don't handle utf8/16 correctly
>>>
>>> https://github.com/Licenser/erlniftest is an example of the problem.
>>>
>>> Regards,
>>> Heinz
>>> --
>>> Heinz N. Gies
>>> heinz@REDACTED
>>> http://licenser.net
>>>
>>> _______________________________________________
>>> erlang-bugs mailing list
>>> erlang-bugs@REDACTED
>>> http://erlang.org/mailman/listinfo/erlang-bugs
>