<div dir="ltr">Hey Tony,<div><br></div><div>Yes, this binary is in utf8 - and it is what it's being fed to xmerl library, which returns it in the other encoding by some reasons.</div></div><div class="gmail_extra"><br><div class="gmail_quote">On 17 August 2015 at 13:55, Tony Rogvall <span dir="ltr"><<a href="mailto:tony@rogvall.se" target="_blank">tony@rogvall.se</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word">Sorry for the empty message :-)<div><br></div><div>But the coding you are looking for is already in your binary.</div><div><br><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span class=""> <<131,109,0,0,0,79,60,63,120,109,108,32,118,101,114,115,<br> 105,111,110,61,34,49,46,48,34,32,101,110,99,111,100,105,<br> 110,103,61,34,85,84,70,45,56,34,63,62,60,114,101,115,<br> 112,111,110,115,101,62,60,118,97,108,117,101,62,82,101,<br> 110,<font color="#e63b7a">195,169</font>,60,47,118,97,108,117,101,62,60,47,114,101,<br></span><div class="gmail_extra"><div class="gmail_quote"><div><div><div><div> 115,112,111,110,115,101,62>></div><div><br></div><div>195,169 = c3 a9</div><span class="HOEnZb"><font color="#888888"><div><br></div><div>/Tony</div><div><br></div></font></span></div></div></div></div></div></div></div></div></div><div><br><div><blockquote type="cite"><span class=""><div>On 17 aug 2015, at 14:39, Alexander Turkin <<a href="mailto:snowwlex@gmail.com" target="_blank">snowwlex@gmail.com</a>> wrote:</div><br></span><div><div class="h5"><div><div dir="ltr"><div class="gmail_extra">Hi Hynek, </div><div class="gmail_extra"><br><div class="gmail_quote">On 15 August 2015 at 08:30, Hynek Vychodil <span dir="ltr"><<a href="mailto:vychodil.hynek@gmail.com" target="_blank">vychodil.hynek@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">The same result is in R18 and it it correct result. Letter é has unicode 233 see <a href="http://unicode-table.com/en/#00E9" target="_blank">http://unicode-table.com/en/#00E9</a></div></blockquote><div><br></div><div>Yeah, it has U+00E9 (= 233) code point number, but it is coded in 2 bytes in utf8: c3 a9 </div><div><br></div><div><table style="border-collapse:collapse;font-family:Times;font-size:inherit"><tbody><tr style="background-color:rgb(248,248,248)"><td style="text-align:center;border:1px solid black;padding-left:0.5em;padding-right:0.5em">U+00E9</td><td style="text-align:center;border:1px solid black;padding-left:0.5em;padding-right:0.5em">é</td><td style="text-align:center;border:1px solid black;padding-left:0.5em;padding-right:0.5em">c3 a9</td><td style="margin-left:1.5em;border:1px solid black;padding-left:0.5em;padding-right:0.5em">LATIN SMALL LETTER E WITH ACUTE<br></td></tr></tbody></table></div><div> </div><div><br></div><div>(<a href="http://www.utf8-chartable.de/" target="_blank">http://www.utf8-chartable.de/</a>) </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="gmail_extra"><br><div class="gmail_quote"><div><div>On Fri, Aug 14, 2015 at 6:23 PM, Éric Pailleau <span dir="ltr"><<a href="mailto:eric.pailleau@wanadoo.fr" target="_blank">eric.pailleau@wanadoo.fr</a>></span> wrote:<br></div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div>Hello,<br>
Please precise what Erlang release you are using. Utf8 came lately in Erlang.<br>
Regards<br>
<div><div><br>
Le 14 août 2015 13:49, Alexander Turkin <<a href="mailto:snowwlex@gmail.com" target="_blank">snowwlex@gmail.com</a>> a écrit :<br>
><br>
> Dear list,<br>
><br>
><br>
> I've got a problem with unicode & xmerl library.<br>
><br>
> Input data for xmerl is utf-8 encoded xml, and what I've got as the result is encoded latin1. But I need utf8!<br>
><br>
><br>
> EXAMPLES<br>
><br>
> Body = <<"<?xml version=\"1.0\" encoding=\"UTF-8\"?><response><value>René</value></response>"/utf8>>.<br>
><br>
> (for the sake of portability here is term_to_binary(Body): <br>
><br>
> <<131,109,0,0,0,79,60,63,120,109,108,32,118,101,114,115,<br>
> 105,111,110,61,34,49,46,48,34,32,101,110,99,111,100,105,<br>
> 110,103,61,34,85,84,70,45,56,34,63,62,60,114,101,115,<br>
> 112,111,110,115,101,62,60,118,97,108,117,101,62,82,101,<br>
> 110,195,169,60,47,118,97,108,117,101,62,60,47,114,101,<br>
> 115,112,111,110,115,101,62>><br>
><br>
><br>
><br>
> (1):<br>
><br>
> When I do <br>
><br>
> xmerl_scan:string(binary_to_list(Body)).<br>
><br>
> it returns <br>
><br>
> {#xmlElement{name = response,expanded_name = response,<br>
> nsinfo = [],<br>
> namespace = #xmlNamespace{default = [],nodes = []},<br>
> parents = [],pos = 1,attributes = [],<br>
> content = [#xmlElement{name = value,expanded_name = value,<br>
> nsinfo = [],<br>
> namespace = #xmlNamespace{default = [],nodes = []},<br>
> parents = [{response,1}],<br>
> pos = 1,attributes = [],<br>
> content = [#xmlText{parents = [{value,1},{response,1}],<br>
> pos = 1,language = [],<br>
><br>
><br>
> value = "René",<br>
><br>
><br>
> type = text}],<br>
> language = [],xmlbase = "/Users/aturkin/ws/",<br>
> elementdef = undeclared}],<br>
> language = [],xmlbase = "/Users/aturkin/ws/",<br>
> elementdef = undeclared},<br>
> []}<br>
><br>
><br>
> So, note there is `value = "René"` string, and it uses [233] symbol, which is latin1.<br>
><br>
><br>
><br>
><br>
> (2):<br>
><br>
> xmerl_scan:string(xmerl_ucs:to_utf8(binary_to_list(Body)))<br>
><br>
> returns <br>
><br>
> {#xmlElement{name = response,expanded_name = response,<br>
> nsinfo = [],<br>
> namespace = #xmlNamespace{default = [],nodes = []},<br>
> parents = [],pos = 1,attributes = [],<br>
> content = [#xmlElement{name = value,expanded_name = value,<br>
> nsinfo = [],<br>
> namespace = #xmlNamespace{default = [],nodes = []},<br>
> parents = [{response,1}],<br>
> pos = 1,attributes = [],<br>
> content = [#xmlText{parents = [{value,1},{response,1}],<br>
> pos = 1,language = [],<br>
><br>
><br>
> value = "René",<br>
><br>
><br>
> type = text}],<br>
> language = [],xmlbase = "/Users/aturkin/ws/",<br>
> elementdef = undeclared}],<br>
> language = [],xmlbase = "/Users/aturkin/ws/",<br>
> elementdef = undeclared},<br>
> []}<br>
><br>
> Now `value = "René"`, so 2 bytes are used to code this symbol, and this is utf-8.<br>
><br>
> So in (2) I get what I need, but why I need to force that conversion for xmerl? <br>
><br>
><br>
><br>
><br>
> QUESTIONS<br>
><br>
> 1. I don't understand why xmerl_scan allows you to set input encoding, but it looks like there is no way to set output encoding. Is there any way to make xmerl_scan to return utf8 instead of latin1?<br>
><br>
> 2. How is that happen, that in (1) it does conversion utf-8 -> latin1, and in (2) it's utf-8?<br>
><br>
><br>
><br>
><br>
> --<br>
> Best Regards,<br>
> Alex Turkin<br>
</div></div></div></div>_______________________________________________<br>
erlang-questions mailing list<br>
<a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a><br>
<a href="http://erlang.org/mailman/listinfo/erlang-questions" rel="noreferrer" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a><br>
</blockquote></div><br></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div>Best Regards,<div>Alex Turkin</div></div>
</div></div>
_______________________________________________<br>erlang-questions mailing list<br><a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a><br><a href="http://erlang.org/mailman/listinfo/erlang-questions" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a><br></div></div></div></blockquote></div><br></div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature">Best Regards,<div>Alex Turkin</div></div>
</div>