<div dir="ltr">Thanks everyone! I didn't realize until this conversation how much more important strings-as-binaries are, compared to simple "strings". Everything _works_ now, of course, but I don't think my understanding has caught up 100%<div><br></div><div><div>"by default it guesses that lists containing integers larger than 255</div><div>is not a string but a list of integers" <<< this really set some things straight</div><div><br></div><div>But suffer me this follow-up question, Dan. Using +pc unicode indeed gave me a shell that represents lists of integers using the characters found in Unicode mapping; so now, in error messages, I see arguments reported more clearly:</div><div><br></div><div>
<p class="gmail-m_-3758754219055318119gmail-p1"><span class="gmail-m_-3758754219055318119gmail-s1"><b>7> re:run("йцу.asd", xmerl_regexp:sh_to_awk("*.*"), [{capture, none}]). <span class="gmail-m_-3758754219055318119gmail-Apple-converted-space"> </span></b></span></p>
<p class="gmail-m_-3758754219055318119gmail-p1"><span class="gmail-m_-3758754219055318119gmail-s1"><b>** exception error: bad argument</b></span></p>
<p class="gmail-m_-3758754219055318119gmail-p1"><span class="gmail-m_-3758754219055318119gmail-s1"><b><span class="gmail-m_-3758754219055318119gmail-Apple-converted-space"> </span>in function<span class="gmail-m_-3758754219055318119gmail-Apple-converted-space"> </span>re:run/3</b></span></p>
<p class="gmail-m_-3758754219055318119gmail-p1"><span class="gmail-m_-3758754219055318119gmail-s1"><b><span class="gmail-m_-3758754219055318119gmail-Apple-converted-space"> </span>called as re:run("йцу.asd","^(.*\\..*)$"<wbr>,[{capture,none}])</b></span></p>
<p class="gmail-m_-3758754219055318119gmail-p1"><span class="gmail-m_-3758754219055318119gmail-s1"><b><br></b></span></p><p class="gmail-m_-3758754219055318119gmail-p1"><span class="gmail-m_-3758754219055318119gmail-s1">If I use the binary-string representation, it works _even_without_ /utf8, it works just fine:</span></p><p class="gmail-p1"><span class="gmail-s1"><b>3> re:run(<<"普通話.asd">>, xmerl_regexp:sh_to_awk("*.*"), [{capture, none}]).</b></span></p><p class="gmail-m_-3758754219055318119gmail-p1"><b>
</b></p><p class="gmail-p1"><span class="gmail-s1"><b>match</b></span></p></div><div class="gmail_extra">Note that the call above was executed in the shell started _without_ the +pc unicode, and the binary does _not_ have the /utf8>> thingy... This means my understanding is still lacking... binaries are honest and good, strings are fake and evil, but +px unicode seems to help a little with fake string... while using binary-strings doesn't _always_ require the /utf8 ... what is this sorcery?!</div><div class="gmail_extra">=)</div><div class="gmail_extra"><br></div><div class="gmail_extra"><br><div class="gmail_quote">On Sat, Oct 14, 2017 at 4:24 AM, Dan Gudmundsson <span dir="ltr"><<a href="mailto:dangud@gmail.com" target="_blank">dangud@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">re:run("йцу.asd", xmerl_regexp:sh_to_awk("*.*"), [{capture, none}, unicode]).<br><div>The binary one matches since it works on bytes and not utf-8 characters?<br></div><div><br></div><div>Also the erlang shell doesn't know if a list of integers is a list of integers or a string,</div><div>since they may be represented by the same list of integers.</div><div><br></div><div>So it tries to guess, by default it guesses that lists containing integers larger than 255</div><div>is not a string but a list of integers. You can change that with:</div><div><br></div><div>(w)erl +pc unicode</div><div><br></div><div>1> "йцу.asd".</div><div>"йцу.asd"</div><span class="gmail-m_-3758754219055318119gmail-m_1157110655033658819gmail-HOEnZb"><font color="#888888"><div><br></div><div>/Dan</div><div><br></div></font></span></div><br><div class="gmail_quote"><div><div class="gmail-m_-3758754219055318119gmail-m_1157110655033658819gmail-h5"><div dir="ltr">On Sat, Oct 14, 2017 at 10:12 AM Attila Rajmund Nohl <<a href="mailto:attila.r.nohl@gmail.com" target="_blank">attila.r.nohl@gmail.com</a>> wrote:<br></div></div></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div class="gmail-m_-3758754219055318119gmail-m_1157110655033658819gmail-h5">2017-10-14 4:21 GMT+02:00 Alexandre Karpov <<a href="mailto:alexakarpov@gmail.com" target="_blank">alexakarpov@gmail.com</a>>:<br>
> TL;DR: how do I run erl which understands Unicode?<br>
><br>
> Or, in more detail:<br>
><br>
> (Disclaimer: this official documentation got me really humbled:<br>
> <a href="http://www1.erlang.org/doc/apps/stdlib/unicode_usage.html" rel="noreferrer" target="_blank">http://www1.erlang.org/doc/app<wbr>s/stdlib/unicode_usage.html</a><br>
> , and just a little bit scared =) )<br>
><br>
> Judging by my S/O question, which got 3 upvotes and no answers, I'm not the<br>
> only one wondering:<br>
> <a href="https://stackoverflow.com/questions/46735539/erlang-regexp-matching-on-chinese-characters" rel="noreferrer" target="_blank">https://stackoverflow.com/ques<wbr>tions/46735539/erlang-regexp-m<wbr>atching-on-chinese-characters</a><br>
><br>
> Here's the gist of the problem:<br>
><br>
> 57> "абв".<br>
><br>
> [1072,1073,1074]<br>
><br>
> The codes are correct Unicode for the [Cyrillic] characters - which means my<br>
> Terminal didn't fail to understand my keyboard's input =) but Erlang shell<br>
> didn't recognize Terminal's input as printable characters. And it is my<br>
> understanding that this is exactly why this call fails:<br>
><br>
> 25> re:run("йцу.asd", xmerl_regexp:sh_to_awk("*.*"), [{capture, none}]). **<br>
> exception error: bad argument in function re:run/3 called as<br>
> re:run([1081,1094,1091,46,97,1<wbr>15,100], "^(.*\\..*)$", [{capture,none}])<br>
<br>
Try<br>
<br>
re:run(<<"йцу.asd"/utf8>>, xmerl_regexp:sh_to_awk("*.*"), [{capture, none}]).<br></div></div><span class="gmail-m_-3758754219055318119gmail-m_1157110655033658819gmail-">
______________________________<wbr>_________________<br>
erlang-questions mailing list<br>
<a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a><br>
<a href="http://erlang.org/mailman/listinfo/erlang-questions" rel="noreferrer" target="_blank">http://erlang.org/mailman/list<wbr>info/erlang-questions</a><br>
</span></blockquote></div>
</blockquote></div><br></div></div></div>