<div dir="ltr">Nif would make erlang distribution way less portable than it is now. At the moment erlang compiled on with static openss can be moven just by copying between most modern linux distros of the same bits. ICU is a C++ lib, anything with linkage against libstdc++ is not portable, new versions of GCC break compatibility of libstdc++ quite often.</div><div class="gmail_extra"><br><div class="gmail_quote">2017-01-14 19:06 GMT+03:00 Benoit Chesneau <span dir="ltr"><<a href="mailto:bchesneau@gmail.com" target="_blank">bchesneau@gmail.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><br><br><div class="gmail_quote"><span class=""><div dir="ltr">On Sat, Jan 14, 2017 at 4:53 PM Oliver Korpilla <<a href="mailto:Oliver.Korpilla@gmx.de" target="_blank">Oliver.Korpilla@gmx.de</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Could the Unicode support in elixir serve as a starting point?<br class="m_-5931038163397019274gmail_msg">
<br class="m_-5931038163397019274gmail_msg">
<a href="https://hexdocs.pm/elixir/1.3.3/String.html#content" rel="noreferrer" class="m_-5931038163397019274gmail_msg" target="_blank">https://hexdocs.pm/elixir/1.3.<wbr>3/String.html#content</a><br class="m_-5931038163397019274gmail_msg">
<br class="m_-5931038163397019274gmail_msg">
String.upcase/1 and String.downcase/1 seem to be Unicode-aware. And a lot of effort seems have gone in scenarios like this:<br class="m_-5931038163397019274gmail_msg">
<br class="m_-5931038163397019274gmail_msg">
"For example, the codepoint “é” is two bytes:<br class="m_-5931038163397019274gmail_msg">
<br class="m_-5931038163397019274gmail_msg">
iex> byte_size("é")<br class="m_-5931038163397019274gmail_msg">
2"<br class="m_-5931038163397019274gmail_msg">
<br class="m_-5931038163397019274gmail_msg">
Given that both Erlang and elixir are implemented on top of BEAM, the wheel might not need reinventing? I know engineers and programmers love inventing stuff, and this discussion seems to point in that direction, but...<br class="m_-5931038163397019274gmail_msg">
<br class="m_-5931038163397019274gmail_msg">
Cheers,<br class="m_-5931038163397019274gmail_msg">
Oliver<br class="m_-5931038163397019274gmail_msg">
<br class="m_-5931038163397019274gmail_msg"></blockquote><div><br></div></span><div>If I remember correctly the unicode support of Elixir is written in elixir and data come from the unicode/icu projects. data resources (codepoints and so on ) are compiled as beam. (I do the dame in my idna lib).<br><br></div><div>The work may be simpler in using/wrting a nif over the well supported ICU lib thoug. I'm curious about the reasonning that conducted to the current implementation in elixir.<span class="HOEnZb"><font color="#888888"><br><br></font></span></div><span class="HOEnZb"><font color="#888888"><div>- benoit<br></div></font></span><div><div class="h5"><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br class="m_-5931038163397019274gmail_msg">
<br class="m_-5931038163397019274gmail_msg">
Gesendet: Freitag, 13. Januar 2017 um 23:34 Uhr<br class="m_-5931038163397019274gmail_msg">
Von: "Michał Muskała" <<a href="mailto:michal@muskala.eu" class="m_-5931038163397019274gmail_msg" target="_blank">michal@muskala.eu</a>><br class="m_-5931038163397019274gmail_msg">
An: "Richard A. O'Keefe" <<a href="mailto:ok@cs.otago.ac.nz" class="m_-5931038163397019274gmail_msg" target="_blank">ok@cs.otago.ac.nz</a>>, "Steve Davis" <<a href="mailto:steven.charles.davis@gmail.com" class="m_-5931038163397019274gmail_msg" target="_blank">steven.charles.davis@gmail.<wbr>com</a>>, <a href="mailto:g@rre.tt" class="m_-5931038163397019274gmail_msg" target="_blank">g@rre.tt</a>, "Jesper Louis Andersen" <<a href="mailto:jesper.louis.andersen@gmail.com" class="m_-5931038163397019274gmail_msg" target="_blank">jesper.louis.andersen@gmail.<wbr>com</a>><br class="m_-5931038163397019274gmail_msg">
Cc: "Erlang Questions" <<a href="mailto:erlang-questions@erlang.org" class="m_-5931038163397019274gmail_msg" target="_blank">erlang-questions@erlang.org</a>><br class="m_-5931038163397019274gmail_msg">
Betreff: Re: [erlang-questions] Erlang basic doubts about String, message passing and context switching overhead<br class="m_-5931038163397019274gmail_msg">
<br class="m_-5931038163397019274gmail_msg">
I fully agree there are no languages that deal with strings perfectly. That said there are those that are better at it and those that aren't so good. A language, where I need to look for a library to upcase or downcase my own name, fits into the second group in my book.<br class="m_-5931038163397019274gmail_msg">
<br class="m_-5931038163397019274gmail_msg">
Michał.<br class="m_-5931038163397019274gmail_msg">
On 13 Jan 2017, 13:20 +0100, Jesper Louis Andersen <<a href="mailto:jesper.louis.andersen@gmail.com" class="m_-5931038163397019274gmail_msg" target="_blank">jesper.louis.andersen@gmail.<wbr>com</a>>, wrote:<br class="m_-5931038163397019274gmail_msg">
<br class="m_-5931038163397019274gmail_msg">
Richard is indeed right, depending on what your definition of "String" is.<br class="m_-5931038163397019274gmail_msg">
If a "String" is "An array of characters from some alphabet", then you need to take into account Strings are Unicode codepoints in practice. This is also the most precise definition from a technical point of view.<br class="m_-5931038163397019274gmail_msg">
When I wrote my post, I was--probably incorrectly--assuming the older notion of a "String" where the representation is either ASCII or something like ISO-8859-15. In this case, a string coincides with a stream of bytes.<br class="m_-5931038163397019274gmail_msg">
Data needs parsing. A lot of data comes in as some kind of stringy representation: UTF-8, byte array (binary), and so on.<br class="m_-5931038163397019274gmail_msg">
And of course, that isn't the whole story, since there are examples of input which are not string-like in their forms.<br class="m_-5931038163397019274gmail_msg">
<br class="m_-5931038163397019274gmail_msg">
<br class="m_-5931038163397019274gmail_msg">
On Fri, Jan 13, 2017 at 2:34 AM Richard A. O'Keefe <<a href="mailto:ok@cs.otago.ac.nz" class="m_-5931038163397019274gmail_msg" target="_blank">ok@cs.otago.ac.nz</a>[mailto:<a href="mailto:ok@cs.otago.ac.nz" class="m_-5931038163397019274gmail_msg" target="_blank">ok@<wbr>cs.otago.ac.nz</a>]> wrote:<br class="m_-5931038163397019274gmail_msg">
<br class="m_-5931038163397019274gmail_msg">
On 13/01/17 8:56 AM, Jesper Louis Andersen wrote:<br class="m_-5931038163397019274gmail_msg">
> Strings are really just streams of bytes.<br class="m_-5931038163397019274gmail_msg">
<br class="m_-5931038163397019274gmail_msg">
That was true a long time ago. Maybe.<br class="m_-5931038163397019274gmail_msg">
But it isn't anywhere near accurate as a description<br class="m_-5931038163397019274gmail_msg">
of Unicode:<br class="m_-5931038163397019274gmail_msg">
- Unicode is made of 21-bit code points, not bytes.<br class="m_-5931038163397019274gmail_msg">
- Most possible code points are not defined.<br class="m_-5931038163397019274gmail_msg">
- Some of those that are defined are defined as<br class="m_-5931038163397019274gmail_msg">
"it is illegal to use this".<br class="m_-5931038163397019274gmail_msg">
- Unicode sequences have *structure*; it is simply<br class="m_-5931038163397019274gmail_msg">
not the case that every sequence of allowable<br class="m_-5931038163397019274gmail_msg">
Unicode code points is a legal Unicode string.<br class="m_-5931038163397019274gmail_msg">
- As a special case of that, if s is a non-empty<br class="m_-5931038163397019274gmail_msg">
valid Unicode string, it is not true that every<br class="m_-5931038163397019274gmail_msg">
substring of s is a valid Unicode string.<br class="m_-5931038163397019274gmail_msg">
<br class="m_-5931038163397019274gmail_msg">
In case you were thinking of UTF-8, not all byte<br class="m_-5931038163397019274gmail_msg">
sequences are valid UTF-8.<br class="m_-5931038163397019274gmail_msg">
<br class="m_-5931038163397019274gmail_msg">
Byte streams are as important as you say, but it's<br class="m_-5931038163397019274gmail_msg">
really hard to see the software for a radar or a<br class="m_-5931038163397019274gmail_msg">
radio telescope as processing strings...<br class="m_-5931038163397019274gmail_msg">
_____________________________<wbr>__________________<br class="m_-5931038163397019274gmail_msg">
erlang-questions mailing list<br class="m_-5931038163397019274gmail_msg">
<a href="mailto:erlang-questions@erlang.org" class="m_-5931038163397019274gmail_msg" target="_blank">erlang-questions@erlang.org</a><br class="m_-5931038163397019274gmail_msg">
<a href="http://erlang.org/mailman/listinfo/erlang-questions_______________________________________________" rel="noreferrer" class="m_-5931038163397019274gmail_msg" target="_blank">http://erlang.org/mailman/<wbr>listinfo/erlang-questions_____<wbr>______________________________<wbr>____________</a> erlang-questions mailing list <a href="mailto:erlang-questions@erlang.org" class="m_-5931038163397019274gmail_msg" target="_blank">erlang-questions@erlang.org</a> <a href="http://erlang.org/mailman/listinfo/erlang-questions%5Bhttp://erlang.org/mailman/listinfo/erlang-questions%5D" rel="noreferrer" class="m_-5931038163397019274gmail_msg" target="_blank">http://erlang.org/mailman/<wbr>listinfo/erlang-questions[<wbr>http://erlang.org/mailman/<wbr>listinfo/erlang-questions]</a><br class="m_-5931038163397019274gmail_msg">
______________________________<wbr>_________________<br class="m_-5931038163397019274gmail_msg">
erlang-questions mailing list<br class="m_-5931038163397019274gmail_msg">
<a href="mailto:erlang-questions@erlang.org" class="m_-5931038163397019274gmail_msg" target="_blank">erlang-questions@erlang.org</a><br class="m_-5931038163397019274gmail_msg">
<a href="http://erlang.org/mailman/listinfo/erlang-questions" rel="noreferrer" class="m_-5931038163397019274gmail_msg" target="_blank">http://erlang.org/mailman/<wbr>listinfo/erlang-questions</a><br class="m_-5931038163397019274gmail_msg">
</blockquote></div></div></div></div>
<br>______________________________<wbr>_________________<br>
erlang-questions mailing list<br>
<a href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a><br>
<a href="http://erlang.org/mailman/listinfo/erlang-questions" rel="noreferrer" target="_blank">http://erlang.org/mailman/<wbr>listinfo/erlang-questions</a><br>
<br></blockquote></div><br></div>