<!DOCTYPE html>
<html>
<head>
<title></title>
</head>
<body><div>Or better yet, a bif :)<br></div>
<div><br></div>
<div id="sig19496053"><div class="signature">--<br></div>
<div class="signature"> Tristan Sloughter<br></div>
<div class="signature"> "I am not a crackpot" - Abe Simpson<br></div>
<div class="signature"> t@crashfast.com<br></div>
</div>
<div><br></div>
<div><br></div>
<div>On Sat, Jan 14, 2017, at 08:06 AM, Benoit Chesneau wrote:<br></div>
<blockquote type="cite"><div dir="ltr"><div><br></div>
<div><br></div>
<div defang_data-gmailquote="yes"><div dir="ltr">On Sat, Jan 14, 2017 at 4:53 PM Oliver Korpilla <<a href="mailto:Oliver.Korpilla@gmx.de">Oliver.Korpilla@gmx.de</a>> wrote:<br></div>
<blockquote defang_data-gmailquote="yes" style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204, 204, 204);padding-left:1ex;"><div>Could the Unicode support in elixir serve as a starting point?<br></div>
<div> <br></div>
<div> <a href="https://hexdocs.pm/elixir/1.3.3/String.html#content">https://hexdocs.pm/elixir/1.3.3/String.html#content</a><br></div>
<div> <br></div>
<div> String.upcase/1 and String.downcase/1 seem to be Unicode-aware. And a lot of effort seems have gone in scenarios like this:<br></div>
<div> <br></div>
<div> "For example, the codepoint “é” is two bytes:<br></div>
<div> <br></div>
<div> iex> byte_size("é")<br></div>
<div> 2"<br></div>
<div> <br></div>
<div> Given that both Erlang and elixir are implemented on top of BEAM, the wheel might not need reinventing? I know engineers and programmers love inventing stuff, and this discussion seems to point in that direction, but...<br></div>
<div> <br></div>
<div> Cheers,<br></div>
<div> Oliver<br></div>
<div> <br></div>
</blockquote><div><br></div>
<div><div>If I remember correctly the unicode support of Elixir is written in elixir and data come from the unicode/icu projects. data resources (codepoints and so on ) are compiled as beam. (I do the dame in my idna lib).<br></div>
</div>
<div><div>The work may be simpler in using/wrting a nif over the well supported ICU lib thoug. I'm curious about the reasonning that conducted to the current implementation in elixir.<br></div>
</div>
<div>- benoit<br></div>
<div><br></div>
<div> <br></div>
<blockquote defang_data-gmailquote="yes" style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204, 204, 204);padding-left:1ex;"><div> <br></div>
<div> <br></div>
<div> Gesendet: Freitag, 13. Januar 2017 um 23:34 Uhr<br></div>
<div> Von: "Michał Muskała" <<a href="mailto:michal@muskala.eu">michal@muskala.eu</a>><br></div>
<div> An: "Richard A. O'Keefe" <<a href="mailto:ok@cs.otago.ac.nz">ok@cs.otago.ac.nz</a>>, "Steve Davis" <<a href="mailto:steven.charles.davis@gmail.com">steven.charles.davis@gmail.com</a>>, <a href="mailto:g@rre.tt">g@rre.tt</a>, "Jesper Louis Andersen" <<a href="mailto:jesper.louis.andersen@gmail.com">jesper.louis.andersen@gmail.com</a>><br></div>
<div> Cc: "Erlang Questions" <<a href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a>><br></div>
<div> Betreff: Re: [erlang-questions] Erlang basic doubts about String, message passing and context switching overhead<br></div>
<div> <br></div>
<div> I fully agree there are no languages that deal with strings perfectly. That said there are those that are better at it and those that aren't so good. A language, where I need to look for a library to upcase or downcase my own name, fits into the second group in my book.<br></div>
<div> <br></div>
<div> Michał.<br></div>
<div> On 13 Jan 2017, 13:20 +0100, Jesper Louis Andersen <<a href="mailto:jesper.louis.andersen@gmail.com">jesper.louis.andersen@gmail.com</a>>, wrote:<br></div>
<div> <br></div>
<div> Richard is indeed right, depending on what your definition of "String" is.<br></div>
<div> If a "String" is "An array of characters from some alphabet", then you need to take into account Strings are Unicode codepoints in practice. This is also the most precise definition from a technical point of view.<br></div>
<div> When I wrote my post, I was--probably incorrectly--assuming the older notion of a "String" where the representation is either ASCII or something like ISO-8859-15. In this case, a string coincides with a stream of bytes.<br></div>
<div> Data needs parsing. A lot of data comes in as some kind of stringy representation: UTF-8, byte array (binary), and so on.<br></div>
<div> And of course, that isn't the whole story, since there are examples of input which are not string-like in their forms.<br></div>
<div> <br></div>
<div> <br></div>
<div> On Fri, Jan 13, 2017 at 2:34 AM Richard A. O'Keefe <<a href="mailto:ok@cs.otago.ac.nz">ok@cs.otago.ac.nz</a>[mailto:<a href="mailto:ok@cs.otago.ac.nz">ok@cs.otago.ac.nz</a>]> wrote:<br></div>
<div> <br></div>
<div> On 13/01/17 8:56 AM, Jesper Louis Andersen wrote:<br></div>
<div> > Strings are really just streams of bytes.<br></div>
<div> <br></div>
<div> That was true a long time ago. Maybe.<br></div>
<div> But it isn't anywhere near accurate as a description<br></div>
<div> of Unicode:<br></div>
<div> - Unicode is made of 21-bit code points, not bytes.<br></div>
<div> - Most possible code points are not defined.<br></div>
<div> - Some of those that are defined are defined as<br></div>
<div> "it is illegal to use this".<br></div>
<div> - Unicode sequences have *structure*; it is simply<br></div>
<div> not the case that every sequence of allowable<br></div>
<div> Unicode code points is a legal Unicode string.<br></div>
<div> - As a special case of that, if s is a non-empty<br></div>
<div> valid Unicode string, it is not true that every<br></div>
<div> substring of s is a valid Unicode string.<br></div>
<div> <br></div>
<div> In case you were thinking of UTF-8, not all byte<br></div>
<div> sequences are valid UTF-8.<br></div>
<div> <br></div>
<div> Byte streams are as important as you say, but it's<br></div>
<div> really hard to see the software for a radar or a<br></div>
<div> radio telescope as processing strings...<br></div>
<div> _______________________________________________<br></div>
<div> erlang-questions mailing list<br></div>
<div> <a href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a><br></div>
<div> <a href="http://erlang.org/mailman/listinfo/erlang-questions_______________________________________________">http://erlang.org/mailman/listinfo/erlang-questions_______________________________________________</a> erlang-questions mailing list <a href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a> <a href="http://erlang.org/mailman/listinfo/erlang-questions%5Bhttp://erlang.org/mailman/listinfo/erlang-questions%5D">http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions]</a><br></div>
<div> _______________________________________________<br></div>
<div> erlang-questions mailing list<br></div>
<div> <a href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a><br></div>
<div> <a href="http://erlang.org/mailman/listinfo/erlang-questions">http://erlang.org/mailman/listinfo/erlang-questions</a><br></div>
</blockquote></div>
</div>
<div><u>_______________________________________________</u><br></div>
<div>erlang-questions mailing list<br></div>
<div><a href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a><br></div>
<div><a href="http://erlang.org/mailman/listinfo/erlang-questions">http://erlang.org/mailman/listinfo/erlang-questions</a><br></div>
</blockquote><div><br></div>
</body>
</html>