<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-2">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
</head>
<body dir="ltr">
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;font-family:Calibri,Arial,Helvetica,sans-serif;" dir="ltr">
<p><br>
</p>
<div>>> <span>Elixir takes the files directly from unicode distribution, and uses macros to compile them down into regular functions.</span></div>
<div><br>
</div>
<div>Code generation. Can be done without macros, just generating Erlang files. So everyone in BEAM community can use. That is my point.</div>
<div><br>
<br>
</div>
<div style="color: rgb(0, 0, 0);">
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="divRplyFwdMsg" dir="ltr"><font color="#000000" face="Calibri, sans-serif" style="font-size:11pt"><b>From:</b> Michał Muskała <michal@muskala.eu><br>
<b>Sent:</b> Saturday, January 14, 2017 22:56<br>
<b>To:</b> Oliver Korpilla; Ilya Khaprov<br>
<b>Cc:</b> Erlang Questions<br>
<b>Subject:</b> RE: [erlang-questions] Erlang basic doubts about String, message passing and context switching overhead</font>
<div> </div>
</div>
<div>
<div style="font-size:14px" name="messageBodySection">On 14 Jan 2017, 17:00 +0100, Ilya Khaprov <ilya.khaprov@publitechs.com>, wrote:<br>
</div>
<div style="font-size:14px" name="messageReplySection"><br>
<blockquote style="margin:5px 5px; padding-left:10px; border-left:thin solid #1abc9c" type="cite">
<meta name="x_Generator" content="Microsoft Word 15 (filtered medium)">
<div lang="EN-US">
<div class="x_WordSection1">
<p class="x_MsoNormal">>> Given that both Erlang and elixir are implemented on top of BEAM, the wheel might not need reinventing?</p>
<p class="x_MsoNormal"> </p>
<p class="x_MsoNormal">Why Elixir implements Unicode in Elixir? You have to rewrite it anyway. </p>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>Elixir takes the files directly from unicode distribution, and uses macros to compile them down into regular functions. BEAM is wonderful at optimising bitstring pattern matching, so the resulting code is quite efficient and does not need NIFs. The whole
implementation is also about 500 LoC <a id="LPlnk26348" href="https://github.com/elixir-lang/elixir/blob/master/lib/elixir/unicode/unicode.ex" previewremoved="true">https://github.com/elixir-lang/elixir/blob/master/lib/elixir/unicode/unicode.ex</a>, so it's
not a monstrosity and the code is quite readable.</div>
<div id="LPBorder_GT_14844249114140.3615429786955964" style="width: 100%; text-indent: 0px; overflow: auto; margin-bottom: 20px;">
<table id="LPContainer_14844249114120.4128737934233542" style="width: 90%; overflow: auto; padding-top: 20px; padding-bottom: 20px; margin-top: 20px; border-top-color: rgb(200, 200, 200); border-bottom-color: rgb(200, 200, 200); border-top-width: 1px; border-bottom-width: 1px; border-top-style: dotted; border-bottom-style: dotted; position: relative; background-color: rgb(255, 255, 255);" cellspacing="0">
<tbody>
<tr valign="top" style="border-spacing: 0px;">
<td id="ImageCell_14844249114120.08076961126658766" style="width: 250px; padding-right: 20px; display: table-cell; position: relative;" colspan="1">
<div id="LPImageContainer_14844249114120.1160602688868646" style="margin: auto; width: 250px; height: 250px; display: table; position: relative; background-color: rgb(255, 255, 255);">
<a id="LPImageAnchor_14844249114130.7082125635200362" style="text-align: center; display: table-cell;" href="https://github.com/elixir-lang/elixir/blob/master/lib/elixir/unicode/unicode.ex" target="_blank"><img width="250" height="250" id="LPThumbnailImageID_14844249114130.6875144340589112" style="border-width: 0px; width: 250px; height: 250px; vertical-align: bottom; display: inline-block; max-height: 250px; max-width: 250px;" aria-label="Preview image with link selected. Double-tap to open the link." src="https://avatars2.githubusercontent.com/u/1481354?v=3&s=400"></a></div>
</td>
<td id="TextCell_14844249114130.5232928719783496" style="padding: 0px; vertical-align: top; display: table-cell; position: relative;" colspan="2">
<div id="LPRemovePreviewContainer_14844249114130.8899936461369555"></div>
<div id="LPTitle_14844249114130.11560629874506611" style="top: 0px; color: rgb(0, 120, 215); line-height: 21px; font-family: "wf_segoe-ui_light","Segoe UI Light","Segoe WP Light","Segoe UI","Segoe WP",Tahoma,Arial,sans-serif; font-size: 21px; font-weight: 400;">
<a id="LPUrlAnchor_14844249114140.6632444810280371" style="text-decoration: none;" href="https://github.com/elixir-lang/elixir/blob/master/lib/elixir/unicode/unicode.ex" target="_blank">elixir/unicode.ex at master · elixir-lang/elixir · GitHub</a></div>
<div id="LPMetadata_14844249114140.6142861760735347" style="margin: 10px 0px 16px; color: rgb(102, 102, 102); line-height: 14px; font-family: "wf_segoe-ui_normal","Segoe UI","Segoe WP",Tahoma,Arial,sans-serif; font-size: 14px; font-weight: 400;">
github.com</div>
<div id="LPDescription_14844249114140.9168132488687726" style="color: rgb(102, 102, 102); line-height: 20px; overflow: hidden; font-family: "wf_segoe-ui_normal","Segoe UI","Segoe WP",Tahoma,Arial,sans-serif; font-size: 14px; font-weight: 400; display: block; max-height: 100px;">
elixir - Elixir is a dynamic, functional language designed for building scalable and maintainable applications</div>
</td>
</tr>
</tbody>
</table>
</div>
<br>
<div><br>
</div>
<div>On 14 Jan 2017, 16:53 +0100, Oliver Korpilla <Oliver.Korpilla@gmx.de>, wrote:<br>
<blockquote style="margin:5px 5px; padding-left:10px; border-left:thin solid #1abc9c" type="cite">
Could the Unicode support in elixir serve as a starting point?<br>
<br>
https://hexdocs.pm/elixir/1.3.3/String.html#content<br>
<br>
String.upcase/1 and String.downcase/1 seem to be Unicode-aware. And a lot of effort seems have gone in scenarios like this:<br>
<br>
"For example, the codepoint “é” is two bytes:<br>
<br>
iex> byte_size("é")<br>
2"<br>
<br>
Given that both Erlang and elixir are implemented on top of BEAM, the wheel might not need reinventing? I know engineers and programmers love inventing stuff, and this discussion seems to point in that direction, but...<br>
</blockquote>
</div>
<div><br>
</div>
<div>All the functions in the Elixir's String module are unicode aware. There are functions for unicode normalisation (both nfd and nfc) and checking equivalency (e.g. "é" can be a single code point, or two code points - the letter "e" and the combining diacritic
acute). String.length/1 works in terms of unicode graphemes (multiple codepoints can compose into a single grapheme):</div>
<div><br>
</div>
<div>iex> String.length("é")</div>
<div>1</div>
<div><br>
</div>
<div>On 14 Jan 2017, 16:30 +0100, Benoit Chesneau <bchesneau@gmail.com>, wrote:<br>
<blockquote style="margin:5px 5px; padding-left:10px; border-left:thin solid #1abc9c" type="cite">
<div dir="ltr">someone has to write a good binding (non blocking) of icu :) I think it would be easier than reinventing the wheel. i18n [1] was a good start but looks abandoned these days. Also I disliked the load of resources at startup in ETS: <a id="LPlnk949323" href="https://github.com/erlang-unicode/i18n" previewremoved="true">https://github.com/erlang-unicode/i18n</a></div>
<div id="LPBorder_GT_14844249707730.6558399135859607" style="width: 100%; text-indent: 0px; overflow: auto; margin-bottom: 20px;">
<table id="LPContainer_14844249707700.35647078006471155" style="width: 90%; overflow: auto; padding-top: 20px; padding-bottom: 20px; margin-top: 20px; border-top-color: rgb(200, 200, 200); border-bottom-color: rgb(200, 200, 200); border-top-width: 1px; border-bottom-width: 1px; border-top-style: dotted; border-bottom-style: dotted; position: relative; background-color: rgb(255, 255, 255);" cellspacing="0">
<tbody>
<tr valign="top" style="border-spacing: 0px;">
<td id="ImageCell_14844249707710.05275600158535476" style="width: 250px; padding-right: 20px; display: table-cell; position: relative;" colspan="1">
<div id="LPImageContainer_14844249707710.3477505567191854" style="margin: auto; width: 250px; height: 250px; display: table; position: relative; background-color: rgb(255, 255, 255);">
<a id="LPImageAnchor_14844249707710.6193000523359213" style="text-align: center; display: table-cell;" href="https://github.com/erlang-unicode/i18n" target="_blank"><img width="250" height="250" id="LPThumbnailImageID_14844249707710.39832095750218355" style="border-width: 0px; width: 250px; height: 250px; vertical-align: bottom; display: inline-block; max-height: 250px; max-width: 250px;" aria-label="Preview image with link selected. Double-tap to open the link." src="https://avatars0.githubusercontent.com/u/2264078?v=3&s=400"></a></div>
</td>
<td id="TextCell_14844249707720.15141847968582311" style="padding: 0px; vertical-align: top; display: table-cell; position: relative;" colspan="2">
<div id="LPRemovePreviewContainer_14844249707720.12927996072661574"></div>
<div id="LPTitle_14844249707720.4354816599166045" style="top: 0px; color: rgb(0, 120, 215); line-height: 21px; font-family: "wf_segoe-ui_light","Segoe UI Light","Segoe WP Light","Segoe UI","Segoe WP",Tahoma,Arial,sans-serif; font-size: 21px; font-weight: 400;">
<a id="LPUrlAnchor_14844249707720.27428191937468415" style="text-decoration: none;" href="https://github.com/erlang-unicode/i18n" target="_blank">GitHub - erlang-unicode/i18n: icu nif: international ...</a></div>
<div id="LPMetadata_14844249707720.9036091056651665" style="margin: 10px 0px 16px; color: rgb(102, 102, 102); line-height: 14px; font-family: "wf_segoe-ui_normal","Segoe UI","Segoe WP",Tahoma,Arial,sans-serif; font-size: 14px; font-weight: 400;">
github.com</div>
<div id="LPDescription_14844249707730.13922640926911867" style="color: rgb(102, 102, 102); line-height: 20px; overflow: hidden; font-family: "wf_segoe-ui_normal","Segoe UI","Segoe WP",Tahoma,Arial,sans-serif; font-size: 14px; font-weight: 400; display: block; max-height: 100px;">
i18n - icu nif: international components for unicode from Erlang (unicode, date, string, number, format, locale, localization, transliteration, icu4e)</div>
</td>
</tr>
</tbody>
</table>
</div>
<br>
</blockquote>
</div>
<div><br>
</div>
<div>icu is problematic in the fact that it works mostly with UTF-16.</div>
<div><br>
</div>
<div>Michał</div>
</div>
</div>
</div>
</div>
</body>
</html>