<html><body><div style="color:#000; background-color:#fff; font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:10pt"><div><span><br></span></div><div style="color: rgb(0, 0, 0); font-size: 13px; font-family: HelveticaNeue, 'Helvetica Neue', Helvetica, Arial, 'Lucida Grande', sans-serif; background-color: transparent; font-style: normal;"><span>Hey OTPeople,</span></div><div style="color: rgb(0, 0, 0); font-size: 13px; font-family: HelveticaNeue, 'Helvetica Neue', Helvetica, Arial, 'Lucida Grande', sans-serif; background-color: transparent; font-style: normal;"><span><br></span></div><div style="color: rgb(0, 0, 0); font-size: 13px; font-family: HelveticaNeue, 'Helvetica Neue', Helvetica, Arial, 'Lucida Grande', sans-serif; background-color: transparent; font-style: normal;"><span>Since this comes up every now and then, does Ericsson have any plan to introduce atom garbage collection or suchlike in Erlang/OTP?
Some official info would be great.</span></div><div style="color: rgb(0, 0, 0); font-size: 13px; font-family: HelveticaNeue, 'Helvetica Neue', Helvetica, Arial, 'Lucida Grande', sans-serif; background-color: transparent; font-style: normal;"><span><br></span></div><div style="color: rgb(0, 0, 0); font-size: 13px; font-family: HelveticaNeue, 'Helvetica Neue', Helvetica, Arial, 'Lucida Grande', sans-serif; background-color: transparent; font-style: normal;"><span>Best,</span></div><div style="color: rgb(0, 0, 0); font-size: 13px; font-family: HelveticaNeue, 'Helvetica Neue', Helvetica, Arial, 'Lucida Grande', sans-serif; background-color: transparent; font-style: normal;"><span>Thomas</span></div><div style="color: rgb(0, 0, 0); font-size: 13px; font-family: HelveticaNeue, 'Helvetica Neue', Helvetica, Arial, 'Lucida Grande', sans-serif; background-color: transparent; font-style: normal;"><span><br></span></div><div class="yahoo_quoted" style="display:
block;"> <br> <br> <div style="font-family: HelveticaNeue, 'Helvetica Neue', Helvetica, Arial, 'Lucida Grande', sans-serif; font-size: 10pt;"> <div style="font-family: HelveticaNeue, 'Helvetica Neue', Helvetica, Arial, 'Lucida Grande', sans-serif; font-size: 12pt;"> <div dir="ltr"> <font size="2" face="Arial"> On Friday, March 14, 2014 4:20 PM, Fred Hebert <mononcqc@ferd.ca> wrote:<br> </font> </div> <blockquote style="border-left: 2px solid rgb(16, 16, 255); margin-left: 5px; margin-top: 5px; padding-left: 5px;"> <div class="y_msg_container">Hi Joe,<br clear="none"><br clear="none">I think the library you have described (does 99% of the work) is the<br clear="none">equivalent of bait-and-switch at the language/library level:<br clear="none"><br clear="none">1) it's not just a question of do floating point numbers survive the<br clear="none"> roadtrip, but also integers.<br clear="none">2) json tags are more or less strings and expect
utf-8. Currently, we<br clear="none"> 'support' utf8 atoms but we don't. See<br clear="none"> http://www.erlang.org/erldoc?q=list_to_atom.<br clear="none"> This doesn't mention what you do in case of trying to encode a map<br clear="none"> which currently uses keys such as '1.0', 1.0, <<"1.0">>, and "1.0" at<br clear="none"> the same time. We currently have 4 data types that will possibly need<br clear="none"> an identical representation while being converted.<br clear="none"><br clear="none"> Woops, that doesn't work super well and may in fact cover far less<br clear="none"> than 99% of the cases. We have to consider all the other cases such<br clear="none"> as just 1, 1.0, "1.00", "1.000", ..., and so on.<br clear="none"><br clear="none"><br clear="none">3) That can be made to work<br clear="none">4) No opinion on this one<br clear="none">5) This can also be read as "best
effort not corrupting values read"<br clear="none"> which scares me a lot if the end result is not "raise an error when you can't<br clear="none"> figure it out reliably"<br clear="none">6) Amen to that.<br clear="none"><br clear="none">This doesn't even take into account the issue that by using atoms by<br clear="none">default, you're actively using a source of memory leaks into the<br clear="none">library. This guarantees that every tutorial out there will recommend<br clear="none">not using the standard library for your tasks.<br clear="none"><br clear="none">What I'm getting at here is that while your scheme might work for 99% of<br clear="none">possible JSON -> Erlang decodings, it will do so in a risky way that<br clear="none">cannot be advocated.<br clear="none"><br clear="none">If you consider all the possible Erlang -> JSON mappings (and this is<br clear="none">where the biggest problem always is), then it covers far, far
less than<br clear="none">99% of the cases, because there is not one good way to do it (how do you<br clear="none">even represent binaries that are not UTF-8 and distinguish them from<br clear="none">actual strings? You validate the entire thing unless you want to create<br clear="none">unparseable JSON).<br clear="none"><br clear="none">I used the words bait-and-switch and I mean it there. This is one of the<br clear="none">points where Jose Valim and I disagree the most.<br clear="none"><br clear="none">I hate, absolutely hate solutions that bring you 70% of the way (to use<br clear="none">a number we discussed between ourselves). Why? Because you start by<br clear="none">learning the 70% solution and then it doesn't work. Suddenly you have to<br clear="none">go out and look for the 99% and the 100% solutions out there.<br clear="none"><br clear="none">Except you now have a crusty code base full of legacy, and you start<br clear="none">supporting,
one, two libraries, which you never envisioned. And you find<br clear="none">out you're married to encoders and decoders that decided to do things<br clear="none">differently, but you just don't have the time to fix everything right<br clear="none">now.<br clear="none"><br clear="none">You start seeing node crashes because someone decided atoms is a good<br clear="none">way to receive unvalidated data without first having implemented a<br clear="none">scheme to GC them (say EEP-20:<br clear="none"><a shape="rect" href="http://www.erlang.org/eeps/eep-0020.html" target="_blank">http://www.erlang.org/eeps/eep-0020.html</a>).<br clear="none"><br clear="none">You start being pretty angry that a language so focused on getting<br clear="none">productions systems live has a standard library that lets you hang dry.<br clear="none"><br clear="none">Then you get even angrier when you figure out a crapload of frameworks<br clear="none">in the wild all use that
faulty function because it was in the standard<br clear="none">library.<br clear="none"><br clear="none">Then you get certainly extremely angry and leave for Go or some other<br clear="none">language when you figure out the community already had solutions to<br clear="none">these problems in the year 2014 (and before!) but they were just<br clear="none">overlooked because we wanted an easy JSON implementation in the stdlib.<br clear="none"><br clear="none">I can't for the life of me see the benefit of canonizing a bad library<br clear="none">when tradeoffs are already known and worked around in the wild.<br clear="none"><br clear="none">What we should focus on is explaining these tradeoffs and making it easy<br clear="none">to show the different options. Currently, picking a JSON lib is hard<br clear="none">because there is such a very poor match between what you can possibly<br clear="none">encode in Erlang and how you can translate this back and forth
with<br clear="none">JSON. Not just because it's not in the standard library.<br clear="none"><br clear="none">Not speaking about the problem doesn't make it go away, it makes it more<br clear="none">surprising, which is not a desirable property.<br clear="none"><br clear="none">Regards,<br clear="none">Fred.<br clear="none"><br clear="none">On 03/14, Joe Armstrong wrote:<div class="yqt6294578441" id="yqtfd77966"><br clear="none">> This is what most libraries do - they work 99% of the time.<br clear="none">> <br clear="none">> (( the 99% is horrible - just when I think a library is great I find - *but<br clear="none">> I can't do X* then<br clear="none">> I'm in trouble - but I guess this is in the nature of the beast - only<br clear="none">> pure mathematical<br clear="none">> functions have the kind of "platonic interfaces" that will never change<br clear="none">> - real<br clear="none">>
world problems with time and space involved are just plain messy ))<br clear="none">> <br clear="none">> If we freeze the JSON design and say:<br clear="none">> <br clear="none">> 1) floating point number will not survive a round-trip<br clear="none">> 2) JSON tags will be erlang atoms<br clear="none">> 3) Terms are not-infinite and rather small (few MB at max)<br clear="none">> 4) we want a parse tree<br clear="none">> 5) best effort at sorting out character set encodings<br clear="none">> 6) Pure erlang<br clear="none">> <br clear="none">> Then the problem becomes "easy" and can be knocked out in a few lies of code<br clear="none">> using maps:from_list and maps:to_list.<br clear="none">> <br clear="none">> And *because* it's easy nobody writes a library to do this.<br clear="none">> <br clear="none">> The trouble
is a beginner probably does not find this easy and would<br clear="none">> appreciate<br clear="none">> a library that worked 99% of the time on simple problems.<br clear="none">> <br clear="none">> /Joe<br clear="none">> <br clear="none"><br clear="none">_______________________________________________<br clear="none">erlang-questions mailing list<br clear="none"><a shape="rect" ymailto="mailto:erlang-questions@erlang.org" href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a><br clear="none"><a shape="rect" href="http://erlang.org/mailman/listinfo/erlang-questions" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a><br clear="none"></div><br><br></div> </blockquote> </div> </div> </div> </div></body></html>