[erlang-questions] No JSON/MAPS interoperability in 17.0?

Fri Mar 14 15:40:36 CET 2014

On Fri, Mar 14, 2014 at 12:55 PM, Joe Armstrong <erlang@REDACTED> wrote:

>
>
>
> On Fri, Mar 14, 2014 at 9:38 AM, Pieter Hintjens <ph@REDACTED> wrote:
>
>> When you judge an encoding like JSON you're weighing simplicity of use
>> against efficiency. For messages you send a few times a second, at
>> most, text encoding costs are irrelevant. The big advantage of JSON,
>> like XML and YAML, is the ability to extend contracts experimentally
>> and cheaply. In comparison, any binary format raises steep costs for
>> change, creating absurd incompatibility that has no benefit. The
>> OpenStack RPC API being a good example of this.
>>
>> One answer I've used with success is the "Cheap and Nasty" pattern,
>> where slow-changing high-volume data uses a binary encoding, and
>> fast-changing low-volume data uses JSON, XML, or similar. By forcing
>> protocols into these extremes you can win both games. Using a single
>> approach for the two cases creates mediocrity, in my experience. Thus
>> mspack, protobufs, etc. are poor choices either way.
>>
>
>
> Good point.
>
> I think people "understand" JSON on two levels - at one level of
> understanding
> JSON is "easy" - in this mindset encodings etc. multiple keys etc are
> irrelevant.
>
> To an implementor of a library JSON is complicated.
> When I think of *implementing* JSON<->Erlang maps My hair goes even greyer.
>
> I think:
>
>      1) does a floating point number survive a round-trip from erlang ->
> JSON -> Erlang
>           is it "bit identical" after the round trip? - why does this
> matter? - well it
>          might just matter - I might want to cache the result or compute a
> SHA1 from it.
>       2) Do I have to worry about filling the erlang atom table?
>       3) Do I have to handle "infinite"JSON streams (for some meaning of
> "infinite")
>       4) Do want parse trees, or a SAX like interface
>       5) Do I want pure erlang or a NIF (the NIF might crash erlang)
>
> Now suppose there are two answers to each question. That's 2^5
> implementations.
>
> That's why there are multiple libraries - they make different design
> choices.
>
> It's also extremely confusing to a beginner since the different design
> choices
> are not clearly stated - I find the choice of libraries difficult for just
> this reason,
> they all appear to do the same thing.
>
> A user of a JSON library would by now be crying out loud - "just give me
> a ****ing library - all I want to do is parse this JSON that I have in a
> file.
>
> This is what most libraries do - they work 99% of the time.
>
> (( the 99% is horrible - just when I think a library is great I find -
> *but I can't do X* then
>     I'm in trouble - but I guess this is in the nature of the beast - only
> pure mathematical
>     functions have the kind of "platonic interfaces" that will never
> change - real
>     world problems with time and space involved are just plain messy ))
>
> If we freeze the JSON design and say:
>
>     1) floating point number will not survive a round-trip
>     2) JSON tags will be erlang atoms
>     3) Terms are not-infinite and rather small (few MB at max)
>     4) we want a parse tree
>     5) best effort at sorting out character set encodings
>     6) Pure erlang
>
> Then the problem becomes "easy" and can be knocked out in a few lies of
> code
> using maps:from_list and maps:to_list.
>
> And *because* it's easy nobody writes a library to do this.
>
> The trouble is a beginner probably does not find this easy and would
> appreciate
> a library that worked 99% of the time on simple problems.
>
> I watched DaveTomas and Jose Valim's Erlang factory lecture where
> they criticised Erlang for not having simple ways of doing common things.
>
> I think they were right.
>
> I think we to make some simple ways to solve common problems.
>
> The problem is simple things are very difficult to invent.
>
> I think we need a "otp subset" and "library subset" that is designed for
> beginners.
>
>
I think having something like OTP but focused around web technologies (http
client+server, websocket, spdy, json, database connectors, email sending),
would be very useful to help.

>
> Cheers
>
> /Joe
>
>
>>
>> What this comes to is, often, a protocol message split into binary
>> fields that are extremely tuned and fast to parse, plus optionally, an
>> extensible hash or JSON document or similar.
>>
>> We've built a toolkit for this, zeromq/zproto, which is simple and
>> efficient.
>>
>> Loic is right that lack of URLs in JSON makes it harder to use for
>> RESTful work, however that's trivially solved by defining URLs
>> explicitly. Then you can do very nice RESTful work without caring what
>> encoding you use.
>>
>> E.g. https://github.com/UnifyProject/RFC/blob/master/draft/rfc-2.md
>>
>> tl;dr use the best tool for the job, no single encoding can solve all use
>> cases.
>>
>> -Pieter
>>
>>
>> On Fri, Mar 14, 2014 at 8:56 AM, Benoit Chesneau <bchesneau@REDACTED>
>> wrote:
>> > So I am curious in what could replace json today as an interoperable
>> > exchange data format. Message pack is interesting but not sure it's a
>> > good one (no type support). Any idea?
>> >
>> > - benoit
>> >
>> > On Fri, Mar 14, 2014 at 12:57 AM, Anthony Ramine <n.oxyde@REDACTED>
>> wrote:
>> >> Where is the scientific study that proves this?
>> >>
>> >> Or is that just your personal experience?
>> >>
>> >> --
>> >> Anthony Ramine
>> >>
>> >> Le 13 mars 2014 à 21:16, Carsten Bormann <cabo@REDACTED> a écrit :
>> >>
>> >>> On 10 Mar 2014, at 15:54, Loïc Hoguin <essen@REDACTED> wrote:
>> >>>
>> >>>> Could you elaborate on what the dominant usage of JSON is?
>> >>>
>> >>> Certainly.
>> >>>
>> >>> JSON is used for data interchange between applications or within
>> components of an application.
>> >>>
>> >>> The most extensive use is from JavaScript-based Web application
>> front-ends (running in the browser) to their backends (running on a server)
>> and back; these are shipped as application/json representations in AJAX
>> accesses (HTTP access from JavaScript via the confusingly named
>> XMLHttpRequest object).
>> >>>
>> >>> Many Web APIs that have been designed in the last half decade are
>> described in JSON form, so applications that access another application via
>> a Web API are likely to ship back and forth application/json
>> representations via HTTP accesses.
>> >>>
>> >>> Grüße, Carsten
>> >>>
>> >>
>> >> _______________________________________________
>> >> erlang-questions mailing list
>> >> erlang-questions@REDACTED
>> >> http://erlang.org/mailman/listinfo/erlang-questions
>> > _______________________________________________
>> > erlang-questions mailing list
>> > erlang-questions@REDACTED
>> > http://erlang.org/mailman/listinfo/erlang-questions
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20140314/dff119fe/attachment.htm>