[erlang-questions] No JSON/MAPS interoperability in 17.0?

Joe Armstrong erlang@REDACTED
Fri Mar 14 12:55:45 CET 2014 

On Fri, Mar 14, 2014 at 9:38 AM, Pieter Hintjens <ph@REDACTED> wrote:

> When you judge an encoding like JSON you're weighing simplicity of use
> against efficiency. For messages you send a few times a second, at
> most, text encoding costs are irrelevant. The big advantage of JSON,
> like XML and YAML, is the ability to extend contracts experimentally
> and cheaply. In comparison, any binary format raises steep costs for
> change, creating absurd incompatibility that has no benefit. The
> OpenStack RPC API being a good example of this.
>
> One answer I've used with success is the "Cheap and Nasty" pattern,
> where slow-changing high-volume data uses a binary encoding, and
> fast-changing low-volume data uses JSON, XML, or similar. By forcing
> protocols into these extremes you can win both games. Using a single
> approach for the two cases creates mediocrity, in my experience. Thus
> mspack, protobufs, etc. are poor choices either way.
>


Good point.

I think people "understand" JSON on two levels - at one level of
understanding
JSON is "easy" - in this mindset encodings etc. multiple keys etc are
irrelevant.

To an implementor of a library JSON is complicated.
When I think of *implementing* JSON<->Erlang maps My hair goes even greyer.

I think:

     1) does a floating point number survive a round-trip from erlang ->
JSON -> Erlang
          is it "bit identical" after the round trip? - why does this
matter? - well it
         might just matter - I might want to cache the result or compute a
SHA1 from it.
      2) Do I have to worry about filling the erlang atom table?
      3) Do I have to handle "infinite"JSON streams (for some meaning of
"infinite")
      4) Do want parse trees, or a SAX like interface
      5) Do I want pure erlang or a NIF (the NIF might crash erlang)

Now suppose there are two answers to each question. That's 2^5
implementations.

That's why there are multiple libraries - they make different design
choices.

It's also extremely confusing to a beginner since the different design
choices
are not clearly stated - I find the choice of libraries difficult for just
this reason,
they all appear to do the same thing.

A user of a JSON library would by now be crying out loud - "just give me
a ****ing library - all I want to do is parse this JSON that I have in a
file.

This is what most libraries do - they work 99% of the time.

(( the 99% is horrible - just when I think a library is great I find - *but
I can't do X* then
    I'm in trouble - but I guess this is in the nature of the beast - only
pure mathematical
    functions have the kind of "platonic interfaces" that will never change
- real
    world problems with time and space involved are just plain messy ))

If we freeze the JSON design and say:

    1) floating point number will not survive a round-trip
    2) JSON tags will be erlang atoms
    3) Terms are not-infinite and rather small (few MB at max)
    4) we want a parse tree
    5) best effort at sorting out character set encodings
    6) Pure erlang

Then the problem becomes "easy" and can be knocked out in a few lies of code
using maps:from_list and maps:to_list.

And *because* it's easy nobody writes a library to do this.

The trouble is a beginner probably does not find this easy and would
appreciate
a library that worked 99% of the time on simple problems.

I watched DaveTomas and Jose Valim's Erlang factory lecture where
they criticised Erlang for not having simple ways of doing common things.

I think they were right.

I think we to make some simple ways to solve common problems.

The problem is simple things are very difficult to invent.

I think we need a "otp subset" and "library subset" that is designed for
beginners.


Cheers

/Joe


>
> What this comes to is, often, a protocol message split into binary
> fields that are extremely tuned and fast to parse, plus optionally, an
> extensible hash or JSON document or similar.
>
> We've built a toolkit for this, zeromq/zproto, which is simple and
> efficient.
>
> Loic is right that lack of URLs in JSON makes it harder to use for
> RESTful work, however that's trivially solved by defining URLs
> explicitly. Then you can do very nice RESTful work without caring what
> encoding you use.
>
> E.g. https://github.com/UnifyProject/RFC/blob/master/draft/rfc-2.md
>
> tl;dr use the best tool for the job, no single encoding can solve all use
> cases.
>
> -Pieter
>
>
> On Fri, Mar 14, 2014 at 8:56 AM, Benoit Chesneau <bchesneau@REDACTED>
> wrote:
> > So I am curious in what could replace json today as an interoperable
> > exchange data format. Message pack is interesting but not sure it's a
> > good one (no type support). Any idea?
> >
> > - benoit
> >
> > On Fri, Mar 14, 2014 at 12:57 AM, Anthony Ramine <n.oxyde@REDACTED>
> wrote:
> >> Where is the scientific study that proves this?
> >>
> >> Or is that just your personal experience?
> >>
> >> --
> >> Anthony Ramine
> >>
> >> Le 13 mars 2014 à 21:16, Carsten Bormann <cabo@REDACTED> a écrit :
> >>
> >>> On 10 Mar 2014, at 15:54, Loïc Hoguin <essen@REDACTED> wrote:
> >>>
> >>>> Could you elaborate on what the dominant usage of JSON is?
> >>>
> >>> Certainly.
> >>>
> >>> JSON is used for data interchange between applications or within
> components of an application.
> >>>
> >>> The most extensive use is from JavaScript-based Web application
> front-ends (running in the browser) to their backends (running on a server)
> and back; these are shipped as application/json representations in AJAX
> accesses (HTTP access from JavaScript via the confusingly named
> XMLHttpRequest object).
> >>>
> >>> Many Web APIs that have been designed in the last half decade are
> described in JSON form, so applications that access another application via
> a Web API are likely to ship back and forth application/json
> representations via HTTP accesses.
> >>>
> >>> Grüße, Carsten
> >>>
> >>
> >> _______________________________________________
> >> erlang-questions mailing list
> >> erlang-questions@REDACTED
> >> http://erlang.org/mailman/listinfo/erlang-questions
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://erlang.org/mailman/listinfo/erlang-questions
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20140314/09264581/attachment.htm>

More information about the erlang-questions mailing list