[eeps] JSON

[Paul Davis]

On Mon, Jul 4, 2011 at 11:48 PM, Richard O'Keefe <ok@REDACTED> wrote:
>
> On 5/07/2011, at 1:39 PM, Alisdair Sullivan wrote:
>
>>
>> On 2011-07-04, at 5:09 PM, Richard O'Keefe wrote:
>>
>>> You cannot represent JSON numbers as strings if you want to claim "fidelity".
>>>
>>>>>> json.loads('1.2e500')
>>> inf
>>>
>>> The problem here is that we can't do that, because (at least in R14B), Erlang flatly
>>> refuses to treat inf as a number.
>>
>> You cannot represent Javascript numbers as strings if you want to claim "fidelity". You CAN represent JSON numbers as strings.
>
> You cannot represent BOTH numbers and strings as strings if you want to claim
> "fidelity".  That's the point.  I did not say that you cannot represent JSON numbers
> as something other than Erlang numbers, just that you cannot represent them as strings.
>
>> If Erlang had a decimal number type, would you insist JSON numbers still be represented as iee754 doubles? What if Erlang moves to extended precision floats? Should we not use those? JSON isn't Javascript, what Javascript is capable of is irrelevant.
>
> Remember what the name stands for: ">>>JavaScript Object Notation".
> It's the official title.
> The RFC actually says explicitly that
> "It was derived from the ECMAScript Programming Language Standard."
> and ECMA 262 is one of its "Normative References".
> The point of Normative References is that they provide information
> that can be used to fill out a standard or other specification.
>

Happily, JSON has far exceeded its original implementation and become
ubiquitous data transport. Sadly, the JS still stands for JavaScript.
Also sadly, people still argue that JSON is tied JavaScript. Its quite
obvious that the RFC 4627 was influenced by JavaScript, and that it
takes some things for granted by under specifying some hugely
important points that we're discussion at length. Unfortunately, (or
fortunately as the case may be) JSON has grown into a thing that
considering the absurdities of JavaScript in it JSON's implementation
is at best, misleading, and at worst, err, something worse that
misleading.

> Most unfortunately, RFC 4627 provides
>  *no* minimum range and
>  *no* minimum precision
> for numbers.  So we have to ask "what is needed for interoperability".
> And if we are honest for just one minute here, IEEE doubles are where
> it's at.
>
> What the RFC *does* say is that a JSON parser implementation can
> impose limits of its own choice on numbers.  Any JSON source that
> expects its receiver to cope with 40-decimal-digit floating point
> numbers is in for a sad disappointment.
>

Alisdair is more concerned than I am about number fidelity here. I
would agree in general principle that most parsers are going to mangle
numbers that are "out of the ordinary". Doing things to try and
prevent that is mostly a waste because JSON is an interchange format,
so what use is it that no other implementation speaks all the fine
details of Erlang's implementation (cause we could use term_to_binary
if we only cared about Erlang).

>>
>> The more I think about it, the more I think floats should be avoided unless explicitly requested by the user. Using floats makes JSON unsuitable for any sort of financial data interchange. That alone should be enough to bring pause to the notion that we should just do what Javascript does.
>
> Nobody in this thread has ever said that "we should just do what Javascript does",
> and the EEP does not say that.  And it has already been agreed that treating
> numbers that _look_ like integers _as_ integers is an excellent default.  Nobody
> is suggesting truncating, wrapping, rounding, or otherwise mangling integers.
>

Your earlier message was close enough that both Alisdair and I read
the same "JSON is JavaScript" argument. Earlier in this email you
cited parts of RFC 4627 in an attempt to say exactly that. Or more
pedantically, if you will, that JSON is *influenced* by JavaScript.

>> Perhaps a compromise would be to represent JSON numbers internally as a list of unicode codepoints (all valid JSON numbers formatted in this way are also valid Erlang strings) and allow the user to provide a function that formats that string in whatever manner they choose at parse time.
>
> Sigh.  Why not go the whole hog and let people plug in their own JSON parser?
> JSON is, let's face it, a pretty trivial notation.  That's the whole *point*
> of JSON, really.  The only reason for having an EEP in the first place is so
> that people without special peculiar needs could have a common efficient
> and well documented implementation to call on.
>

Is it? Or is it so that people with common needs have an efficiently
implemented solution for the common case that allows people with
particular needs to customize as they see fit?

> For what it is worth, every valid JSON number can be represented as a pair
> {Integer, Scale}, standing for Integer * 10**Scale.  There is no need to use
> textual representations.
>
> Since JSON numbers are, in all of the JSON parsers I've checked, implemented
> as IEEE doubles, and since IEEE doubles can represent integers up to
> ±9,007,199,254,740,991 with perfect accuracy for addition, subtraction,
> multiplication, remainder, and comparison, it's not clear to me that
> representing say $1,234.45 as the JSON number 12345.0 is in any way unfit
> for financial data interchange (up to 90 millions of millions of dollars).
> Actually, {"currency": "NZD", "amount": 12345.0, "scale": 100} would be
> better, of course.

The issue with money math in particular is that rounding is involved
and can do funny things. Imagine spending days trying to find a
missing penny due to a truncation or other implementation detail of
doubles. I've heard war stories and they don't sound like fun.

But in the end I mostly agree with you, people that need weird number
passing can't rely on the channel to maintain bit-level identities and
we shouldn't be expected to bend over backwards to handle that.

>
> I repeat what I've said before: given the nature of the JSON RFC, there seems
> to be no good reason why a "core" JSON library should go to heroic measures
> to support extreme numbers, especially given that this is not common practice
> in other programming languages with JSON libraries.  (I don't say that _no_
> library does this, just that it is not _common_.)
>

Indeed.

>>
> _______________________________________________
> eeps mailing list
> eeps@REDACTED
> http://erlang.org/mailman/listinfo/eeps
>