[eeps] JSON

[Richard O'Keefe]

On 5/07/2011, at 2:09 PM, Paul Davis wrote:
>> 
>> What spec and where?  RFC 4627 says that keys SHOULD be unique.
>> The author of the RFC says that it SHOULD read that they MUST be unique.
>> 
> 
> I really can't say that I care what he was thinking when he wrote it.
> He could've been thinking about squirrels playing in the tree.

You haven't answered the question.  You accused me of "twisting the spec".
What spec, where, and twisted how?
>> 
> 
> I'm not sure how you got here. The wording is quite clear that we are
> within the limits of RFC 4627 to accept and generate duplicate keys.

The wording is quite clear that we are within the limits of RFC 4627
to *accept* duplicate keys, yes.

That we are within the limits of RFCS 4627 to *generate* duplicate
keys, in a narrow, pedantic, and practically insignificant way, yes.
But the plain fact of the matter as things stand in the actual world,
if you generate a JSON object with a repeated key you have *NO* idea 
what the receiver will do with it.  It might take the first binding.
It might take the last binding.  It might take both but act in some
other ways as if it had done something else.  It might raise an
exception.  It might launch competition-buster missiles at your site.

> Arguing that Crockford wasn't thinking about squirrels and was
> thinking unique keys doesn't provide a solid argument for adopting
> this particular constraint. There may be other arguments to include
> it, but right now I don't see a clear winner amongst any of the
> possible behaviors that have been discussed other than supporting all
> of them and making it configurable.

Which has already been proposed.  Twice.

> 
>> 
>> I note that Javascript and Python accept several things that are
>> not legal according to the JSON RFC.  That's fine.  They are allowed
>> to support extensions.
>> 
>> I note also that Python's json module is perfectly happy to GENERATE
>> illegal JSON:
>>        >>> json.dumps(12)
>>        '12'
>> not legal JSON
>>        >>> json.dumps([1.2e500])
>>        '[Infinity]'
>> not legal JSON
>> 
> 
> You're missing a key qualifier.
> 
> json.dumps(12) is an illegal JSON text but is a valid JSON value. Do
> you want a parser that can't parser all JSON values?

Yes.  Of b----y course!  That's like saying "You say you want a
C compiler.  So why are you whining that the compiler accepts
'errno++;' as a legal program?  Statements are C, aren't they?"

I want a JSON parser that parses JSON values *AS PARTS OF JSON texts*
and only as parts of JSON texts.  Or more precisely, if there is also
a parser for arbitrary JSON values, I want that clearly distinguished
from the parser for JSON texts.

This *is* a weakness of EEP 18.  I completely failed to make that
distinction clear, and for that I apologise.  Clearly a revision is
needed, but I don't have time to do that today.

But if you read what I wrote before again, you'll see that I was not
criticising Python for *accepting* JSON values that are not JSON
texts but for *generating* them without warning.

>> It is OK to ACCEPT such extensions; it is not OK to GENERATE them.

>> Tell you what.  Why not fish around in some CouchDB stores and see just
>> how many repeated keys there are?
>> 
> 
> Probably very few. But seeing as the spec allows this behavior, who cares?

There is at least no doubt that the spec strongly discourages such
behaviour.  And if you don't care, why the heck are you arguing about it?

Do I have to remind readers that existing JSON parsers do *different*
things with repeated keys?  So that JSON objects with repeated keys are
*not* safe for inter-operation?  So that silently allowing them does
nobody any kindness?

>> And they will say "Then why in the name of common human decency didn't
>> you tell me LAST year when I entered the data with repeated keys?  How
>> am I supposed to fix it NOW?"
>> 
> 
> "Do it the same way you did it last year," would be the obvious
> solution I suppose.

The situation we were discussing is one where doing it the same way you
did it last year is precisely what your hypothetical victims *are* doing
and being punished for.

> 
> I prefaced this very specifically with "in my experience" because I
> was trying to say "I thought this, it is not something I would find
> terribly surprising for other people to think, perhaps we should
> change it in an attempt to avoid such confusion."

But the *only* thing in your experience that supports this wild flight
of generalisation is term_to_binary/1, which _does_ accept all terms.
Other "_to_" functions in Erlang just do not provide grounds for a
belief that "_to_" functions are were or ever should be total.
> 
> 
> I definitely agree that fixing the atom table would be even better. If
> that were to come to pass I would be whole heartedly in favor of keys
> to atoms, but until then I'm still in favor of binaries because of the
> oppurtunnity for errors, especially the ones that implicitly depend on
> system state.

I wonder if you noticed the fine print in EEP 18 which makes
turning keys to binaries the default?