[erlang-questions] EEP18 Considered Harmful, With Examples: An opinion rejecting EEP 18

[John Haugeland]

I apologize for not moving faster.  I thought I had more time.  It is my
strong belief that EEP 18 is for many reasons critically flawed, and must
not be introduced into the language as BIFs.  I have been working on a
contrary approach library, called scjson, because I was going to submit an
EEP when I had actually implemented something and thus understood the
various problems involved.  Erlang and JSON offer different fundamental
datatypes, and attempting to represent one as the other is significantly
dangerous.

This should also serve as an exposition of my opinion that EEPs should not
be accepted prior to an existing implementation.  An implementation of EEP18
would have exposed what I believe are serious flaws in its design.  It is my
belief that things like this should _always_ be a module first, if not just
permanently - we are able to replace, for example, the regular expression
system without violating backwards compatability because it is a module.  If
this was being done as a module, then the other JSON author and I could
engage in healthy competition the way that YAWS does with Mochiweb.
Introducing this into the language takes the early position that the first
approach we've considered is correct enough to be the permanent,
irreplaceable default.  I suspect many here would agree that it's a good
thing that regular expressions were not approached as BIFs.

At this time I have not yet sufficiently documented my beliefs regarding the
problems with 18.  However, here is a quick overview:

1) http://sc.tri-bit.com/outgoing/scjson%20parser%20halp.html
2)
http://fullof.bs/prototype-eep-18-considered-harmful-the-problems-with-erlang-to-json-term-translation

There is one primary branch of concern which I have not yet come to in
either of those documents: JSON explicitly requires, and frequently relies
on in the wild, unicode support.  Unicode in turn requires UTF-8 support.
There is a way to attack the parser halp document by artificially
restricting all strings to being binary strings, and assuming all lists to
not be strings ever (something which would have painful consequences to
application design and interoperability).  Taking this approach means that a
supporting Erlang implementation would have to support UTF-8, with its
varying width characters, in binaries, which would have a significant impact
on available efficient approaches to string parsing.  It is a performance
nightmare in the making.  Artificially discarding Erlang's primary string
type to resolve some of the ambiguities makes the problem worse, not
better.

Unfortunately I have not had time to add that (or several other serious
concerns) to the document.  That one could undermine those documents, so I'm
getting it out now; the rest will come with examples, since they're
standalone concerns.

I have been creating a library which uses an annotated representation which
has an optional lossy downconversion, rather than pursuing what I see as the
folly of native re-representation, but since I've been focussing on
documenting scutil, it looks like I've fallen behind.  The lossy
downconversion is equally convenient to proposed BIFs, offers several
configurability advantages, and most importantly, does not impede the
unambiguous representation of every JSON term in full unicode.

Please very seriously consider those two documents before proceeding with
making EEP18 a permanent part of the language.  It is quite possible that
I'm wrong; if so I will of course apologize.  However, I believe there are
serious, dangerous, unnecessary issues here which can be resolved with a
simple annotated representation, rather than trying to present JSON as
native erlang terms.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20081206/7a8b2f2f/attachment.htm>