[erlang-questions] json in erlang

[Paul Davis]

On Wed, Jun 29, 2011 at 11:12 PM, Wes James <comptekki@REDACTED> wrote:
> I see this:
>
> http://www.erlang.org/eeps/eep-0018.html
>
> but it doesn't exist in erlang proper (it seems).  Does that mean the
> eep hasn't been accepted and won't be implemented?  What does the S ->
> standards track eep mean?
>
> just wondering.
>
> thx,
>
> -wes
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>

Apologies for missing this for so long. But a brief history of JSON as
I've seen it over the last couple years.

Originally in CouchDB we used mochijson2 and that was fine. There were
also a few other pure erlang modules around this time as well,
specifically I think RabbitMQ still uses something that isn't
mochijson2. Though I haven't checked lately.

Through various benchmarks of CouchDB we started seeing it popping up
as a place in the code that was taking a significant amount of time.
Around the end of 2008 me and another guy (who's name escapes me at
the moment) started working on an implementation of eep0018 as a
linked in port driver. Shortly-ish after we had a mostly working
version of this, NIF's came along and I switched to using those for
eep0018. There have been a number of people using it and I've fixed a
couple bugs but its been mostly stable since that update.

Recently in CouchDB land there was a big push to finally move to a NIF
based JSON parser. To make this happen we had to go through and make
sure that the behavior was much closer to mochijson2 than eep0018 was
originally. Specifically large number support and some UTF-8 behaviors
needed to be changed. Filipe Manana and Damien Katz did this update to
minimize behavior changes. The fork on Benoît's GitHub account
(referenced in this thread) appears to have two line patch applied to
this second version.

I make a note of the second version because in order to match the
mochijson2 semantics they ended making a more hybrid solution for the
parser which slows it down a bit. When I was looking at this I decided
that I'd spent enough time around the JSON spec to have a decent
chance at writing a parser that is much quicker that can minimize the
amount of effort needed in Erlang to make things as fast as possible.
The new project is called Jiffy [1] and should be a completely (with
caveats [2]) working parser. In my tests its also noticeably faster
than the newer eep0018 and mochijson2, but only slightly faster or
tied with the older eep0018 (though still matches mochijson2 better).

Also I should point out that when I say "implementation of eep0018," a
more apt description would be "an implementation of mochijson2 in C".
For the most part I think most people have abandoned most of the
points in eep0018 that aren't captured in mochijson2. For the stuff in
eep0018 that isn't in one of the current parsers I'd suggest just
adding them as a library on top of a JSON encoder/decoder.

Also, my focus for JSON encoding/decoding has always been on making it
very fast to process smallish bits of data. If you need a library that
will allow you to stream parsing events for a large JSON stream I
would highly recommend Alisdair Sullivan's jsx library [3].

This got a bit longer than I expected so I'm gonna cut myself off
before I ramble even further. If you have any questions feel free to
ask.

[1] https://github.com/davisp/jiffy
[2] I only support a strict implementation of UTF-8.
[3] https://github.com/talentdeficit/jsx