[erlang-questions] EEP0018/native JSON parsing

[Enrico Thierbach]

Hi guys,

I have just finished what I would call the first stage of the native
JSON parser implementation. This is the state as of now at
http://github.com/pboy/eep0018/, Please see the readme file.

In short, this is the status:

- I parse everything that comes along like mochijson2 and rabbitmq
- optionally I can parse according to eep0018
- my code runs 6 times as fast as  mochijson2/rabbitmq at JSON input
of a certain size, and is usally not slower on very small JSON input.

jan tried the module along with couchdb, and find one issue regarding
UTF8 characters; besides of that everything seemed to run fine and
much faster. The utf8 parsing issue is resolved (or better: worked
around: the JSON parser and the CouchDB tests have different ideas on
what is valid UTF8).

What would be next?

1. I would like to invite you all to review and try the code.

2. I need some hints regarding "parallel execution". The native driver
does not support multithreading (and why should it? It only
complicates things where OTP can do that kind of stuff by itself
already.) With the current code the driver gets loaded only once.
Therefore on a multicore machine only one CPU gets really used. Is it
somehow possible to load the driver multiple times? The only way I see
so far is having the driver compiled and installed multiple times with
different names; but I guess there is a better way. The code itself
should luckily run in a parallel situation.

3. and finally I'll have to tackle the Erlang->JSON issue. I don't
expect a speedup as big.

Please see my next mail for some comments on EEP0018.

/eno

====================================================================
A wee piece of ruby every monday: http://1rad.wordpress.com/