[erlang-questions] EEP0018/native JSON parsing

[Enrico Thierbach]

>
> >
> >>>2, I actually hacked yajl a bit to decode utf-8 data into the same
> buffer
> >>> it's reading from
> > specifically to avoid any memcpy stuff. The downside is that this
> > destroys the original binary term.
> >
> > I wonder if that is safe on the data you get from control().  The docs
> don't
> > seem to say.   The pointer is a char* not a const char*, but that might
> > simply be oversight.
> >
>
> Absolutely no idea. From the stuff I read there was very little of the
> over view type of info that can be important.
>

I bet there is something bad about destroying that content. This is erlnag
after all, so each object should be immutable once created.

>
> >
> >>>With some more hacking on yajl, it's possible that we could change
> around
> >>> the
> > original unicode buffer stuff to decode all unicode strings into a
> > single buffer and then reference that buffer etc.
> >
> > Yes you could allocate a single driver binary of conservative size,
> > incrementally fill it with data (using sub-binary references in the
> > constructed term) and then realloc the binary to the final size (if it is
> > too large) at the end.   Good idea.    I thought I read somewhere that
> Yajl
> > will only copy strings that have unicode characters, so this method would
> > presumably only copy those strings.  All ascii/latin1 strings (most I
> would
> > imagine) wouldn't need to be copied (if outputv were used).
> >
>
Before implementing anything complicated I would suggest to return the data
from within the originating binary if yajl didn't have to convert anything
(which it doesn't do if the input is UTF-8), and only in those other cases
to make a new binary. The erl-driver doc says something about
a BINARY_FROM_BUFFER thingy and suggests that as pretty fast. I would like
to see how well that would work.

>>>I hadn't heard of EDTK prior to now, I'll take a look in the near future.
>
> It's the Erlang Driver Toolkit.

It would b nice to have known that before :) From what I read here it seems
a bit overcomplicated for what we want to achive, I would say.


/eno


-- 
====================================================================
A wee piece of ruby every monday: http://1rad.wordpress.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20090127/9de2e560/attachment.htm>