[erlang-questions] io_lib and unicode post processing?

[Tim Watson]

On 23 Aug 2010, at 18:37, Musumeci, Antonio S wrote:

> I've got a webmachine/mochiweb setup and it seems much slower/higher CPU than I expected. I'm creating JSON for 10k records using jsonerl and sending on to webmachine. After the serializing of the data the CPU is still pegged and I see in etop io_lib and unicode being called. Does anyone know why those would be called? It takes several seconds.

Surely they're getting called a lot during serialisation. Haven't used jsonerl before but what kind of output is getting produced - strings, iolists, binaries? Does it not offer a streaming API so you can forward the data as multipart/chunked? There are some useful profiling tools available with OTP and there's a good library that'll allow you use them on production systems without incurring dangerous overhead - eper. You'll find it on google code IIRC. I've been using a newer json library called jsx recently (http://github.com/talentdeficit/jsx) and find it performs really well, although I haven't load tested it so don't know if it's production worthy.