[erlang-questions] Speed of CSV parsing: how to read 1M of lines in 1 second

[james]

I think you should be able to parse something like this at approaching 
the speed of memcpy, but you have to try hard.  Have you tried writing 
something that runs outside erlang and just tries to parse the file into 
numbers in memory, say using re2c or ragel?

You might need to try using mmap and a thread that just reads every 16th 
byte and forces the file to read in.  Or double buffer with readahead.

Is the data in your operating system's VM cache already, or on disk?

James