[erlang-questions] why is mmap so darn difficult
Wed Jun 12 20:47:29 CEST 2013
mmap is faster than file when you are reading from vfs cache. It is
faster on, lets say 5%, or even 15%.
But speed of microbenchmarks is nothing when you run real service. You
need to make your software controllable.
For example, erlyvideo makes gen_server:call to a special queue before
each disk read. It is slower, but it makes server much more stable.
It makes many other different checks like process_info(Pid,
message_queue_len) to make things more stable and server behaviour
If you chase for speed of microbenchmarks, you can build a server that
runs excellently on your laptop, but begins to block on production.
This is my lesson from my experience.
On Wed, Jun 12, 2013 at 9:44 PM, Tim Watson <watson.timothy@REDACTED> wrote:
> Hi Max,
> On 12 Jun 2013, at 10:43, Max Lapshin wrote:
>> Of course, you will not be able to handle such load with mmap, because
>> you will suffer from blocking whole VM on mmap page faults.
> Yes of course you're right about that, with mmap running in a NIF. I attempted to use both emmap and the mmap driver you wrote for erlyvideo and found the resulting behaviour highly variable.
>> Add more async threads and profile application. Don't waste your time on mmap.
> I'm starting to wonder whether attempting to provide very high speed concurrent reads + writes in Erlang is going to work at all. I might spend some time looking at lmdb instead, since it claims some extremely high numbers. I was hoping to avoid writing a driver though, but ho hum...
More information about the erlang-questions