[erlang-questions] Mnesia record count

[Maarten Engelen]

2007/11/27, Ludovic Coquelle <lcoquelle@REDACTED>:
>
>
> On 11/27/07, Maarten Engelen <maarten@REDACTED> wrote:
> > [...]
> > It is not so much memory consumption as a speed consideration. The
> > scale of the application is that it needs to handle about 5000
> > concurrent connections that do around 1000 requests per second on the
> > system. Each of these requests needs to check the size of a subset of
> > a table with about 50000 rows. If mnesia is fast enough to handle it
> > by reading in the complete subset of a few thousand rows and checking
> > it's length every time, I'm fine with that.
>
>  1000 requests coming from 5000 concurrent connections ... that's nice!
> In order to have an idea of what kind of load mnesia can support, could you
> give some precision about the kind of requests (proportion of read/write,
> complexity of the request), and also how many erlang nodes are used? ... no
> need for statistic measures or benchmark, just enough to satisfy the
> curiosity of the readers :). Thanks.

Just to be on the safe side: The total is 1000 requests coming from
5000 connections. Not 1000 requests per second on each of the 5000
connections. ;)

Currently I'm in a rewrite of the software to Erlang. This started as
a proof of concept and is on its way to become the new software used
in production. The current system is written in c, but introducing new
features is very difficult next to the fact that it doesn't scale very
well.
It is used to support the authentication and monitoring of about 25000
concurrent users. It has to support about 40 frontend systems in the
near future with around 1000-2000 connections per system. Each user
connection generates 2-3 requests per minute on the backend system
we're writing.

We haven't been testing it on the production load yet, but it looks
like this system can be supported with two multicore systems each
running one erlang node and a distributed mnesia database which
handles all configuration and session data.

The requests are pretty simple. Just boundary checking and updating
usage counters in the mnesia database. The proportion between
reads/writes will be around 3/1.

Nevertheless, this sytem still has to prove itself. I'm using mnesia
based on the reading I did through the archives where I saw mnesia
setups mentioned with a lot more data and comparable load.

Maarten.