[erlang-questions] Mnesia ets usage oddities.

[Sverker Eriksson]

The default maximum number of ets tables is 2053.

I don't know the rationale behind this magic number but is has been like 
this at least since R13B03 where my git history ends.
This is also the minimum limit, i.e you can not force a lower limit than 
this.

So, the documentation is wrong or at least misleading when it says 
"approximately 1400 tables".

/Sverker, Erlang/OTP


On 04/21/2014 08:40 PM, Charles Zilm wrote:
> All,
>
> I'm currently debugging a system that uses mnesia as a data cache layer to
> store the state of client processes. Each client process represents a
> single TCP connection to the application, there may be thousands of
> connections. When a connection is lost the client process is alerted and
> the mnesia cache is updated. Similarly when a new connection is made a
> client process is spawned and the mnesia cache is updated to reflect the
> new connection.
>
> Testing has shown that losing thousands of connections at once can cause
> the application to crash due to ets table exhaustion. This makes perfect
> sense, thousands of transactions being spawned in short order is likely to
> exhaust the pool. What does not make sense is that the transactions spawned
> by the client processes never trigger the system_limit error. That is
> always triggered by a separate timer triggered transaction that just
> happens to run while the client process transactions are processing.
>
> Tracing calls to ets:new and printing out length(ets:all()) after each call
> revealed that prior to crash our system reports using more ets tables than
> is should be allowed to.
>
> So a few questions,
>
> 1) How does ets determine if it can create a new table? length(ets:all())
> returning 1800 when the system limit is 1400 is confusing.
>
> 2) Why could it be that only the timer triggered transaction crashes due to
> ets table exhaustion? With the number of terminating connections I'd expect
> to see a number of client processes terminating for this same reason. Log
> verification shows all client processes terminated with the expected reason.
>
> Any insights would be most appreciated.
>
> Thanks,
>
> Charles
>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20140422/b4e6cb3d/attachment.htm>