Mnesia overloaded

[Scott Lystig Fritchie]

>>>>> "ma" == Magnus Ahltorp <map@REDACTED> writes:

ma> [...] but this just hangs and I get these messages:

ma> Mnesia('incomingproxy@REDACTED'): ** WARNING ** Mnesia is
ma> overloaded: {dump_log, time_threshold}

I don't know about the "just hangs" part, but I believe that they are
(mostly) harmless.  Quoting Ulf Wiger from Message-ID
<D4ADC896EC3F5948A9D3899986CEB9DE03C7864B@REDACTED>
on Tue, 9 May 2006 21:35:16 +0200:

uw> The warning about mnesia being overloaded
uw> mean that a dump of the transaction log
uw> has been triggered before the previous 
uw> transaction log dump has been completed.
uw> 
uw> That's ok. Mnesia can handle it, but it is
uw> an indication that mnesia is temporarily 
uw> backlogged.

I said "(mostly)" because I've encountered a situation where being
backlogged for long periods of time can be A Bad Thing.  If Mnesia is
so busy that it cannot periodically truncate its transaction log, then
that log file (the "LATEST.LOG" file in the Mnesia data directory, if
I'm not mistaken) will continue to grow until all available disk space
is consumed.

I wrote a simple Erlang process that periodically checks how big that
file is ... and if it gets too big, then it adds an artificial
throttle: other parts of the application start calling timer:sleep/1.
The resulting reduction of DB transactions gives Mnesia enough time to
catch up.  When the size of "LATEST.LOG" falls below the high water
mark, the calls to timer:sleep/1 stop, and transaction processing
resumes at full speed.

-Scott