Mnesia Overloading Problem (contd.)

[Dan Gudmundsson]

luvish satija writes:
 >  Hello all,
 > 
 > =ERROR REPORT==== 24-Jun-2005::10:48:50 ===
 > Mnesia(server@REDACTED): ** WARNING ** Mnesia is
 > overloaded: {mnesia_tm, message_queue_len,
 > [3156,3105]}
 > 
 > 
 > Now I have certain doubts:
 > 1. When I am creating only 4 tables, then how come
 > "too many db tables" error is thrown?

Each transaction uses ONE ets table to store everthing before data
i commited.

 > 
 > 2. To solve the problem at my level, I made some
 > changes in the program. Now instead of all the
 > processes simultaneously jumping on mnesia, there is
 > only one process responsible for the transaction with
 > mnesia. All the client processes contact that process.
 > To my surprise this solution worked, and I could
 > safely reach a limit of 10000. To check mnesia's
 > status, the following was the output of ets:i().

Then you have syncronized everything, if you do that you don't 
have to use transactions at all. Since no transactions will run in parallell
you can change it to use mnesia:async_dirty(Fun) instead of mnesia:transaction(Fun) 
to reduce the transaction overhead.

But if you want to add more machines in to your cluster it isn't 
a good solution.

Or if you can slow down/reduce your no clients in some other way, 
so that mnesia can have the resources it needs, i.e. a limit to 10000
clients per machine, or atleast 10000 active transactions per machine.
(or whatever your ERL MAX ETS TABLES is set to).

Overload control is tricky that why mnesia leaves it to the user :-)

 > 3. So, the result which I have drawn is that Mnesia
 > was getting overloaded because of so many processes
 > contending for transaction. AM I THINKING IN RIGHT
 > DIRECTION or is there anything which is beyond this?

Yes, actually it is to many simultanous transactions.

 > 4. Why the entry mnesia_transient_decision having id
 > 30 is having such a large size and taking large memory
 > (nearly double of my tables's size. Tables created by
 > my process are client_buddylist,
 > client_offlinemessages, client_online,
 > client_profile)?  

Thats a cache of the commit results, it should be ok.
You have commited a lot of transactions..

 > 
 > Thanks in anticipation,
 > /luvish
 >