Gen_server and states

[Ulf Wiger]

On 27 Oct 1999, Luke Gorrie wrote:

luke>Mickael Remond <mikl@REDACTED> writes:
luke>
luke>> How would you put up with a server as soon as you get a 
luke>> huge amount of data, that cannot fit into your RAM ? 
luke>> Would you use a mnesia table or Dets and use
luke>> the state as an handler to those data ?
luke>
luke>Yes, though it might affect your design more than this. For
luke>instance, if you were storing all your state in Mnesia, you might
luke>throw the server away and execute requests in the client process,
luke>since you don't have to worry about serialising access to the
luke>state when you're using transactions.

Another option is to use ETS. If you have lots of data in your state,
you'll want to break it up into smaller pieces so that you can lookup one
item at a time.

Depending on your problem, you may then also allow clients to read the data
directly from ETS, while serialising all writes, using the server.
Sometimes this is a good idea, but think about it carefully, so you don't
cause race conditions.

The main problem with having a huge state record in a gen_server is GC.
Keeping the data in ETS eliminates this. Keeping it in dets would probably
incur overhead similar to the GC overhead.

Personally, I've tried keeping a data set of 30,000 small records on the
process heap, with relative success, so you may have to experiment with the
limits to see what works for you.

/Uffe

Ulf Wiger, Chief Designer AXD 301
Ericsson Telecom AB                          tfn: +46  8 719 81 95
Varuvägen 9, Älvsjö                          mob: +46 70 519 81 95
S-126 25 Stockholm, Sweden                   fax: +46  8 719 43 44