[erlang-questions] Memory Usage w/disc_copies

Kevin A. Smith <>
Thu Feb 19 05:38:55 CET 2009


I know this has been discussed on the list before but I haven't found  
a satisfactory answer either in the list archives or Google.

I've got some test code which queries 9 disc_copies tables in a tight  
loop. The tables have been preloaded and no new data is created during  
the test. Querying is done via dirty_match_object/1.

I've noticed that the memory footprint of the Erlang VM grows as long  
as I leave my test running. The VM will start with a low memory  
footprint, say around 3-4 MB, and then grow to over 360 MB in the  
space of a minute. I know I could use ets or dets directly and  
probably workaround this issue but I'm intrigued and I'd like to  
understand exactly what's happening.

erlang:memory/0 reports this after a typical test run:

[{total,385341784},
  {processes,694596},
  {processes_used,686596},
  {system,384647188},
  {atom,402833},
  {atom_used,372656},
  {binary,13728},
  {code,2993679},
  {ets,380521792}]

It looks like ets is responsible for the lion's share of memory usage.

I've tried all of the standard tricks:

* setting fullsweep_after via erlang:system_flag/2 (before any test  
code is run)
* calling erlang:garbage_collect/0 at the top of each loop iteration
* calling mnesia_recover:allow_garb/0 and mnesia_recover:start_garb/0  
frequently

None of these changes have resulted in any measurable improvement. I  
did find this post from last year:

http://tinyurl.com/mnesia

I've verified that Dan Gudmundsson's changes (see the end of the above  
thread) are present in mnesia_dumper and mnesia_recover. I'm running  
R12B-5 so it looks like Dan's changes were incorporated at some point.

The odd thing is if I change the table type to disc_only_copies the  
problem goes away. Memory usage peaks at around 20 MB and averages out  
around 11 MB.

Can anyone shed some light on the behavior I'm seeing?

Thanks,
Kevin



More information about the erlang-questions mailing list