[erlang-questions] Memory Usage w/disc_copies
Kevin A. Smith
Thu Feb 19 05:38:55 CET 2009
I know this has been discussed on the list before but I haven't found
a satisfactory answer either in the list archives or Google.
I've got some test code which queries 9 disc_copies tables in a tight
loop. The tables have been preloaded and no new data is created during
the test. Querying is done via dirty_match_object/1.
I've noticed that the memory footprint of the Erlang VM grows as long
as I leave my test running. The VM will start with a low memory
footprint, say around 3-4 MB, and then grow to over 360 MB in the
space of a minute. I know I could use ets or dets directly and
probably workaround this issue but I'm intrigued and I'd like to
understand exactly what's happening.
erlang:memory/0 reports this after a typical test run:
It looks like ets is responsible for the lion's share of memory usage.
I've tried all of the standard tricks:
* setting fullsweep_after via erlang:system_flag/2 (before any test
code is run)
* calling erlang:garbage_collect/0 at the top of each loop iteration
* calling mnesia_recover:allow_garb/0 and mnesia_recover:start_garb/0
None of these changes have resulted in any measurable improvement. I
did find this post from last year:
I've verified that Dan Gudmundsson's changes (see the end of the above
thread) are present in mnesia_dumper and mnesia_recover. I'm running
R12B-5 so it looks like Dan's changes were incorporated at some point.
The odd thing is if I change the table type to disc_only_copies the
problem goes away. Memory usage peaks at around 20 MB and averages out
around 11 MB.
Can anyone shed some light on the behavior I'm seeing?
More information about the erlang-questions