[erlang-questions] Memory Usage w/disc_copies

[Scott Zhang]

Hi. Kevin.I am not sure your exact question is. But based on my experience
with mnesia with table type disc_copies. mnesia will keep a complete copy of
table in ram too. The first time you access that table,(Ususally I am using
tv:start()). It reside in RAM. Comsume your RAM until it crash your erlang
VM.

It is a potential dangerous to use ram_copies and disc_copies, because
unconscious it will crash itself.


Regards.
Scott



On Thu, Feb 19, 2009 at 12:38 PM, Kevin A. Smith
<kevin@REDACTED>wrote:

> I know this has been discussed on the list before but I haven't found
> a satisfactory answer either in the list archives or Google.
>
> I've got some test code which queries 9 disc_copies tables in a tight
> loop. The tables have been preloaded and no new data is created during
> the test. Querying is done via dirty_match_object/1.
>
> I've noticed that the memory footprint of the Erlang VM grows as long
> as I leave my test running. The VM will start with a low memory
> footprint, say around 3-4 MB, and then grow to over 360 MB in the
> space of a minute. I know I could use ets or dets directly and
> probably workaround this issue but I'm intrigued and I'd like to
> understand exactly what's happening.
>
> erlang:memory/0 reports this after a typical test run:
>
> [{total,385341784},
>  {processes,694596},
>  {processes_used,686596},
>  {system,384647188},
>  {atom,402833},
>  {atom_used,372656},
>  {binary,13728},
>  {code,2993679},
>  {ets,380521792}]
>
> It looks like ets is responsible for the lion's share of memory usage.
>
> I've tried all of the standard tricks:
>
> * setting fullsweep_after via erlang:system_flag/2 (before any test
> code is run)
> * calling erlang:garbage_collect/0 at the top of each loop iteration
> * calling mnesia_recover:allow_garb/0 and mnesia_recover:start_garb/0
> frequently
>
> None of these changes have resulted in any measurable improvement. I
> did find this post from last year:
>
> http://tinyurl.com/mnesia
>
> I've verified that Dan Gudmundsson's changes (see the end of the above
> thread) are present in mnesia_dumper and mnesia_recover. I'm running
> R12B-5 so it looks like Dan's changes were incorporated at some point.
>
> The odd thing is if I change the table type to disc_only_copies the
> problem goes away. Memory usage peaks at around 20 MB and averages out
> around 11 MB.
>
> Can anyone shed some light on the behavior I'm seeing?
>
> Thanks,
> Kevin
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20090219/d471dcc0/attachment.htm>