[erlang-questions] mnesia memory isn't freed after search ?

Leon de Rooij leon@REDACTED
Sun May 24 22:34:08 CEST 2009

Hi all,

I'm doing some tests with Mnesia, trying to load a milion records and
read them, but apparently memory isn't freed after I do a search. After
a few searches my whole erl shell crashes :(

At the end of this mail is the module I wrote with a test that breaks
when run multiple times and attached is an image of graphs from the
system monitor of Ubuntu.

I started erl (R13B) with:

+A 128
-smp enable
-mnesia dump_log_write_threshold 50000
-mnesia dc_dump_limit 40

I did the following things:
(every peak in the image of CPU history is a search)

1> breakage:init().              % at ~300 sec
2> breakage:fill_table(1000000)  % from ~250 to ~225 sec
..some warnings about mnesia overload..
3> breakage:list_users().        % from ~190 to ~175 sec
4> breakage:list_users().        % from ~165 to ~155 sec
6> breakage:list_users().        % from ~145 to ~130 sec
7> breakage:list_users().        % from ~120 to ~105 sec
8> breakage:list_users().        % from ~95 to ~85 sec
9> breakage:list_users().        % from ~80 to ~70 sec
10> breakage:list_users().       % from ~65 to ~50 sec
11> breakage:list_users().       % from ~40 to ~30 sec
12> breakage:list_users().       % from ~25 to ~20 sec

Crash dump was written to: erl_crash.dump
eheap_alloc: Cannot allocate 3563526520 bytes of memory (of type


I don't understand why memory usage isn't the same after as before
list_users() is called.. Actually, the third time when list_users() is
called, the memory drops a bit and then goes back to the level of before
the first search! Is Erlang caching variables even though they went out
of scope ?

I understand it's not wise to read so many records at once, but better
do it in small batches. But still, I assume that if memory isn't freed
after a large search, it won't be after a small one either, it's just
less noticable then, right?

I tried running fill_table(N) with smaller values and then let
quick_break() run for half an hour or so, but it didn't crash then -
also didn't see a noticable memory increase. (Up to 800k goes alright,
850k and up breaks)

Can anyone tell me what I'm doing wrong ?

thanks & kind regards,

Leon de Rooij



-record(user, {username, attrs=[], params=[], vars=[], groups=[]}).

init() ->
        mnesia:create_table(user, [
                {disc_copies, [node()]},
                {attributes, record_info(fields, user)}]).

fill_table(N) ->
        F = fun(C) ->
                                attrs=[{"ak1", "av1"}, {"ak2", "av2"}],
                                params=[{"pk1", "pv1"}, {"pk2", "pv2"}],
                                vars=[{"vk1", "vv1"}, {"vk2", "vv2"}],
                                groups=["group1", "group2"]
        lists:foreach(fun(Z) -> F(Z) end, lists:seq(1,N)).

list_users() ->
        mnesia:dirty_match_object({user, '_', '_', '_', '_', '_'}),

quick_break() ->
quick_break(I) ->
	io:format("iteration ~p~n", [I]),
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Screenshot-System Monitor-1.png
Type: image/png
Size: 13892 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20090524/83e1783c/attachment.png>

More information about the erlang-questions mailing list