Loading Mnesia dbase
Fri Nov 18 22:20:07 CET 2005
> I'll give the thread pool a try. I'm maxing out at 1.8 GB RAM so
> everthing is still nicely in RAM space.
That doesn't leave much room for everything else running on the
machine. These symptoms do tend to indicate that you are going into
> -------- Original Message --------
> From: "Ulf Wiger" <>
> To: ,
> Subject: Re: Loading Mnesia dbase
> Date: Fri, 18 Nov 2005 21:53:37 +0100
>> First of all, try enabling the thread pool.
>> I would pick a nice round number like 255, i.e.
>> erl +A 255.
>> Also, is your erlang node running out of physical
>> RAM? It seems like it wouldn't, with 2 GB RAM, but
>> I guess if you keep items both in process space and
>> in mnesia, there would be a chance... Anyway, you
>> can easily track memory use using 'top' or 'vmstat'.
>> Den 2005-11-18 20:13:39 skrev <>:
>>> I have total of 14 million entries in several files to load into a
>>> Mnesia dbase. I used one process per file and found that the
>>> first 10
>>> million entries took around 15 mins to load into Mnesia (ram_copy).
>>> However after this initial 10 million entries things started to
>>> crawl. A
>>> 'ps' shows beam mainly blocking in I/O with around 3% CPU usage.
>>> I then restarted the test with a table of 15 fragments (ram_copy)
>>> thinking I hit some Mnesia limit. This dropped my initial 10 million
>>> entries to 8.5 mins. However its now back down to a crawl. The last
>>> 72000 entries took over 40 mins.
>>> I'm running on SuSe 10 (64bits), dual AMD Opteron 246 with 2GB
>>> RAM. I
>>> have 5 processes with 3 million entries each which is around 50MB of
>>> disc space per file. Each entry is a list of 6 integers. Each
>>> record in
>>> the dbase has 7 integers. All processes and Mnesia are running of
>>> instance of the VM. Also using dirty_write's.
>>> Does anyone have any suggestions on speeding this ?
>> Ulf Wiger
More information about the erlang-questions