Loading Mnesia dbase
Fri Nov 18 21:53:37 CET 2005
First of all, try enabling the thread pool.
I would pick a nice round number like 255, i.e.
erl +A 255.
Also, is your erlang node running out of physical
RAM? It seems like it wouldn't, with 2 GB RAM, but
I guess if you keep items both in process space and
in mnesia, there would be a chance... Anyway, you
can easily track memory use using 'top' or 'vmstat'.
Den 2005-11-18 20:13:39 skrev <>:
> I have total of 14 million entries in several files to load into a
> Mnesia dbase. I used one process per file and found that the first 10
> million entries took around 15 mins to load into Mnesia (ram_copy).
> However after this initial 10 million entries things started to crawl. A
> 'ps' shows beam mainly blocking in I/O with around 3% CPU usage.
> I then restarted the test with a table of 15 fragments (ram_copy)
> thinking I hit some Mnesia limit. This dropped my initial 10 million
> entries to 8.5 mins. However its now back down to a crawl. The last
> 72000 entries took over 40 mins.
> I'm running on SuSe 10 (64bits), dual AMD Opteron 246 with 2GB RAM. I
> have 5 processes with 3 million entries each which is around 50MB of
> disc space per file. Each entry is a list of 6 integers. Each record in
> the dbase has 7 integers. All processes and Mnesia are running of one
> instance of the VM. Also using dirty_write's.
> Does anyone have any suggestions on speeding this ?
More information about the erlang-questions