[erlang-questions] Memory Usage
Tue Sep 29 18:48:32 CEST 2009
On Tue, Sep 29, 2009 at 10:11:57AM -0600, Erickson, John wrote:
> Hi, I am trying to store a large list of items and am a little surprised how much memory erlang is using. As a test, I tried creating a list of 10M integers and measuring memory usage. It takes around 600M. If I convert this list to a binary using term_to_binary, it is only about 50M, which is more like what I would expect. So I am wondering why there is 10x memory usage in the usual form? One other small oddity, it seems after calling garbage_collect I am using more memory.
Erlang lists incur about an 8x expansion in memory usage since each
element is a pair of 32 bit integers. I just dealt with this myself in
my gen_smtp erlang email server. I was having a terrible time reading in
large emails (3mb or larger) because of the ballooning memory usage. I
rewrote everything to use binaries wherever possible (had to write some
functions for working with binary strings) but I've seen a huge
reduction in the time and memory needed to handle large emails.
So yeah, binaries are a much better solution for any larger data,
they're just a little tricker to work with because the bitstring library
proposed by EEP9 doesn't seem to exist.
More information about the erlang-questions