Unaccounted memory usage...

[Shawn Pearce]

Can someone help me explain this a little?  My beam process is currently at
a little over 200 MB resident heap size.  I have approx 1700 processes running
in the node.  Only 19 are registered.  This is R9B-0, 32 bit (Linux).  I'm not
running distributed, so no epmd, no connected nodes.

i() in the shell reports I'm using 2430190 words heap, 19828 words stack, for
a total of 2,450,018 = 9.57 MB.

ets:i() reports 16 tables are allocated.  Most of them aren't interesting, as
they are the standard erts/OTP tables, and aren't of an obscene size.  The
7 application tables report a total memory usage of 234,183 words = 915 KB.

Thus I figure my total application memory usage at about 11 MB.  Yet I
have a 200 MB heap size.

41> statistics(garbage_collection).
{152876,298095162,0}
42> (298095162*4)/(1024*1024).
1137.14

Clearly with only 200 MB of actual process heap, we don't have 1.1 GB
currently free, but have made 1.1 GB of junk overall.  That's fine, but
where is my ~180 MB?

43> erlang:system_info(allocated_areas).
[{definite_alloc,2097152,1529816},
 {static,402553},
 {atom_space,65544,36019},
 {binary,2776510},
 {atom_table,25317},
 {module_table,824},
 {export_table,14864},
 {register_table,144},
 {loaded_code,1100740},
 {process_desc,516800,503120},
 {table_desc,1520,1444},
 {link_desc,240480,236520},
 {atom_desc,93120,93096},
 {export_desc,98880,98784},
 {module_desc,4000,3320},
 {preg_desc,400,380},
 {mesg_desc,56400,0},
 {plist_desc,0,0},
 {fixed_deletion_desc,0,0},
 {bif_timer_rec_desc,1280,64},
 {fun_desc,14560,13608}]

Manual says these are in bytes.  I don't see my 180 MB here.  :(
How can I get the amount of memory that is free within erts, but that
can't be reclaimed by the OS?

I can only come up with two explanations for the widely different memory
information:

	- My handful of transient processes that were created used more memory
	  than the others, and pushed the heap up.  (The 1700 is what is left
	  after about 850 transients were created and shutdown.)

	- The number of monitors I have in 1 process (~ 850) is eating a large
	  amount of memory.  Or I'm leaking monitors at some point.  Does one have
	  to demonitor a monitor if we get a 'DOWN' message noting the process has
	  exited?

I'm putting money on the first one being the problem.

-- 
Shawn.

  FORTUNE PROVIDES QUESTIONS FOR THE GREAT ANSWERS: #4
  A:	Go west, young man, go west!
  Q:	What do wabbits do when they get tiwed of wunning awound?