[erlang-questions] process and closure memory usage
Thu Mar 19 14:38:57 CET 2009
On Thu, Mar 19, 2009 at 10:42 AM, Ladislav Lenart <lenartlad@REDACTED>wrote:
> here are my thought vut I am by no means an Erlang expert...
> 1. I'd say that using gen_server has no visible memory or performance
> overhead compared to raw Erlang processes. It is more a question of
> whether gen_server way of handling incoming messages in order as they
> come suits your needs. Also you can make a raw Erlang process into an
> OTP-aware process using proc_lib.
> 2. I think that process_info reports all memory allocated by the
> process. From my experience a process roughly uses twice as much
> memory as it needs.
Doubling memory footprint depends in way how Erlang GC works. It is
trade-off between scalability and soft real-time characteristics and memory
This can be quite a lot for a large number of
> processes. If only a handful of processes are active in any given
> time, you could use erlang:hibernate/3 for a raw Erlang process
> or specify hibernate instead of timeout in gen_server return tuple.
> 3. This is the most interesting part. Just to clarify, by closure I
> mean fun (...) -> ... end. If you are sending funs between processes
> then things can get interesting because beside the code pointer also
> the lexical scope of the function is sent over the wire. And in order
> to be sent it has to be serialized first. During this serialization,
> the "data identity" is not preserved. So if you use complex structure
> that contains multiple identical parts the scope will be larger after
> the serialization. This is true for any Erlang term. Example: [A, A,
> A, A] where A is bound to some tuple. If you send this as a message,
> the receiver will get [A1, A2, A3, A4, A5] where all received As are
> equal but no longer point to the same memory location.
> 4. None that I know of.
> Ladislav Lenart
> Matt Handler wrote:
> > I'm trying to cut down on the process memory usage for a non-relational
> > database i'm working on. essentially there are 10,000 processes (setup
> > as gen_servers) that send information to each other in various ways. i
> > have several questions:
> > 1. how much of an overhead does using the gen_server model assert?
> > if i'm not as worried about dynamic code loading, supervising,
> > etc... am i wasting memory and processing using the gen_server?
> > 2. how/where can i find out more information about the memory usage
> > of a process? i've used process_info/2 heavily to gather some of
> > this information, but my problem is that if i look at the rough
> > size of the state of a process (using erlang:iolist_size/1), it's
> > significantly less than the size of the process as reported by
> > process_info/2. i want to know where the rest of the memory is
> > being eaten up (initially the processes are only 1k), on average,
> > the process size is about 4x the state size.
> > 3. i am passing around many closures between the nodes of the
> > database, and they are getting to be quite large (start out as
> > 100bytes, end up as 40kbytes). these are what is taking up most
> > of the memory of the state, and so i'm wondering if there is some
> > inner workings of the erlang system that make this a bad way of
> > programming in terms of memory usage. it seems like there are
> > certain situations that make closures take up a lot of memory, but
> > i'm having trouble pinpointing when.
> > 4. is there a good tool out there for profiling the memory usage
> > rather than executiontime usage?
> > thanks for the help,
> > -matt handler
> > ------------------------------------------------------------------------
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://www.erlang.org/mailman/listinfo/erlang-questions
> erlang-questions mailing list
--Hynek (Pichi) Vychodil
Analyze your data in minutes. Share your insights instantly. Thrill your
boss. Be a data hero!
Try Good Data now for free: www.gooddata.com
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the erlang-questions