[erlang-questions] simple question about list memory consumption

[CGS]

Well, I think I have to apologize. I had the impression that the machine I
was working on was 64-bits. It is 32-bits, so, yes, 5 words per character
means the list from the lists:seq/2 resides fully in memory. My bad, sorry.

I did the test again using the loop from Richard's test and, indeed, I got
about 80 MB memory consumption for 10^7 characters which is consistent with
2 words per character in the list.

Now, to answer how free can be used in such a test:
1. execute command: watch -n 1 'free -m' (every 1 second calls free -m
(output in MB) and reports its output);
2. stabilize your OS by killing all the processes which have a non-constant
memory usage at the level of MB (that is, for several seconds the output of
the previous command should remain constant);
3. start an Erlang shell (that adds usually something like 4 MB to the used
memory);
4. load the module (usually not visible in the output of (1), but it may
add 1 MB, depending on the system state by that time);
5. execute `<module>:loop(10000000), ok.';
6. the difference in between the output from (1) after step (5) and the
output from (1) after step (4) is giving the memory used by Erlang for the
list (+ some overhead).
This is much of how Richard did it in Erlang, but using external tools to
monitor the RAM consumption. I repeated several times this and I got almost
the same result (1 MB error, but that may be an artifact of the shell or
system).

Again, I am sorry for my mistake thinking that the sequence list was empty
when lists:map/2 exited, but I was really under the impression that the
machine I was working on is 64-bits (1 word = 8 B). And thank you for
spotting my mistakes in judgement.

CGS



On Tue, Jul 10, 2012 at 3:21 AM, Richard O'Keefe <ok@REDACTED> wrote:

>
> On 9/07/2012, at 10:34 PM, CGS wrote:
> > > About the two lists you were speaking about (I suppose you were
> referring to the output of lists:map/2 and lists:seq/2), the second is
> destroyed as soon as lists:map/2 exits.
> >
> > No it isn't.  It is only reclaimed when the garbage collector runs.
> >
> > That is a bit puzzling.
>
> It seems that you are muddling up "is LIVE" with "EXISTS".
> When you compute
>         map(F, L)
> and there are no other references to L anywhere,
> L becomes *DEAD* as soon as the call to map/2 completes.
> (In fact, each cell of L becomes dead in turn during the
> call.)  But DEAD and DESTROYED are very different properties.
> L is not "destroyed" until the garbage collector runs;
> until the garbage collector runs the cells of L still occupy
> space on the heap.  When, if ever, that space is reclaimed
> depends on how much other need there is for it.
>
> > Looking at the code for lists:map/2:
> >
> > map(F, [H|T]) ->
> >     [F(H)|map(F, T)];
> > map(F, []) when is_function(F, 1) -> [].
> >
> > I can see that when lists:map/2 exits, the only remaining list is an
> empty list.
>
> No you can't see that, because there is nothing in the code
> for map/2 to release any storage whatever.
>
> > I might be wrong, but I see only one word remaining for the garbage
> collector to take care of.
>
> What do you think makes the other words go away?
> >
> >
> > You got the result from process_info, I got it from `free' under Linux.
>
> Not the best way to do it.  That tells you how much memory
> the operating has handed out but not to which process; even
> ps -o size tells you only how much memory Erlang has claimed from
> the operating system, and nothing about what Erlang is doing with
> that memory.
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20120710/0918efb0/attachment.htm>