geometric memory growth
Ulf Wiger (AL/EAB)
Wed Nov 30 17:23:55 CET 2005
Thomas Lindgren wrote:
> > In the case of a record access (assuming that a guard
> > test or other exists that asserts that it is in fact
> > a record), you can't observe a functional difference
> > in this case.
> As long as we know the variable has the right record type, yes.
> (The "proper" way of reasoning for a compiler is to
> talk about ALL programs, rather than specific ones.
> So, an optimization has to be safe for all programs;
> while profitability is a tradeoff by the compiler
> writer, influenced by the compiler users, of course.)
So we agreed that it's safe in this case (when you
know that it's really a record). Now to whether it is
> Often, expensive operations are simply left where they
> occur, while lightweight operations can be moved
> around (since the cost of being wrong is slight).
On my machine, moving a record field selection outside
the fun costs about 0.07 usec (mean value based on 10,000
iterations). Of course, on a gumstix, it would be more.
The cost doesn't quite seem to scale linearly -
moving 25 record field accesses out of the fun
added 0.6 usec to the cost, compared to just
inheriting the record.
I don't know if that qualifies as cheap enough.
Of course, if we're talking >10 field selections,
it would amount to
> To make it clearer, the compiler has to be careful not
> to make spectacularly bad decisions on its own.
To make my point clearer, I think this particular
behaviour should be documented _somewhere_. What
happened was oviously a combination of two things:
- since State#state.channel was (understandably)
interpreted as basically a call to element(P,State),
State was bound in the fun's environment.
- During message passing, not only does the runtime
system copy the data - it expands all relative
references into individual copies (I knew this
happened when sending between nodes, but hadn't
considered that it also might happen during
local message passing.)
As far as I can tell, none of these things are
actually written out anywhere in the documentation,
either in the 4.7 spec, in the reference manual,
or in the efficiency guide.
To boot, the "bug" was very difficult to spot,
and afterwards, I "spotted" it in several other
places, but decided that it was safe, since the
fun would never be passed in a message, but
was only used on the local process heap and
Alternatively, the compiler could lift record field
accesses out of the fun, possibly with a limit of
say 10, and it still would never be spectacularly
bad or unsafe. Lifting 10 accesses essentially
doubled the cost of the function, which is to
say that the cost went up from about 0.3 usec to
Opinions may vary, but as a representative of a
project that builds resonably performance-critical
software in Erlang, I would still rather take
a very slight performance hit in order to have
a treatment of a reasonably commonplace pattern
that works wonderfully 99.9% of the time, and
kills the system if someone has the bad luck
of passing the variable in a message (perhaps
as part of a debug printout.)
> For example, if you had written your OPTIMIZED code at
> first and the compiler then turned it into the
> original code -- ie, if the compiler moved the record
> INTO the closure -- then you probably would have
> complained bitterly. (I know I would :-)
Oh, I never get bitter - just passionate. ;-)
More information about the erlang-questions