[erlang-questions] At what point am I "playing compiler"

[Richard Carlsson]

Thomas Lindgren wrote:
> ----- Original Message ----
>> Maybe it's time to integrate some of the high order lists
>> functions as part of the language ?
>
> Doing that could be well worth the effort. In 2001, I got a nearly 3x
> speedup of the beam compiler from converting a small collection of
> lists functions into local functions. (See EUC'01 proceedings.) The
> current system may behave differently, of course.

Actually... there is an undocumented compiler option 'inline_list_funcs'
that makes the compiler detect and inline expand the most common higher
order list functions (handled in lib/compiler/src/sys_core_fold.erl).

Currently, the following seem to be handled:
  all/2, any/2, foreach/2, map/2, flatmap/2, filter/2, foldl/3, foldr/3,
  mapfoldl/3, mapfoldr/3.

I implemented this years ago, along with the inliner. (I expected the
inliner to be able to reduce these further, when the fun is given
explicitly, but something seems to prevent this from happening at
present - needs more investigation, but the inliner is complicated.)

OTP kept it in there for experimental purposes, but weren't too excited
about it because: 1) it changes the semantics so that even if the lists
module is upgraded, these calls will stay as they were at compile time
(not that I think that will ever be a problem); 2) there is of course
code duplication - each such call becomes a separate little loop
function 0 (but so do list comprehensions); 3) debugging is affected,
because code that crashes in these functions will have different stack
traces compared to plain calls to the lists module; and 4) it is yet
another place where the compiler might introduce bugs if it generates
the wrong specialized code (which is trickier to debug than the source
code of the lists module) - but generate code is what compilers _do_.

If you can live with all that, I still think it's a good idea.  :-)

However, since this has been so little tested, it is not totally
impossible that in some case the generated inline code is wrong, for
any of the above list functions. Do try it out for speed, but be
careful with using it for production code as for now.

    /Richard