[erlang-bugs] Lengthy subfunctions in Core Erlang code crash the compiler if inlining is turned on

[Björn Gustavsson]

On Mon, Oct 10, 2011 at 10:49 PM, Rene Kijewski
<rene.kijewski@REDACTED> wrote:
> Hello,
>
> (I first asked this question in Stackoverflow but was hinted that this ML
> is a better place for such questions.)
>
> I wanted to test the inlining capabilities of the compiler and wrote a
> method containing three lengthy subfunctions. (See attachment.)
>
> Compiling it without a 'inline' flag works as expected but supplying it
> renders a crash:
>
>> $ erlc +verbose +from_core +inline lengthySubfun.core
>> Inlining: inline_size=24 inline_effort=150
>> Function: test/6
>> ./lengthySubfun.core:none: internal error in core_dsetel_module;
>> crash reason: {{badmatch,error},
>>                [{sys_core_dsetel,visit,2},
>>                 {lists,mapfoldl,3},
>>                 {sys_core_dsetel,visit,2},
>>                 {sys_core_dsetel,visit,2},
>>                 {sys_core_dsetel,visit,2},
>>                 {sys_core_dsetel,visit,2},
>>                 {sys_core_dsetel,'-visit_def_list/2-anonymous-0-',2},
>>                 {lists,mapfoldl,3}]}
>
> unless I specify an insanely high value in range of 10 000 for both
> inline_effort and inline_size.
>

I am not sure whether this should be classified as a bug or not.

The case statement in the "sub-function" continue/1 only contains
one clause:

case <V> of
    <{V250}> when 'true' ->
        ...
end

The Core Erlang specification says that the behavior is undefined
if no clause can be selected in a case. Looking at only at the
case statement itself, it is impossible to tell whether the clause
will always be selected. Only by looking at the entire program can
it been seen that the clause will indeed always be selected.

The inliner assumes that the compiler has generated
code similar to this:

case <V> of
    <{V250}> when 'true' ->
        ...
    <Other> when 'true' ->
       primop 'function_clause'
end

so that it can clearly be clearly seen looking just at
the case statement that one of the clauses is guaranteed
to be selected.

I am not sure yet whether we will attempt to fix this
problem in the inliner, since it can only happen when
compiling from Core Erlang code, never from Erlang
code.

/Björn
-- 
Björn Gustavsson, Erlang/OTP, Ericsson AB