[erlang-bugs] Mnesia/R15B: TYPE ASSERTION FAILED, erl_term.c line 109 (when stopping mnesia)

[Sverker Eriksson]

Winston Smith wrote:
> On Tue, Apr 3, 2012 at 9:59 PM, Winston Smith
> <smith.winston.101@REDACTED> wrote:
>   
>> On Mon, Apr 2, 2012 at 11:04 PM, Winston Smith
>> <smith.winston.101@REDACTED> wrote:
>>     
>>> I have run into the following issue with R15B cross compiled to an
>>> AVR32 (similar to ARM) system (no HiPE).
>>>
>>>       
> [snip]
>   
>> 2> mnesia:stop().
>> TYPE ASSERTION FAILED, file beam/utils.c, line 2380: !is_header(x)
>> Aborted (core dumped)
>>     
> [snip]
>   
>> I still have no backtrace, but indeed line 2380 of utils.c is in the
>> middle of the function "cmp" that I found previously.  I did
>> rebuilding the debug build with -fno-omit-frame-pointer, but it hasn't
>> improved the stack trace.
>>     
>
> Don't know what's up with bt/where not working in gdb, but with
> -fno-omit-frame-pointer (and a bit of research into the AVR32 ABI), I
> can trace the stack via the fp register R7; it points to the return
> address followed by the previous value of R7 (like EBP on x86).
>
> So I have been able to recreate the "C" stack manually:
>
>     0 - cmp(a,b)  --  utils.c:2380
>     1 - seqeq_2(l)  --  erl_bif_op.c:115
>     2 - db_prog_match()  --  erl_db_util.c:1997
>     3 - db_match_dbterm()  --  erl_db_util.c:5051
>     4 - db_select_chunk_hash()  --  erl_db_hash.c:1485
>     5 - db_select_hash()  --  erl_db_hash.c:1404
>     6 - ets_select2()  --  erl_db.c:2376
>     7 - ets_select2()  --  erl_db.c:2350
>     8 - process_main()  --  beam_emu.c:2642
>     9 - ?? (0x02)
>
> Unfortunately, I have not yet figured out how to get to local
> variables from the AVR32 ABI, I'd really like to see what's going on
> at frame #2.  It's trying to process a matchCall2 where it puts the
> arguments to seqeq_2 into the stack by accessing esp[-1] and esp[-2].
>
> I have tried to recompile with -DHARDDEBUG to turn on more tracing as
> described at the bottom of this page:
>
> http://carpanta.dc.fi.udc.es/docs/erlang/dbg.html
>
> Although it looks like the code -DHARDDEBUG enables doesn't compile in
> R15B01.  I will see if I can fix that ...
>
>   
This is an ETS-bug according to your call trace. ets:select/2 is called 
and the matchspec seems to contain '==' as the BIF seqeq_2 is called.
db_prog_match() is the stack based VM that executes matchspec's. For 
some reason an invalid term (a "header" with two lowest bits as zero) is 
read from the stack.

I don't have any ARM machine to test on at the moment.
Some debugging ideas:

1. Add "#define HARDDEBUG" only to erl_db*.c (the ETS-code).
2. Add "#define DMC_DEBUG" in erl_db_util.c.
3. Add your own erts_printf's. Use %T to print erlang terms (Eterm's).
4. Add a check in db_prog_match to assert that the esp-stack is ok 
before/after each instruction in the big for-loop.

Something like

Eterm* esp_start = esp;

for (;;) {

    ASSERT(esp >= esp_start);
    for (tp = esp_start; tp < esp; tp++) {
        ASSERT(!is_header(*tp));
    }

    switch (*pc++) {



/Sverker, Erlang/OTP Ericsson