[erlang-bugs] Bugs with hibernate/3 and HiPE

[Mikael Pettersson]

Paul Guyot writes:
 > Hello,
 > 
 > There are several bugs related to hibernate/3 BIF and HiPE:
 > 
 > 1. erlang:hibernate/3 simply fails with badarg when called from HiPE.
 > 2. erlang:hibernate/3 simply fails with badarg when called dynamically, in a way that the compiler and the loader cannot replace with the i_hibernate beam instruction.
 > 3. There is a segfault in the garbage collector when calling erlang:hibernate/3 (from beam unless bug #1 is fixed) if the process went through HiPE. This is because erts_garbage_collect_hibernate would not clean up the hipe process state.
 > 
 > These bugs are highly related: 1 and 2 are actually the same bug (hibernate/3 BIF isn't implemented, and apparently wasn't since it was introduced), and 3 causes segfaults with any reasonable test once bug #1 is fixed.
 > 
 > The following patch fixes these issues:
 > http://github.com/pguyot/otp/commit/fcc005c8758d97d7f0c402bfffede0b22b8ab401
 > git fetch git://github.com/pguyot/otp.git fix-hibernate-with-hipe

Your "fix" for #3 is wrong I think.  hibernate should empty the stack.
If you don't empty the native stack then yes, at the next GC, we'd
scan a dead stack, find dead pointers to stale data, and do bad things.
Your call to fullsweep_nstack() changes this so that in the next GC
we scan a dead stack, find dead pointers to tenured data, and don't
crash.

The correct fix ought to be to simply empty the native stack.  There's
no existing procedure for doing that, but something like

void hipe_empty_nstack(Process *p)
{
    p->hipe.nstgraylim = NULL;
#if defined(HIPE_NSTACK_GROWS_DOWN)
    p->hipe.nsp = p->hipe.nstend;
#else
    p->hipe.nsp = p->hipe.nstack;
#endif
}

should be close to what's needed.  hipe_native_bifs.c right after the
definitions of hipe_inc_nstack() seems like a reasonable place to put it.

/Mikael