[erlang-bugs] bug in HiPE for <<_/utf8,...>>

[Johannes Weißl]

Hello Sverker,

Thanks a lot! The patch solves the bug in the 'crash.erl' program as
well as in our application.

Johannes

On Mon, Apr 20, 2015 at 04:51PM +0200, Sverker Eriksson wrote:
> Here is a pull request that i think solved this problem.
> 
> https://github.com/erlang/otp/pull/690
> 
> It would be good if you could test it out.
> The fix is both in the hipe compiler and in the VM,
> which means you have to rebuild both
> and then your own code with the new compiler.
> 
> /Sverker, Erlang/OTP
> 
> On 04/02/2015 10:20 AM, Johannes Weißl wrote:
> > Hi,
> > 
> > A short update: The reported HiPE bug is still present and reproducible
> > in OTP 17.5 and 18.0-rc1.
> > 
> > We had to modify the 'crash.erl' program a little bit after commits
> > 5a7b211 and 7b10ff7 (this shows how fragile it is!), so I'm attaching
> > the latest version:
> > 
> > MD5 (crash.erl) = 41d0e4e8ed7039dc898a16135aa62bcb
> > MD5 (crash_it.escript) = f7756e997d9ca28f6d523086e8c37f91
> > MD5 (data.jsn) = a0ee43e0e63aea6f3c89c41cc3b5d378
> > 
> > To reproduce the bug, save all three files in one directory, make
> > 'crash_it.escript' executable and run it. It compiles 'crash.erl'
> > with HiPE enabled, and produces a crash, which does not occur without
> > HiPE. See also the 'Details' section below for more information about
> > the bug.
> > 
> > Regards,
> > Johannes Weißl and Sebastian Egner
> > 
> > On Mon, Sep 09, 2013 at 02:20PM +0000, Sebastian Egner wrote:
> > > Hi,
> > > 
> > > There seems to be a Heisenbug in HiPE related to matching <<_/utf8,...>>.
> > > 
> > > After a long and bloody fight, we have been able to isolate the problem to the degree
> > > that it is sufficiently reproducible. See details below.
> > > 
> > > We strongly suspect that the problem is a genuine bug related to the binary matching
> > > and the garbage collector. Whether the bug is hit depends on the memory contents
> > > of previously allocated heap-allocated binaries.
> > > 
> > > Best regards,
> > > Johannes Weissl and Sebastian Egner.
> > > 
> > > --
> > > 
> > > Details:
> > > - The program 'crash.erl' loads a JSON sample file. Then it parses the file again and again,
> > >   and after a wildly varying number of iterations (100-100000) the parser fails.
> > > - To run the program, execute "crash_it" in a directory containing "crash.erl" and "data.jsn".
> > >   When the bug is hit, the program stops. This takes several seconds to minutes.
> > > - The problem manifests itself when <<"0123...">> does not match <<_/utf8,_/binary>>
> > >   in the function crash:check_utf8_binary/1. (The program aborts with an exception exit.)
> > > - Surprisingly, we have not been able to reduce the program even more.
> > >   In particular, when randomize_memory/0 is not called, the problem is much less frequent.
> > > - The bug is present in R13B02, R14B04, R16B01, "maint" (2f28245) and master (45eaf81).
> > > - The bug is present under MacOSX (10.8.4), Debian GNU/Linux and a Linux in an ARM emulator.
> > >   This indicates that the bug is not related to the operating system platform.
> > > - We have run the program in Valgrind and found conditionals that depend on uninitialised
> > >   values. Refer to "valgrind.out" for details.
> > > 
> > > Attachments:
> > > MD5 (crash.erl) = 1f1507c8238e2136d9163314bcac0045
> > > MD5 (crash_it) = 4061276b89dfc822cbfc22002f202358
> > > MD5 (data.jsn) = c5b503cc61d76adc7dcb60832a123b99
> > > MD5 (valgrind.out) = 2e6f67bf06b3df66c6daf728444b9b66