[erlang-bugs] Erlang problems with big binaries

[Mikael Pettersson]

Mikael Pettersson writes:
 > Nico Kruber writes:
 >  > I'd like to 'bor' (binary or) a big binary but the result is an empty list(?!)
 >  > Smaller binaries work as expected.
 >  > 
 >  > Steps to reproduce (at least on Erlang 17.4):
 >  > 
 >  > A=erlang:term_to_binary(lists:seq(1000000,2200000)).
 >  > B=erlang:term_to_binary(lists:seq(1000001,2200001)).
 >  > ASize=erlang:bit_size(A).
 >  > BSize=erlang:bit_size(B).
 >  > <<ANr:ASize>> = A.
 >  > <<BNr:BSize>> = B.
 >  > XNr = ANr bor BNr.
 >  > <<XNr:BSize>>.
 >  > 
 >  > 
 >  > Please note that printing the actual values of A or B is not a good idea ;)
 >  > and so would XNr but including it directly into a binary fails with the same 
 >  > error: "** exception error: bad argument"
 >  > <<(ANr bor BNr):BSize>>.
 > 
 > The problem is that erts/emulator/beam/big.c:big_norm() contains:
 > 
 >     if ((arity = BIG_NEED_SIZE(xl)-1) > BIG_ARITY_MAX)
 >       return NIL;  /* signal error (too big) */
 > 
 > where BIG_ARITY_MAX is 0x7ffff, but:
 > 
 > 1) this is an implementation restriction which can, as this
 > case proves, be exceeded by simple application code, and
 > 
 > 2) erts_{band,bor,bxor}() check for this with ASSERT(is_not_nil(...)),
 > but that's wrong since this is about input validation and implementation
 > limits not fundamental logic, and it's ineffective for release builds.
 > 
 > These three functions need to turn this case into a system limit
 > exception, just like a lot of other places in erl_arith.c already do.
 > (Well, at least _bor() and _bxor() do, perhaps _band() doesn't need to.)

Actually, it's worse than I thought.  Observe:

1>  A=erlang:term_to_binary(lists:seq(1000000,2200000)).
<<131,108,0,18,79,129,98,0,15,66,64,98,0,15,66,65,98,0,15,
  66,66,98,0,15,66,67,98,0,15,...>>
2> ASize=erlang:bit_size(A).
48000096
3> <<ANr:ASize>> = A.
<<131,108,0,18,79,129,98,0,15,66,64,98,0,15,66,65,98,0,15,
  66,66,98,0,15,66,67,98,0,15,...>>
4> ANr band ANr.
big_norm: xl 750002 arity 0xb71b2 > 0x7ffff
beam/erl_arith.c:1050:erts_band() Assertion failed: ((arg1) != ((~((Uint) 0) << 6) | ((0x3 << 4) | ((0x2 << 2) | 0x3))))
                Abort

So band:ing a bignum with itself (which by definition requires no more
bits than the input bignum) triggers the limit check.  I also get errors
for trivial ops like ANr - 1.

Looks like <<ANr:ASize>> = A can construct invalid bignums.