[erlang-bugs] Bug in ei_decode_big, implementation for ei_*encode_big

[Paul Guyot]

Hello,

ei.h has the following FIXME notice before the prototype of the big  
functions:

/* FIXME: is this really the best way to handle bignums? */

The thing is that ei_decode_big has a bug that makes it unusable,  
around line 54 of decode_big.c:

       for (i = 0; i < b->arity; ++i) {
	  dt[i] = u[i*2];
	  dt[i] |= ((unsigned short) u[(i*2)+1]) << 8;           <--- this  
is line 54
       }

The high order bits of the 16-bits digit (short) are read, even if  
the byte (at u[(i*2)+1]) doesn't belong to the big. In other words,  
big numbers such as 16#1234567890 with an odd number of bytes are  
decoded with a garbage byte (i.e. 16#XX1234567890).

The following change fixes the problem:

       for (i = 0; i < b->arity; ++i) {
	  dt[i] = u[i*2];
	  if ((i*2 + 1) < digit_bytes)
	  {
               dt[i] |= ((unsigned short) u[(i*2)+1]) << 8;
	  }
       }

Also, ei_*encode_big is missing. Here is a sample implementation:

int ei_encode_big(char *buf, int *index, erlang_big* big)
{
     unsigned short* digits = big->digits;
     char *s = buf + *index;
     char *s0 = s;
     int arity = big->arity;
     long digit_bytes = (arity * 2);

     if (digit_bytes < 256)
     {
         if (buf != NULL)
         {
             put8(s, ERL_SMALL_BIG_EXT);
             put8(s, digit_bytes);
         } else {
             s += 2;
         }
     } else {
         if (buf != NULL)
         {
             put8(s, ERL_LARGE_BIG_EXT);
             put32be(s, digit_bytes);
         } else {
             s += 5;
         }
     }
     if (buf != NULL)
     {
         put8(s, big->is_neg);
         int i;
         for (i = 0; i < arity; ++i)
         {
             put16le(s, digits[i]);
         }
     } else {
         s += 1 + (arity * 2);
     }

     *index += s-s0;

     return 0;
}

int ei_x_encode_big(ei_x_buff* x, erlang_big* big)
{
     int i = x->index;
     ei_encode_big(NULL, &i, big);
     if (!x_fix_buff(x, i))
	return -1;
     return ei_encode_big(x->buff, &x->index, big);
}

Regards,

Paul