bugfix: c_num won't convert most negative value.

This bug causes a problem particularly in FFI. THe conversion
of an integer to the FFI int type begins by conversion via
c_num to the cnum type, which is at least as wide as int.
Alas, the INT_MIN value (e.g. #x-80000000 on 32 bits) will
not convert!

Fixing this has a ripple effect. We alter the INT_PTR_MIN
constant to include this value. This causes the derived
NUM_MIN to also include the extra negative value, extending
the fixnum range by one value. Everything seems to be okay.

* configure: Decrease value of INT_PTR_MIN and
DOUBLE_INTPTR_MIN constants by one to include the most
negative two's complement value.

* ffi.c (make_ffi_type_enum): We must not subtract 1 from
INT_PTR_MIN here any more.

* mpi.c (mp_in_range): If the bignum is negative, then extend
the range check by one value, so that we don't reject the
most negative two's complement value.

1 files changed, 3 insertions, 2 deletions

diff --git a/mpi/mpi.c b/mpi/mpi.c
index 9ece8047..5befd2b3 100644
--- a/mpi/mpi.c
+++ b/mpi/mpi.c
@@ -463,8 +463,9 @@ int mp_in_range(mp_int *mp, uint_ptr_t lim, int unsig)
 {
   const unsigned ptrnd = (SIZEOF_PTR + MP_DIGIT_BIT - 1) / MP_DIGIT_BIT;
   mp_size nd = USED(mp);
+  int neg = ISNEG(mp);
 
-  if (unsig && ISNEG(mp))
+  if (unsig && neg)
     return 0;
 
   if (nd < ptrnd)
@@ -476,7 +477,7 @@ int mp_in_range(mp_int *mp, uint_ptr_t lim, int unsig)
   {
     mp_digit top = DIGITS(mp)[ptrnd - 1];
     lim >>= ((ptrnd - 1) * MP_DIGIT_BIT);
-    return top <= lim;
+    return (top - neg) <= lim;
   }
 }