bug in generic math-emu

From: schwidefsky@de.ibm.com
Date: Mon Feb 19 2001 - 14:18:02 EST


Hi,
I found a bug in the generic floating point emulation code. The glibc
math testcases showed errors for the nearbyint(0.5) test. The problem
is in the _FP_FRAC_SRS_2 macro. The expression to find out if some 1
bits have been shifted out is wrong. Try the _FP_FRAC_SRS_2 macro
with the following parameters: X_f1 = 0x00800000, X_f0 = 0x00000000,
N = 53, sz = 56. It comes back with 0x0000000000000005 instead of
0x0000000000000004. Now you can do the math ;-)
With the following patch test-float and test-double (both from glibc)
now run without errors on a S/390 without IEEE fpu.

diff -u -r1.1.1.1 op-2.h
--- include/math-emu/op-2.h 2000/02/04 13:53:47 1.1.1.1
+++ include/math-emu/op-2.h 2001/02/19 18:53:41
@@ -79,7 +79,7 @@
     else \
       { \
     X##_f0 = (X##_f1 >> ((N) - _FP_W_TYPE_SIZE) | \
- (((X##_f1 << (sz - (N))) | X##_f0) != 0)); \
+ (((X##_f1 << (2*_FP_W_TYPE_SIZE - (N))) | X##_f0) != 0)); \
     X##_f1 = 0; \
       } \
   } while (0)

blue skies,
   Martin

Linux/390 Design & Development, IBM Deutschland Entwicklung GmbH
Schönaicherstr. 220, D-71032 Böblingen, Telefon: 49 - (0)7031 - 16-2247
E-Mail: schwidefsky@de.ibm.com

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/



This archive was generated by hypermail 2b29 : Fri Feb 23 2001 - 21:00:20 EST