math-emu issue with fp divide

[Kumar Gala]

David,

You seem to be the last person to touch the math-emu code in the  
kernel so I figured you might have some insights.  I'm trying to move  
the powerpc math-emu code to use the include/math-emu bits.

In doing so I've been using TestFloat to see how good or bad we are  
doing.  For the most part the current math-emu code that PPC uses has  
a number of issues that the code in include/math-emu seems to solve  
(plus bugs we've had for ever that no one every realized).

Anyways, I've come across a case that we are flagging underflow and  
inexact because we think we have a denormalized result from a double  
precision divide:

000.FFFFFFFFFFFFF / 3FE.FFFFFFFFFFFFE
	soft: 001.0000000000000 .....  syst: 001.0000000000000 ...ux

What it looks like is the results out of FP_DIV_D are:

D:
sign:	  0
mantissa: 01000000 00000000
exp:	 -1023 (0)

The problem seems like we aren't normalizing the result and bumping  
the exp.  So I was wondering if you had any idea if FP_DIV_D should be  
doing that or if when we call FP_PACK_DP/_FP_PACK_CANONICAL if that  
could should be doing it.

thanks

- k
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/