David,
Are the max errors absolute? Or relative to the recip or rsqrt, which is presumably in the range (1.0, 2.0]?
That you use [0.5, 1] when you might have meant [0.5, 1) leaves some question about what is happening with the powers of two (even powers for rsqrt). Hopefully, they're always precise and there's no issue of error there.
Bill
toggle quoted message
Show quoted text
On 7/31/20 4:33 AM, David Horner wrote: EXTERNAL MAIL
The current LUT generator assumes N-by-N look up table.
I will load in my github Andrew's program modified to take input (index size) and output (estimate number of bits) arguments. (--verilog still generates LUT, and test generates the values below),
Of course, how well the table can be synthesized is more importance than LUT dimensions per se. I have used yosys with varying results.
I continue to try to profile the accuracy within input segments rather than over total float range.
Other dimensions are possible with resultant increase of decrease in accuracy: parameters are: output width / input width array size
7/7 896 max recip error on [0.5, 1]: 2^-7.36951 max rsqrt error on [0.25, 1]: 2^-7.2998 8/7 1024 max recip error on [0.5, 1]: 2^-7.78083 max rsqrt error on [0.25, 1]: 2^-7.69943 9/7 1152 max recip error on [0.5, 1]: 2^-7.89148 max rsqrt error on [0.25, 1]: 2^-7.8783 10/7 1280 max recip error on [0.5, 1]: 2^-7.94879 max rsqrt error on [0.25, 1]: 2^-7.91063 11/7 1408 by 1 max recip error on [0.5, 1]: 2^-7.97629 max rsqrt error on [0.25, 1]: 2^-8 11/7 1408 by 0x1000 max recip error on [0.5, 1]: 2^-8 max rsqrt error on [0.25, 1]: 2^-8 6/8 1536 max recip error on [0.5, 1]: 2^-6.90724 max rsqrt error on [0.25, 1]: 2^-6.88438 7/8 1792 max recip error on [0.5, 1]: 2^-7.78083 max rsqrt error on [0.25, 1]: 2^-7.73897 8/8 2048 by 0X1000 max recip error on [0.5, 1]: 2^-8.45311 max rsqrt error on [0.25, 1]: 2^-8.36032 10/8 2560 by 0X1000 max recip error on [0.5, 1]: 2^-8.88626 max rsqrt error on [0.25, 1]: 2^-8.8425 10/8 2560 by 0X1 max recip error on [0.5, 1]: 2^-8.88626 max rsqrt error on [0.25, 1]: 2^-8.83164 12/8 max recip error on [0.5, 1]: 2^-8.98041 max rsqrt error on [0.25, 1]: 2^-8.9616 13/8 2560 max recip error on [0.5, 1]: 2^-9 max rsqrt error on [0.25, 1]: 2^-9 9/9 max recip error on [0.5, 1]: 2^-9.47252 max rsqrt error on [0.25, 1]: 2^-9.31953
observation: 7 input bits has a minimum max error of 2^-8. 8 input bits has a minimum max error of 2^-9.
|