#### Re: VFRECIP/VFRSQRT instructions

Bill Huffman

David,

Are the max errors absolute? Or relative to the recip or rsqrt, which
is presumably in the range (1.0, 2.0]?

That you use [0.5, 1] when you might have meant [0.5, 1) leaves some
question about what is happening with the powers of two (even powers for
rsqrt). Hopefully, they're always precise and there's no issue of error
there.

Bill

On 7/31/20 4:33 AM, David Horner wrote:
EXTERNAL MAIL

The current LUT generator assumes N-by-N look up table.

I will load in my github Andrew's program modified to  take input (index
size) and output (estimate number of bits) arguments.
(--verilog still generates LUT, and test generates the values below),

Of course, how well the table can be synthesized is more importance than
LUT dimensions per se.
I have used yosys with varying results.

I continue to try to profile the accuracy within input segments rather
than over total float range.

Other dimensions are possible with resultant increase of decrease in
accuracy:
parameters are: output width / input width  array size

7/7 896
max recip error on [0.5, 1]: 2^-7.36951
max rsqrt error on [0.25, 1]: 2^-7.2998
8/7 1024
max recip error on [0.5, 1]: 2^-7.78083
max rsqrt error on [0.25, 1]: 2^-7.69943
9/7 1152
max recip error on [0.5, 1]: 2^-7.89148
max rsqrt error on [0.25, 1]: 2^-7.8783
10/7 1280
max recip error on [0.5, 1]: 2^-7.94879
max rsqrt error on [0.25, 1]: 2^-7.91063
11/7 1408 by 1
max recip error on [0.5, 1]: 2^-7.97629
max rsqrt error on [0.25, 1]: 2^-8
11/7 1408 by 0x1000
max recip error on [0.5, 1]: 2^-8
max rsqrt error on [0.25, 1]: 2^-8
6/8 1536
max recip error on [0.5, 1]: 2^-6.90724
max rsqrt error on [0.25, 1]: 2^-6.88438
7/8 1792
max recip error on [0.5, 1]: 2^-7.78083
max rsqrt error on [0.25, 1]: 2^-7.73897
8/8 2048 by 0X1000
max recip error on [0.5, 1]: 2^-8.45311
max rsqrt error on [0.25, 1]: 2^-8.36032
10/8 2560 by 0X1000
max recip error on [0.5, 1]: 2^-8.88626
max rsqrt error on [0.25, 1]: 2^-8.8425
10/8 2560 by 0X1
max recip error on [0.5, 1]: 2^-8.88626
max rsqrt error on [0.25, 1]: 2^-8.83164
12/8
max recip error on [0.5, 1]: 2^-8.98041
max rsqrt error on [0.25, 1]: 2^-8.9616
13/8 2560
max recip error on [0.5, 1]: 2^-9
max rsqrt error on [0.25, 1]: 2^-9
9/9
max recip error on [0.5, 1]: 2^-9.47252
max rsqrt error on [0.25, 1]: 2^-9.31953

observation:
7 input bits has a minimum max error of 2^-8.
8 input bits has a minimum max error of 2^-9.

Join {tech-vector-ext@lists.riscv.org to automatically receive all group messages.