Date   

Re: I have some questions about the VMADC/VMSBC instructions, thank you for your valuable comments.

Andrew Waterman
 



On Thu, Jun 16, 2022 at 2:43 AM <lilei2@...> wrote:
1. Question for tail bits of mask-producing instructions.
In the case of mask-producing instructions, tail elements are the bits with (vl <= bit index < VLEN).So according to riscv-v-spec-1.0, page 46, VMADC/VMSBC instructions operate with tail-agnostic policy, which means the tail bits can be written with 1 or unchanged. 
While according to page 13, we have a more relax constraint: except for mask load instructions, any element in the tail of a mask result can also be written with the value the mask-producing operation would have calculated with vl=VLMAX. Which means we can overwritten all remaining bits that past vl with 1s or with the value the mask-producing operation would have calculated.
For example, if VLEN=128, LMUL=1, SEW=32, there are only 4 body bits for VMADC instruction. If current vl=2, we can write the calculation results to bit[1:0], and write all other 126 bits with 1s.

It is legal to fill bits vl..VLEN-1 with 1s because of the clause that these instructions are always tail-agnostic.

Or we can write 4 bits calculation results to bit[3:0] in which only bit[1:0] are body bits, and write all other 124 bits with 1s.

It is also legal to compute bits 0..VLMAX-1 (as a function of elements 0..VLMAX-1), then fill bits VLMAX..VLEN-1 with 1s, because of the clause that mask-producing instructions are permitted to write the result they would have written if vl had been set to VLMAX.

But I'm not sure that the remark "in which only bit[1:0] are body bits" matters.  In this style of implementation, the behavior is the same as if the body contained elements 0..VLMAX-1.

Whether either of these implementations is legal?
 
2. Question for inactive body bits of mask-producing instructions.
When vtype.vma=0, which means mask-undisturbed policy, the inactive body bits should retain its value. 
For example, when vma=0, the VMSBF_M and VMSEQ with vm=0, should not change the inactive body bits.
Is my understand correct?thanks.

Right.


I have some questions about the VMADC/VMSBC instructions, thank you for your valuable comments.

lilei2@...
 

1. Question for tail bits of mask-producing instructions.
In the case of mask-producing instructions, tail elements are the bits with (vl <= bit index < VLEN).So according to riscv-v-spec-1.0, page 46, VMADC/VMSBC instructions operate with tail-agnostic policy, which means the tail bits can be written with 1 or unchanged. 
While according to page 13, we have a more relax constraint: except for mask load instructions, any element in the tail of a mask result can also be written with the value the mask-producing operation would have calculated with vl=VLMAX. Which means we can overwritten all remaining bits that past vl with 1s or with the value the mask-producing operation would have calculated.
For example, if VLEN=128, LMUL=1, SEW=32, there are only 4 body bits for VMADC instruction. If current vl=2, we can write the calculation results to bit[1:0], and write all other 126 bits with 1s. Or we can write 4 bits calculation results to bit[3:0] in which only bit[1:0] are body bits, and write all other 124 bits with 1s.
Whether either of these implementations is legal?
 
2. Question for inactive body bits of mask-producing instructions.
When vtype.vma=0, which means mask-undisturbed policy, the inactive body bits should retain its value. 
For example, when vma=0, the VMSBF_M and VMSEQ with vm=0, should not change the inactive body bits.
Is my understand correct?thanks.


Re: Zvediv extension discussions

Abel Bernabeu
 

Sorry, I have followed the thread but COVID kept me busy here.

Zvediv is not strictly needed for matrix multiply. I have to correct Peter Lieber who interpreted that from what we talked at the Graphics and ML SIG and said so in this thread.

What I tried to communicate on the Graphics and ML SIG was that: If Zvediv was reintroduced then I would suggest a different behaviour for matrix multiply depending on whether a source operand (left hand side or right hand side matrix) is the same for all the work items or not.

About Zvediv itself (original subject), the value I see is that one can perform several work-items in parallel and change the number of processed elements dynamically.

If there is a reduce operation per work-item, without Zvediv we need one reduce instruction per work-item. If the number of work-items changes, the program has to be recompiled accordingly. So the number of work-items becomes a non-orthogonal state of the program and needs to be recompiled every time this parameter changes.

In some graphics designs I worked with in the past (like Intel's Gen) that implies having different shaders for different patch shapes (8x8, 8x4, 8x2, 8x1 and all the transpositions). Ideally I would like the same shader code to be valid for every possible patch shape: write the shader binary once and use it with any present and future patch size.

Being patch-size agnostic is as beneficial for graphics as being vector-length agnostic is for other domains. And that is the motivation behind reintroducing Zvediv.

Regards.

PD: OoO is not a high priority for graphics or ML. I would be perfectly happy if OoO is not possible for any instruction on a shading core for graphics.


On Wed, Feb 9, 2022 at 8:50 AM Victor Moya <victor.moya@...> wrote:

I have serious doubts Zvediv helps with graphics. My previous experience is that trying to force SIMD4 over vectors doesn't really help nor does optimize hardware usage. Every major vendor moved away from it. My experience is that you get the same or better performance by just unrolling the color (or coordinate for geometry) component loop on flat vectors. So I would need to look at the actual use cases and if these use cases are really relevant for modern 3D graphic workloads.

My experience with ML is somewhat more reduced but I don't see any major issue with the current vector definition (other than the still missing float16 and bfloat16 from the spec). We are also interested in dot products or similar operations that increase compute density for matrix multiplication but the hardware cost versus performance improvement needs to be managed carefully.

If you add 2D shape to a vector there are hardware costs related with register file access and/or the crossbar between computing lanes. So I would like to see the use cases and how they help. I don't have a closed opinion on this topic.

Talking from experience, from a hardware perspective the current V extension is already somewhat challenging when you add OoO so better don't add extra complexity that doesn't add real performance :).

I expect the SIG will work on these proposals and I will be around to collaborate.

Victor


On Wed, Feb 9, 2022 at 7:25 AM Krste Asanovic <krste@...> wrote:

The current vector TG is only tasked with one more deliverable, which
is the IEEE FP16 half-precision extensions (Zvfh, Zvfhmin).  This is
effectively already defined in ratified specification, and so is just
a question of ratifying these extension names and the detailed
contents of each extension.

There is also Zvamo, which was dropped due to conflict with scalar
subword AMO encoding space, but which will need a non-trivial redesign
and hence should be a new TG.  It is probably not a high priority
extension compared to others.

Zvediv was only sketched out during the vector development process,
and is not clear to me (as the original creator) that this is the
right way to go for general matrix operations.

Another thing to be aware of is that from TSC, vector crypto has the
highest priority among any new vector extensions.  Vector BF16 support
is another TSC priority.

There are a lot of ideas in this thread, but there is a lot of work to
define something that will also work well on OoO cores.  The SIG is
probably a good place to work through some ideas and arrive at more
concrete proposals that can lead to a TG.

Krste


>>>>> On Fri, 4 Feb 2022 11:00:23 -0800, "Guy Lemieux" <guy.lemieux@...> said:

| Great points Ken & Earl.
| One thing I'll point out is that this does not necessarily have much to do with EDIV specifically.

| For example, the main goal of EDIV is to support smaller element dot products, particularly for integers. This helps with ML inference, sure. But it leaves a lot of performance on the
| table, may not help much with other operations than dot products, and probably won't help much with other applications (including ML training).

| There are two angles that would supersede this in capability and performance:
| -- adding vector shapes
| -- adding accelerators to the vector unit
| -- in particular, the combination of both shapes + accelerators

| In my work (embodied primarily by the VectorBlox MXP architecture) , these were its two primary advantages.

| a) allow streaming accelerators to replace the regular ALUs. in particular, you can add a systolic array, where you have N^2 PEs and only require O(N) inputs on each edge and produce O
| (N) outputs. such accelerators can be standardized, but first the ISA interface needs to be standardized (see below) and possibly the physical interface (eg, AXI-stream).

| b) allow vector operands to have shapes, in particular to allow tensor shapes. this affects data readout ordering and writeback ordering when connecting to the accelerator. this would
| allow, for example, a traditional 1D vector to be interpreted as a 2D tile, or even a 2D subtile. this affects the address generators to the register file, and may require data swizzling
| to rearrange into the proper shapes and to avoid data bubbles.

| In addition to the above, we loosely organized our register file as a multi-banked scratchpad, rather than having fixed-size disjoint registers. This allowed a named vector register to
| be replaced by a scalar address (a pointer) which fits in the scalar register file. This allowed vectors of arbitrary length, and to start at arbitrary locations, producing much more
| flexible shapes and subshapes to be read out. This property is probably too much for people to accept right away, but it is needed when you want to have maximum flexibility for both (a)
| and (b) above.

| Note that none of this has anything specifically to do with EDIV. However, it could build upon the vtypes system that Krste has devised. (He previously tried to suggest matrix shapes as
| possible vtype. In his suggestion, each vector register had its own type descriptor; in the MXP architecture, the type descriptor is global like the vector length register but there are
| 3 copies of them so src1, src2 and dst can all be different shapes.)

| Guy

| On Fri, Feb 4, 2022 at 10:27 AM Ken Dockser <kad@...> wrote:

|     Thanks folks, these are all very good points.

|     Earl: I absolutely agree that these extensions (like all RISC-V extensions) need to be developed based on real-world needs and need to be able to show their value (including utility
|     and usability),  as well as value/cost ratio.

|     Guy: I agree that we need to look at the other leading architectures as we are well behind them in this area. We then need to come up with our own solutions that address the market
|     needs and fit within the RISC-V approach and philosophy. 

|     Peter: Yes, we need to work to create extensions that take into account our future needs and intentions. In this case, where we are talking about adding instructions that improve
|     matrix performance in the vector registers, we need to keep in mind how this might fit with future extensions that operate on matrices.

|     We still need to figure out how we can effectively and efficiently take this next step in RISC-V Vector.  It seems like the best approach would be to leverage the existing Vector TG
|     by producing an updated charter that is focused on completing the Zvediv extension. Is this permitted/possible? Are the current Chair and Vice Chair amenable to this?

|     Thanks,
|     Ken

|     On Thu, Feb 3, 2022 at 10:11 PM Guy Lemieux <guy.lemieux@...> wrote:

|         The AMX extension for AVX512 is an interesting approach.... 

|         https://en.wikichip.org/wiki/intel/dl_boost

|         On Thu, Feb 3, 2022 at 8:02 PM Earl Killian <earl.killian@...> wrote:

|             I hope that these discussions will begin with algorithms and applications that need the additional performance, and proceed to analyze how proposed instructions address the
|             greater need.

|                 On Feb 3, 2022, at 14:45, Peter Lieber <peteralieber@...> wrote:

|                 In the Graphics/ML SIG, we also discussed matrix operations as well.  We talked about a single matrix opcode with various functions like vector-matrix, matrix-matrix, and
|                 dot product functions, among others.  Just some initial ideas... 

|                 The Zvediv extension would be a great start to get dimensionality to vectors, and we would want to keep matrices in mind when reviving the effort.

|                 Regards,
|                 Peter Lieber

|                 On Thu, Feb 3, 2022 at 2:17 PM Ken Dockser <kad@...> wrote:

|                     I am hearing renewed interest in adding a dot-product extension to the RISC-V Vectors. This includes everything from adding a handful of FP and Int dot-product
|                     instructions all the way to completing the Zvediv extension.

|                     What is the most efficient way to revive the efforts in these areas? Can we reconvene the Vector TG meetings?

|                     Thanks,
|                     Ken

|






Re: Zvediv extension discussions

Victor Moya
 


I have serious doubts Zvediv helps with graphics. My previous experience is that trying to force SIMD4 over vectors doesn't really help nor does optimize hardware usage. Every major vendor moved away from it. My experience is that you get the same or better performance by just unrolling the color (or coordinate for geometry) component loop on flat vectors. So I would need to look at the actual use cases and if these use cases are really relevant for modern 3D graphic workloads.

My experience with ML is somewhat more reduced but I don't see any major issue with the current vector definition (other than the still missing float16 and bfloat16 from the spec). We are also interested in dot products or similar operations that increase compute density for matrix multiplication but the hardware cost versus performance improvement needs to be managed carefully.

If you add 2D shape to a vector there are hardware costs related with register file access and/or the crossbar between computing lanes. So I would like to see the use cases and how they help. I don't have a closed opinion on this topic.

Talking from experience, from a hardware perspective the current V extension is already somewhat challenging when you add OoO so better don't add extra complexity that doesn't add real performance :).

I expect the SIG will work on these proposals and I will be around to collaborate.

Victor


On Wed, Feb 9, 2022 at 7:25 AM Krste Asanovic <krste@...> wrote:

The current vector TG is only tasked with one more deliverable, which
is the IEEE FP16 half-precision extensions (Zvfh, Zvfhmin).  This is
effectively already defined in ratified specification, and so is just
a question of ratifying these extension names and the detailed
contents of each extension.

There is also Zvamo, which was dropped due to conflict with scalar
subword AMO encoding space, but which will need a non-trivial redesign
and hence should be a new TG.  It is probably not a high priority
extension compared to others.

Zvediv was only sketched out during the vector development process,
and is not clear to me (as the original creator) that this is the
right way to go for general matrix operations.

Another thing to be aware of is that from TSC, vector crypto has the
highest priority among any new vector extensions.  Vector BF16 support
is another TSC priority.

There are a lot of ideas in this thread, but there is a lot of work to
define something that will also work well on OoO cores.  The SIG is
probably a good place to work through some ideas and arrive at more
concrete proposals that can lead to a TG.

Krste


>>>>> On Fri, 4 Feb 2022 11:00:23 -0800, "Guy Lemieux" <guy.lemieux@...> said:

| Great points Ken & Earl.
| One thing I'll point out is that this does not necessarily have much to do with EDIV specifically.

| For example, the main goal of EDIV is to support smaller element dot products, particularly for integers. This helps with ML inference, sure. But it leaves a lot of performance on the
| table, may not help much with other operations than dot products, and probably won't help much with other applications (including ML training).

| There are two angles that would supersede this in capability and performance:
| -- adding vector shapes
| -- adding accelerators to the vector unit
| -- in particular, the combination of both shapes + accelerators

| In my work (embodied primarily by the VectorBlox MXP architecture) , these were its two primary advantages.

| a) allow streaming accelerators to replace the regular ALUs. in particular, you can add a systolic array, where you have N^2 PEs and only require O(N) inputs on each edge and produce O
| (N) outputs. such accelerators can be standardized, but first the ISA interface needs to be standardized (see below) and possibly the physical interface (eg, AXI-stream).

| b) allow vector operands to have shapes, in particular to allow tensor shapes. this affects data readout ordering and writeback ordering when connecting to the accelerator. this would
| allow, for example, a traditional 1D vector to be interpreted as a 2D tile, or even a 2D subtile. this affects the address generators to the register file, and may require data swizzling
| to rearrange into the proper shapes and to avoid data bubbles.

| In addition to the above, we loosely organized our register file as a multi-banked scratchpad, rather than having fixed-size disjoint registers. This allowed a named vector register to
| be replaced by a scalar address (a pointer) which fits in the scalar register file. This allowed vectors of arbitrary length, and to start at arbitrary locations, producing much more
| flexible shapes and subshapes to be read out. This property is probably too much for people to accept right away, but it is needed when you want to have maximum flexibility for both (a)
| and (b) above.

| Note that none of this has anything specifically to do with EDIV. However, it could build upon the vtypes system that Krste has devised. (He previously tried to suggest matrix shapes as
| possible vtype. In his suggestion, each vector register had its own type descriptor; in the MXP architecture, the type descriptor is global like the vector length register but there are
| 3 copies of them so src1, src2 and dst can all be different shapes.)

| Guy

| On Fri, Feb 4, 2022 at 10:27 AM Ken Dockser <kad@...> wrote:

|     Thanks folks, these are all very good points.

|     Earl: I absolutely agree that these extensions (like all RISC-V extensions) need to be developed based on real-world needs and need to be able to show their value (including utility
|     and usability),  as well as value/cost ratio.

|     Guy: I agree that we need to look at the other leading architectures as we are well behind them in this area. We then need to come up with our own solutions that address the market
|     needs and fit within the RISC-V approach and philosophy. 

|     Peter: Yes, we need to work to create extensions that take into account our future needs and intentions. In this case, where we are talking about adding instructions that improve
|     matrix performance in the vector registers, we need to keep in mind how this might fit with future extensions that operate on matrices.

|     We still need to figure out how we can effectively and efficiently take this next step in RISC-V Vector.  It seems like the best approach would be to leverage the existing Vector TG
|     by producing an updated charter that is focused on completing the Zvediv extension. Is this permitted/possible? Are the current Chair and Vice Chair amenable to this?

|     Thanks,
|     Ken

|     On Thu, Feb 3, 2022 at 10:11 PM Guy Lemieux <guy.lemieux@...> wrote:

|         The AMX extension for AVX512 is an interesting approach.... 

|         https://en.wikichip.org/wiki/intel/dl_boost

|         On Thu, Feb 3, 2022 at 8:02 PM Earl Killian <earl.killian@...> wrote:

|             I hope that these discussions will begin with algorithms and applications that need the additional performance, and proceed to analyze how proposed instructions address the
|             greater need.

|                 On Feb 3, 2022, at 14:45, Peter Lieber <peteralieber@...> wrote:

|                 In the Graphics/ML SIG, we also discussed matrix operations as well.  We talked about a single matrix opcode with various functions like vector-matrix, matrix-matrix, and
|                 dot product functions, among others.  Just some initial ideas... 

|                 The Zvediv extension would be a great start to get dimensionality to vectors, and we would want to keep matrices in mind when reviving the effort.

|                 Regards,
|                 Peter Lieber

|                 On Thu, Feb 3, 2022 at 2:17 PM Ken Dockser <kad@...> wrote:

|                     I am hearing renewed interest in adding a dot-product extension to the RISC-V Vectors. This includes everything from adding a handful of FP and Int dot-product
|                     instructions all the way to completing the Zvediv extension.

|                     What is the most efficient way to revive the efforts in these areas? Can we reconvene the Vector TG meetings?

|                     Thanks,
|                     Ken

|






Re: Zvediv extension discussions

Krste Asanovic
 

The current vector TG is only tasked with one more deliverable, which
is the IEEE FP16 half-precision extensions (Zvfh, Zvfhmin). This is
effectively already defined in ratified specification, and so is just
a question of ratifying these extension names and the detailed
contents of each extension.

There is also Zvamo, which was dropped due to conflict with scalar
subword AMO encoding space, but which will need a non-trivial redesign
and hence should be a new TG. It is probably not a high priority
extension compared to others.

Zvediv was only sketched out during the vector development process,
and is not clear to me (as the original creator) that this is the
right way to go for general matrix operations.

Another thing to be aware of is that from TSC, vector crypto has the
highest priority among any new vector extensions. Vector BF16 support
is another TSC priority.

There are a lot of ideas in this thread, but there is a lot of work to
define something that will also work well on OoO cores. The SIG is
probably a good place to work through some ideas and arrive at more
concrete proposals that can lead to a TG.

Krste


On Fri, 4 Feb 2022 11:00:23 -0800, "Guy Lemieux" <guy.lemieux@...> said:
| Great points Ken & Earl.
| One thing I'll point out is that this does not necessarily have much to do with EDIV specifically.

| For example, the main goal of EDIV is to support smaller element dot products, particularly for integers. This helps with ML inference, sure. But it leaves a lot of performance on the
| table, may not help much with other operations than dot products, and probably won't help much with other applications (including ML training).

| There are two angles that would supersede this in capability and performance:
| -- adding vector shapes
| -- adding accelerators to the vector unit
| -- in particular, the combination of both shapes + accelerators

| In my work (embodied primarily by the VectorBlox MXP architecture) , these were its two primary advantages.

| a) allow streaming accelerators to replace the regular ALUs. in particular, you can add a systolic array, where you have N^2 PEs and only require O(N) inputs on each edge and produce O
| (N) outputs. such accelerators can be standardized, but first the ISA interface needs to be standardized (see below) and possibly the physical interface (eg, AXI-stream).

| b) allow vector operands to have shapes, in particular to allow tensor shapes. this affects data readout ordering and writeback ordering when connecting to the accelerator. this would
| allow, for example, a traditional 1D vector to be interpreted as a 2D tile, or even a 2D subtile. this affects the address generators to the register file, and may require data swizzling
| to rearrange into the proper shapes and to avoid data bubbles.

| In addition to the above, we loosely organized our register file as a multi-banked scratchpad, rather than having fixed-size disjoint registers. This allowed a named vector register to
| be replaced by a scalar address (a pointer) which fits in the scalar register file. This allowed vectors of arbitrary length, and to start at arbitrary locations, producing much more
| flexible shapes and subshapes to be read out. This property is probably too much for people to accept right away, but it is needed when you want to have maximum flexibility for both (a)
| and (b) above.

| Note that none of this has anything specifically to do with EDIV. However, it could build upon the vtypes system that Krste has devised. (He previously tried to suggest matrix shapes as
| possible vtype. In his suggestion, each vector register had its own type descriptor; in the MXP architecture, the type descriptor is global like the vector length register but there are
| 3 copies of them so src1, src2 and dst can all be different shapes.)

| Guy

| On Fri, Feb 4, 2022 at 10:27 AM Ken Dockser <kad@...> wrote:

| Thanks folks, these are all very good points.

| Earl: I absolutely agree that these extensions (like all RISC-V extensions) need to be developed based on real-world needs and need to be able to show their value (including utility
| and usability),  as well as value/cost ratio.

| Guy: I agree that we need to look at the other leading architectures as we are well behind them in this area. We then need to come up with our own solutions that address the market
| needs and fit within the RISC-V approach and philosophy. 

| Peter: Yes, we need to work to create extensions that take into account our future needs and intentions. In this case, where we are talking about adding instructions that improve
| matrix performance in the vector registers, we need to keep in mind how this might fit with future extensions that operate on matrices.

| We still need to figure out how we can effectively and efficiently take this next step in RISC-V Vector.  It seems like the best approach would be to leverage the existing Vector TG
| by producing an updated charter that is focused on completing the Zvediv extension. Is this permitted/possible? Are the current Chair and Vice Chair amenable to this?

| Thanks,
| Ken

| On Thu, Feb 3, 2022 at 10:11 PM Guy Lemieux <guy.lemieux@...> wrote:

| The AMX extension for AVX512 is an interesting approach.... 

| https://en.wikichip.org/wiki/intel/dl_boost

| On Thu, Feb 3, 2022 at 8:02 PM Earl Killian <earl.killian@...> wrote:

| I hope that these discussions will begin with algorithms and applications that need the additional performance, and proceed to analyze how proposed instructions address the
| greater need.

| On Feb 3, 2022, at 14:45, Peter Lieber <peteralieber@...> wrote:

| In the Graphics/ML SIG, we also discussed matrix operations as well.  We talked about a single matrix opcode with various functions like vector-matrix, matrix-matrix, and
| dot product functions, among others.  Just some initial ideas... 

| The Zvediv extension would be a great start to get dimensionality to vectors, and we would want to keep matrices in mind when reviving the effort.

| Regards,
| Peter Lieber

| On Thu, Feb 3, 2022 at 2:17 PM Ken Dockser <kad@...> wrote:

| I am hearing renewed interest in adding a dot-product extension to the RISC-V Vectors. This includes everything from adding a handful of FP and Int dot-product
| instructions all the way to completing the Zvediv extension.

| What is the most efficient way to revive the efforts in these areas? Can we reconvene the Vector TG meetings?

| Thanks,
| Ken

|


Re: Zvediv extension discussions

Guy Lemieux
 

Great points Ken & Earl.

One thing I'll point out is that this does not necessarily have much to do with EDIV specifically.

For example, the main goal of EDIV is to support smaller element dot products, particularly for integers. This helps with ML inference, sure. But it leaves a lot of performance on the table, may not help much with other operations than dot products, and probably won't help much with other applications (including ML training).

There are two angles that would supersede this in capability and performance:
-- adding vector shapes
-- adding accelerators to the vector unit
-- in particular, the combination of both shapes + accelerators

In my work (embodied primarily by the VectorBlox MXP architecture) , these were its two primary advantages.

a) allow streaming accelerators to replace the regular ALUs. in particular, you can add a systolic array, where you have N^2 PEs and only require O(N) inputs on each edge and produce O(N) outputs. such accelerators can be standardized, but first the ISA interface needs to be standardized (see below) and possibly the physical interface (eg, AXI-stream).

b) allow vector operands to have shapes, in particular to allow tensor shapes. this affects data readout ordering and writeback ordering when connecting to the accelerator. this would allow, for example, a traditional 1D vector to be interpreted as a 2D tile, or even a 2D subtile. this affects the address generators to the register file, and may require data swizzling to rearrange into the proper shapes and to avoid data bubbles.

In addition to the above, we loosely organized our register file as a multi-banked scratchpad, rather than having fixed-size disjoint registers. This allowed a named vector register to be replaced by a scalar address (a pointer) which fits in the scalar register file. This allowed vectors of arbitrary length, and to start at arbitrary locations, producing much more flexible shapes and subshapes to be read out. This property is probably too much for people to accept right away, but it is needed when you want to have maximum flexibility for both (a) and (b) above.

Note that none of this has anything specifically to do with EDIV. However, it could build upon the vtypes system that Krste has devised. (He previously tried to suggest matrix shapes as possible vtype. In his suggestion, each vector register had its own type descriptor; in the MXP architecture, the type descriptor is global like the vector length register but there are 3 copies of them so src1, src2 and dst can all be different shapes.)

Guy



On Fri, Feb 4, 2022 at 10:27 AM Ken Dockser <kad@...> wrote:
Thanks folks, these are all very good points.

Earl: I absolutely agree that these extensions (like all RISC-V extensions) need to be developed based on real-world needs and need to be able to show their value (including utility and usability),  as well as value/cost ratio.

Guy: I agree that we need to look at the other leading architectures as we are well behind them in this area. We then need to come up with our own solutions that address the market needs and fit within the RISC-V approach and philosophy. 

Peter: Yes, we need to work to create extensions that take into account our future needs and intentions. In this case, where we are talking about adding instructions that improve matrix performance in the vector registers, we need to keep in mind how this might fit with future extensions that operate on matrices.

We still need to figure out how we can effectively and efficiently take this next step in RISC-V Vector.  It seems like the best approach would be to leverage the existing Vector TG by producing an updated charter that is focused on completing the Zvediv extension. Is this permitted/possible? Are the current Chair and Vice Chair amenable to this?

Thanks,
Ken

On Thu, Feb 3, 2022 at 10:11 PM Guy Lemieux <guy.lemieux@...> wrote:
The AMX extension for AVX512 is an interesting approach.... 


On Thu, Feb 3, 2022 at 8:02 PM Earl Killian <earl.killian@...> wrote:
I hope that these discussions will begin with algorithms and applications that need the additional performance, and proceed to analyze how proposed instructions address the greater need.

On Feb 3, 2022, at 14:45, Peter Lieber <peteralieber@...> wrote:

In the Graphics/ML SIG, we also discussed matrix operations as well.  We talked about a single matrix opcode with various functions like vector-matrix, matrix-matrix, and dot product functions, among others.  Just some initial ideas... 

The Zvediv extension would be a great start to get dimensionality to vectors, and we would want to keep matrices in mind when reviving the effort.

Regards,
Peter Lieber


On Thu, Feb 3, 2022 at 2:17 PM Ken Dockser <kad@...> wrote:
I am hearing renewed interest in adding a dot-product extension to the RISC-V Vectors. This includes everything from adding a handful of FP and Int dot-product instructions all the way to completing the Zvediv extension.

What is the most efficient way to revive the efforts in these areas? Can we reconvene the Vector TG meetings?

Thanks,
Ken




Re: Zvediv extension discussions

Ken Dockser
 

Thanks folks, these are all very good points.

Earl: I absolutely agree that these extensions (like all RISC-V extensions) need to be developed based on real-world needs and need to be able to show their value (including utility and usability),  as well as value/cost ratio.

Guy: I agree that we need to look at the other leading architectures as we are well behind them in this area. We then need to come up with our own solutions that address the market needs and fit within the RISC-V approach and philosophy. 

Peter: Yes, we need to work to create extensions that take into account our future needs and intentions. In this case, where we are talking about adding instructions that improve matrix performance in the vector registers, we need to keep in mind how this might fit with future extensions that operate on matrices.

We still need to figure out how we can effectively and efficiently take this next step in RISC-V Vector.  It seems like the best approach would be to leverage the existing Vector TG by producing an updated charter that is focused on completing the Zvediv extension. Is this permitted/possible? Are the current Chair and Vice Chair amenable to this?

Thanks,
Ken

On Thu, Feb 3, 2022 at 10:11 PM Guy Lemieux <guy.lemieux@...> wrote:
The AMX extension for AVX512 is an interesting approach.... 


On Thu, Feb 3, 2022 at 8:02 PM Earl Killian <earl.killian@...> wrote:
I hope that these discussions will begin with algorithms and applications that need the additional performance, and proceed to analyze how proposed instructions address the greater need.

On Feb 3, 2022, at 14:45, Peter Lieber <peteralieber@...> wrote:

In the Graphics/ML SIG, we also discussed matrix operations as well.  We talked about a single matrix opcode with various functions like vector-matrix, matrix-matrix, and dot product functions, among others.  Just some initial ideas... 

The Zvediv extension would be a great start to get dimensionality to vectors, and we would want to keep matrices in mind when reviving the effort.

Regards,
Peter Lieber


On Thu, Feb 3, 2022 at 2:17 PM Ken Dockser <kad@...> wrote:
I am hearing renewed interest in adding a dot-product extension to the RISC-V Vectors. This includes everything from adding a handful of FP and Int dot-product instructions all the way to completing the Zvediv extension.

What is the most efficient way to revive the efforts in these areas? Can we reconvene the Vector TG meetings?

Thanks,
Ken




Re: Zvediv extension discussions

Guy Lemieux
 

The AMX extension for AVX512 is an interesting approach.... 


On Thu, Feb 3, 2022 at 8:02 PM Earl Killian <earl.killian@...> wrote:
I hope that these discussions will begin with algorithms and applications that need the additional performance, and proceed to analyze how proposed instructions address the greater need.

On Feb 3, 2022, at 14:45, Peter Lieber <peteralieber@...> wrote:

In the Graphics/ML SIG, we also discussed matrix operations as well.  We talked about a single matrix opcode with various functions like vector-matrix, matrix-matrix, and dot product functions, among others.  Just some initial ideas... 

The Zvediv extension would be a great start to get dimensionality to vectors, and we would want to keep matrices in mind when reviving the effort.

Regards,
Peter Lieber


On Thu, Feb 3, 2022 at 2:17 PM Ken Dockser <kad@...> wrote:
I am hearing renewed interest in adding a dot-product extension to the RISC-V Vectors. This includes everything from adding a handful of FP and Int dot-product instructions all the way to completing the Zvediv extension.

What is the most efficient way to revive the efforts in these areas? Can we reconvene the Vector TG meetings?

Thanks,
Ken




Re: Zvediv extension discussions

Earl Killian
 

I hope that these discussions will begin with algorithms and applications that need the additional performance, and proceed to analyze how proposed instructions address the greater need.

On Feb 3, 2022, at 14:45, Peter Lieber <peteralieber@...> wrote:

In the Graphics/ML SIG, we also discussed matrix operations as well.  We talked about a single matrix opcode with various functions like vector-matrix, matrix-matrix, and dot product functions, among others.  Just some initial ideas... 

The Zvediv extension would be a great start to get dimensionality to vectors, and we would want to keep matrices in mind when reviving the effort.

Regards,
Peter Lieber


On Thu, Feb 3, 2022 at 2:17 PM Ken Dockser <kad@...> wrote:
I am hearing renewed interest in adding a dot-product extension to the RISC-V Vectors. This includes everything from adding a handful of FP and Int dot-product instructions all the way to completing the Zvediv extension.

What is the most efficient way to revive the efforts in these areas? Can we reconvene the Vector TG meetings?

Thanks,
Ken




Re: Zvediv extension discussions

Peter Lieber
 

In the Graphics/ML SIG, we also discussed matrix operations as well.  We talked about a single matrix opcode with various functions like vector-matrix, matrix-matrix, and dot product functions, among others.  Just some initial ideas... 

The Zvediv extension would be a great start to get dimensionality to vectors, and we would want to keep matrices in mind when reviving the effort.

Regards,
Peter Lieber


On Thu, Feb 3, 2022 at 2:17 PM Ken Dockser <kad@...> wrote:
I am hearing renewed interest in adding a dot-product extension to the RISC-V Vectors. This includes everything from adding a handful of FP and Int dot-product instructions all the way to completing the Zvediv extension.

What is the most efficient way to revive the efforts in these areas? Can we reconvene the Vector TG meetings?

Thanks,
Ken


Zvediv extension discussions

Ken Dockser
 

I am hearing renewed interest in adding a dot-product extension to the RISC-V Vectors. This includes everything from adding a handful of FP and Int dot-product instructions all the way to completing the Zvediv extension.

What is the most efficient way to revive the efforts in these areas? Can we reconvene the Vector TG meetings?

Thanks,
Ken


Re: chapter 7.8. Vector Load/Store Segment Instructions

Artem Zhdanov
 

Hello!

As far as I understand, there is no separate field for decoding a segment instruction other than NF. Therefore, those instructions are considered segmented if NFIELDS > 1 (nf != 0).


01.02.2022, 10:54, "Alexander Podoplelov" <alexander.podoplelov@...>:

Hello!

I have a question about vector segment load and stores.

In table 14 we have NFIELDS from 1 to 8.

In paragraphes 7.8.1-3 we have format like

vlseg<nf>e<eew>.v vd, (rs1), vm

vsseg<nf>e<eew>.v vs3, (rs1), vm

From specification it is not clear for me

Is it possible to have instruction like vlseg1e8.v vd, (rs1), vm

This question is about all vector segment load and stores.

Right now assembly do not know opcodes for these instructions.

Despite of there is no any sense of using vlseg1e8.v vd, (rs1), vm (please, correct me if I wrong) I suppose it is need to be noted somewhere about supporting / not supporting these opcodes.

Best Regards, Aleksandr Podoplelov


chapter 7.8. Vector Load/Store Segment Instructions

Alexander Podoplelov
 

Hello!

I have a question about vector segment load and stores.

In table 14 we have NFIELDS from 1 to 8.

In paragraphes 7.8.1-3 we have format like

vlseg<nf>e<eew>.v vd, (rs1), vm

vsseg<nf>e<eew>.v vs3, (rs1), vm

From specification it is not clear for me

Is it possible to have instruction like vlseg1e8.v vd, (rs1), vm

This question is about all vector segment load and stores.

Right now assembly do not know opcodes for these instructions.

Despite of there is no any sense of using vlseg1e8.v vd, (rs1), vm (please, correct me if I wrong) I suppose it is need to be noted somewhere about supporting / not supporting these opcodes.

Best Regards, Aleksandr Podoplelov


about masked-off bits for instructions vmsbf.m, vmsif.m, vmsof.m #defines

lilei2@...
 

Hi,
I have a question about masked-off bits. 
I am not sure what is the behavior of destination inactive masked-off bits for instructions vmsbf.m, vmsif.m, vmsof.m. Does the "xxxx" means we can fill any value to these bits, regardless of vtype.vma? 
I copied the example codes below, which is from section 15.4 of RVV spec 1.0 frozen:
1 1 0 0 0 0 1 1 v0 vcontents
1 0 0 1 0 1 0 0 v3 contents
                       vmsbf.m v2, v3, v0.t
0 1 x x x x 1 1  v2 contents
 
In addition, whether all mask-result instructions need to fill the mask-off bits according to the vtype.vma policy, such as vector integer compare instructions?
And is it allowed that the implementation choose to only support mask-agnostic and tail-agnostic for mask-result instructions?
Thanks.


Re: The Width of vcsr and vstart

Krste Asanovic
 

Thanks for spotting the oversight. 
The spec was updated to indicate these should be treated as XLEN-bit wide registers.

There is no effective difference right now given that upper bits are not currently defined, but there may be some use for >32 bits in vcsr in some distant future. Vstart could also acquire extra exception state in some distant future.

Krste


On Dec 16, 2021, at 1:32 AM, Andrew Waterman <andrew@...> wrote:

For the current V extension, it's correct to treat both vcsr and vstart as 32-bit registers.  I agree the spec should clearly indicate whether or not these registers will always be 32 bits (like fcsr).


On Wed, Dec 15, 2021 at 6:31 PM Tianyi Xia via lists.riscv.org <tianshi.xty=alibaba-inc.com@...> wrote:
Hi,all
Debuggers can use abstract commands to access CSRs. When using abstract commands, debuggers need to specify the bit width of 
CSRs. The bit widths of vcsr and vstart are not clearly defined in Vector Extension version1.0. In an RV64 implementation, The debugger is not clear whether the bit width of these two CSRs should be regarded as 32 or 64. May be we need specify the bit width of these CSRs in the spec, XLEN or fixed-length 32bit.
 The fcsr defined in RISC-V Unprivileged ISA is fixed-length 32bit CSR. The register structure of vcsr is similar to fcsr.So maybe vcsr should also be defined as a fixed-length 32bit register?

 

Thanks,

 

Tianyi Xia




Re: [RISC-V] [tech-unprivileged] [RISC-V] [tech-vector-ext] FP Trapped exceptions needed for portability

Allen Baum
 

IF the Trap-on-masked-fflags op isn't executed often, then a 4 instruction sequence 
(CSRRD FFLAGS, ANDI, BNE, .+4, ECALL) would do that, so there is a workaround.
IF that has a performance impact, then it argues that you may need actual trapping behavior.


On Fri, Dec 17, 2021 at 5:05 PM ghost <ghost@...> wrote:
> I’d suggest identifying important use cases for this. I’d also be looking at
> software techniques where the compiler inserts checks to provide the
> necessary support for the use cases first.

Along with this, I'd suggest considering an extension that consists of just
one instruction: trap if (FP flags & mask in instruction) is non-zero.  I'm
not a hardware designer, but it seems to me that this would allow
floating-point computation to run at full speed until a point selected by
the programmer or compiler where a precise trap was needed, and the more
instructions the compiler can place between the FP computation and the
conditional trap, the less likely a pipeline stall.

--

L Peter Deutsch <ghost@...> :: Aladdin Enterprises :: Healdsburg, CA

         Was your vote really counted?  http://www.verifiedvoting.org






Re: [RISC-V] [tech-unprivileged] [RISC-V] [tech-vector-ext] FP Trapped exceptions needed for portability

Krste Asanovic
 

I'll note that ARM appears to detect tininess before rounding, while
x86 does so after rounding.

Also, current ARM compilers don't support exception trapping on
AArch64.

https://developer.arm.com/documentation/dui0808/a/floating-point-support/exception-types-recognized-by-the-arm-floating-point-environment

These decisions would not seem to match an intent by ARM to emulate
x86 FP behavior to ease porting.

Krste

On Fri, 17 Dec 2021 16:56:58 -0800, Zalman Stern <zalman@...> said:
| I’d suggest identifying important use cases for this. I’d also be looking at software techniques where the compiler inserts checks to provide the necessary support for the
| use cases first.

| Probably the number one use case is a software emulator for x86 binaries on RISC V. (Because one has to provide the exact x86 behavior regardless of whether it is a strong
| requirement for significant applications.) This alone could have driven things for ARM. The way to investigate would be to look at how Apple’s emulator works.

| Glancing at the large corpus of code one can search at Google, yeah, there's enough stuff claiming a SIGFPE is going to happen in certain circumstances that floating point
| exceptions can't be written off. But most of it looks like stuff that would far better be handled by having the compiler check a hardware provided flag and raise the
| exception rather than having hardware do everything. (It is mostly stuff that is providing some fairly widely used, non-HPC, mathematical functionality and trying to ensure
| a program crashes when numerical invariants are violated.)

| My first thought was to ask why one would want this at all as I've done a fair bit of signal-processing/HPC-ish work in shipping applications and floating-point exceptions
| are only ever used as a debugging tool. Generally most of my interaction with the feature has been fixing performance and correctness issues when floating-point exceptions
| inadvertently get enabled.

| -Z-

| -Z-

| On Fri, Dec 17, 2021 at 3:46 PM Bruce Hoult <bruce@...> wrote:

| On Sat, Dec 18, 2021 at 9:09 AM Earl Killian <earl.killian@...> wrote:

| The question I have is whether having this in scalar only would be sufficient? If porting an application were to need exception traps, it seems plausible to disable
| compiler vectorization.

| The MIPS patent should have expired by now, so it would solve the problem (except for inexact) on a simple in-order core. Does anyone know if x86 code uses inexact
| traps?

| What's the use-case for trapping on inexact (or even caring about it) in FP? Using doubles as 53 bit integers? I did that myself in accounting software back in the 80s
| and 90s, but it's a bit pointless on a 64 bit machine.
|


Re: [RISC-V] [tech-unprivileged] [RISC-V] [tech-vector-ext] FP Trapped exceptions needed for portability

ghost
 

I’d suggest identifying important use cases for this. I’d also be looking at
software techniques where the compiler inserts checks to provide the
necessary support for the use cases first.
Along with this, I'd suggest considering an extension that consists of just
one instruction: trap if (FP flags & mask in instruction) is non-zero. I'm
not a hardware designer, but it seems to me that this would allow
floating-point computation to run at full speed until a point selected by
the programmer or compiler where a precise trap was needed, and the more
instructions the compiler can place between the FP computation and the
conditional trap, the less likely a pipeline stall.

--

L Peter Deutsch <ghost@...> :: Aladdin Enterprises :: Healdsburg, CA

Was your vote really counted? http://www.verifiedvoting.org


Re: [RISC-V] [tech-unprivileged] [RISC-V] [tech-vector-ext] FP Trapped exceptions needed for portability

Zalman Stern
 

I’d suggest identifying important use cases for this. I’d also be looking at software techniques where the compiler inserts checks to provide the necessary support for the use cases first.

Probably the number one use case is a software emulator for x86 binaries on RISC V. (Because one has to provide the exact x86 behavior regardless of whether it is a strong requirement for significant applications.) This alone could have driven things for ARM. The way to investigate would be to look at how Apple’s emulator works.

Glancing at the large corpus of code one can search at Google, yeah, there's enough stuff claiming a SIGFPE is going to happen in certain circumstances that floating point exceptions can't be written off. But most of it looks like stuff that would far better be handled by having the compiler check a hardware provided flag and raise the exception rather than having hardware do everything. (It is mostly stuff that is providing some fairly widely used, non-HPC, mathematical functionality and trying to ensure a program crashes when numerical invariants are violated.)

My first thought was to ask why one would want this at all as I've done a fair bit of signal-processing/HPC-ish work in shipping applications and floating-point exceptions are only ever used as a debugging tool. Generally most of my interaction with the feature has been fixing performance and correctness issues when floating-point exceptions inadvertently get enabled.

-Z-

-Z-

On Fri, Dec 17, 2021 at 3:46 PM Bruce Hoult <bruce@...> wrote:
On Sat, Dec 18, 2021 at 9:09 AM Earl Killian <earl.killian@...> wrote:
The question I have is whether having this in scalar only would be sufficient? If porting an application were to need exception traps, it seems plausible to disable compiler vectorization.

The MIPS patent should have expired by now, so it would solve the problem (except for inexact) on a simple in-order core. Does anyone know if x86 code uses inexact traps?

What's the use-case for trapping on inexact (or even caring about it) in FP? Using doubles as 53 bit integers? I did that myself in accounting software back in the 80s and 90s, but it's a bit pointless on a 64 bit machine.


Re: [RISC-V] [tech-unprivileged] [RISC-V] [tech-vector-ext] FP Trapped exceptions needed for portability

Bruce Hoult
 

On Sat, Dec 18, 2021 at 9:09 AM Earl Killian <earl.killian@...> wrote:
The question I have is whether having this in scalar only would be sufficient? If porting an application were to need exception traps, it seems plausible to disable compiler vectorization.

The MIPS patent should have expired by now, so it would solve the problem (except for inexact) on a simple in-order core. Does anyone know if x86 code uses inexact traps?

What's the use-case for trapping on inexact (or even caring about it) in FP? Using doubles as 53 bit integers? I did that myself in accounting software back in the 80s and 90s, but it's a bit pointless on a 64 bit machine.

81 - 100 of 872