Re: Zvediv extension discussions

Guy Lemieux

Great points Ken & Earl.

One thing I'll point out is that this does not necessarily have much to do with EDIV specifically.

For example, the main goal of EDIV is to support smaller element dot products, particularly for integers. This helps with ML inference, sure. But it leaves a lot of performance on the table, may not help much with other operations than dot products, and probably won't help much with other applications (including ML training).

There are two angles that would supersede this in capability and performance:
-- adding vector shapes
-- adding accelerators to the vector unit
-- in particular, the combination of both shapes + accelerators

In my work (embodied primarily by the VectorBlox MXP architecture) , these were its two primary advantages.

a) allow streaming accelerators to replace the regular ALUs. in particular, you can add a systolic array, where you have N^2 PEs and only require O(N) inputs on each edge and produce O(N) outputs. such accelerators can be standardized, but first the ISA interface needs to be standardized (see below) and possibly the physical interface (eg, AXI-stream).

b) allow vector operands to have shapes, in particular to allow tensor shapes. this affects data readout ordering and writeback ordering when connecting to the accelerator. this would allow, for example, a traditional 1D vector to be interpreted as a 2D tile, or even a 2D subtile. this affects the address generators to the register file, and may require data swizzling to rearrange into the proper shapes and to avoid data bubbles.

In addition to the above, we loosely organized our register file as a multi-banked scratchpad, rather than having fixed-size disjoint registers. This allowed a named vector register to be replaced by a scalar address (a pointer) which fits in the scalar register file. This allowed vectors of arbitrary length, and to start at arbitrary locations, producing much more flexible shapes and subshapes to be read out. This property is probably too much for people to accept right away, but it is needed when you want to have maximum flexibility for both (a) and (b) above.

Note that none of this has anything specifically to do with EDIV. However, it could build upon the vtypes system that Krste has devised. (He previously tried to suggest matrix shapes as possible vtype. In his suggestion, each vector register had its own type descriptor; in the MXP architecture, the type descriptor is global like the vector length register but there are 3 copies of them so src1, src2 and dst can all be different shapes.)


On Fri, Feb 4, 2022 at 10:27 AM Ken Dockser <kad@...> wrote:
Thanks folks, these are all very good points.

Earl: I absolutely agree that these extensions (like all RISC-V extensions) need to be developed based on real-world needs and need to be able to show their value (including utility and usability),  as well as value/cost ratio.

Guy: I agree that we need to look at the other leading architectures as we are well behind them in this area. We then need to come up with our own solutions that address the market needs and fit within the RISC-V approach and philosophy. 

Peter: Yes, we need to work to create extensions that take into account our future needs and intentions. In this case, where we are talking about adding instructions that improve matrix performance in the vector registers, we need to keep in mind how this might fit with future extensions that operate on matrices.

We still need to figure out how we can effectively and efficiently take this next step in RISC-V Vector.  It seems like the best approach would be to leverage the existing Vector TG by producing an updated charter that is focused on completing the Zvediv extension. Is this permitted/possible? Are the current Chair and Vice Chair amenable to this?


On Thu, Feb 3, 2022 at 10:11 PM Guy Lemieux <guy.lemieux@...> wrote:
The AMX extension for AVX512 is an interesting approach.... 

On Thu, Feb 3, 2022 at 8:02 PM Earl Killian <earl.killian@...> wrote:
I hope that these discussions will begin with algorithms and applications that need the additional performance, and proceed to analyze how proposed instructions address the greater need.

On Feb 3, 2022, at 14:45, Peter Lieber <peteralieber@...> wrote:

In the Graphics/ML SIG, we also discussed matrix operations as well.  We talked about a single matrix opcode with various functions like vector-matrix, matrix-matrix, and dot product functions, among others.  Just some initial ideas... 

The Zvediv extension would be a great start to get dimensionality to vectors, and we would want to keep matrices in mind when reviving the effort.

Peter Lieber

On Thu, Feb 3, 2022 at 2:17 PM Ken Dockser <kad@...> wrote:
I am hearing renewed interest in adding a dot-product extension to the RISC-V Vectors. This includes everything from adding a handful of FP and Int dot-product instructions all the way to completing the Zvediv extension.

What is the most efficient way to revive the efforts in these areas? Can we reconvene the Vector TG meetings?


Join { to automatically receive all group messages.