I've not seen very many responses here.  I'll try to describe more
precisely what's concerning me.

In a wide, in-order SIMD core, I can expect two VLEN sized memory
accesses per cycle.  So a spill/fill pair costs a cycle in a memory
limited loop and less in an arithmetically limited loop.

I assume that whole register stores and loads (section 7.9 of the spec)
will be used for spills and fills when register allocation is
oversubscribed.  With this proposal, the spill will know the current
element width from the micro-architectural tag and so will adjust
without extra cost.   But the fill will not know what element width will
be needed.  When the fill element width is (often) not the right one,
there will be several cycles lost to do the (unpipelined) fix-up.

So, a spill-fill pair will increase in cost from a partial cycle to
several cycles.  When the core executes several vector instructions per
cycle, this is a worrisome cost.

I'm wondering whether the compiler can do anything to alleviate this.
Can the compiler know, even most of the time, what EEW was used for a
particular register when it spills and fills the register so that most
spill/fill pairs can avoid this overhead?

Whole register loads in general will cause this problem, but I think
fills after spill are the only one where performance matters.

      Bill

On 6/12/20 4:58 PM, Bill Huffman wrote:
> Hi Krste,
>
> I've been thinking about what happens with this proposal and whole
> register loads and stores - under the assumption of adding uops
> (out-of-order or in-order) when a micro-architectural tag doesn't match
> the intended usage.  I think the stores are probably fine as they know
> the EEW that the register arrangement belongs to and can store correctly
> with the same byte movement as a store of that size would.
>
> But whole register loads don't know where to put the bytes and so will
> have to choose and count on being fixed later, which brings up two points:
>
> First, it makes spills/fills more expensive because after a fill,
> there's an extra small number of cycles to fix the expected EEW.
>
> Second, it means it probably matters to re-write the register when the
> current arrangement and the expected one don't match.  Otherwise after,
> say, returning from an interrupt, a register that's used a large number
> of times will cause a large number of lost cycles.
>
> I think I heard the comment this morning that the register probably
> shouldn't be re-written, just rearranged on the fly for the current use.
>    Did I hear that?  What is the argument for that?
>
>         Bill
>
> On 6/12/20 4:05 AM, Krste Asanovic wrote:
>> EXTERNAL MAIL
>>
>>
>>
>> TL;DR: I'm leaning towards mandating SLEN=VLEN layout, at least for
>> application processor profiles.
>>
>>
>> Regarding register layout, I thought it would be good to lay out the
>> landscape and comparison with other SIMD ISAs before diving into a
>> proposal for RVV.
>>
>>
>> I think it's useful to distinguish "bitsliced" operations from
>> "bitcrossing" operations.
>>
>> It's also useful to define a separate term for physical datapath width
>> "DPW".  In sensible designs, VLEN is an integer power-of-2 multiple of
>> DPW.  If
>>
>> Bitsliced operations on elements of size EEW operate entirely within
>> an EEW region of DPW.
>>
>> Bitcrossing operations traverse more than (source/dest) EEW bits of
>> DPW.
>>
>> In all sane general-purpose SIMD designs, memory operations can move
>> vectors that are naturally aligned to element boundaries, not only to
>> VLEN boundaries, so all memory operations are bitcrossing operations
>> assuming DPW > smallest EEW and require at least a memory rotate if
>> not a full crossbar between memory ports and register file ports.
>> (Some specialized SIMD designs might retain a VLEN-alignment
>> constraint, but they're not of interest here).
>>
>> There are specialized register permute instructions that are
>> bitcrossing instructions, such as our slide, vrgather, and compress
>> instructions (reductions also).  All SIMD ISAs add some variants of
>> these.
>>
>> Many simple vector arithmetic operations are bitsliced.
>>
>> The interesting cases are mixed-width operations, which are prevalent
>> in low-precision multiply-accumulate kernels that dominate many
>> existing and emerging compute areas, but there are plenty of other
>> kernels that operate on mixed-width data items.  Classic SIMD ISAs
>> handle mixed-width operations in one of five ways (would be glad to
>> add other known options to this list):
>>
>> 0) Single-width elements.  All operations operate on same width, with
>>      variety of load/stores widening/narrowing into/out of this element
>>      size.  For lower precision arithmetic, this wastes a lot of
>>      register file capacity, regfile port bandwidth, and ALU throughput.
>>
>> 1) Specialized registers. Some SIMD machines have dedicated wider
>>     accumulator registers, so datapath remains effectively bitsliced.
>>     Many machines have dedicated predicate registers (treating
>>     predicates as example of mixed-width operation).
>>
>> 2) Pack/unpack.  Arithmetic instructions are all bitsliced, but with
>>     separate bitcrossing data movement operations to pack/unpack
>>     elements, e.g., register-register unpack will sign/zero-extend
>>     top/bottom of a vector to yield destination vector of 2*source-EEW
>>     elements.  pack will do reverse from parts of two vectors.  Unpacked
>>     memory loads/stores similarly sign/zero-extend or truncate on way
>>     in/out of memory.  The pack/unpack instructions tend to imply
>>     crossing the entire DPW, and also complicates software which has to
>>     unroll loop into hi/lo portions and issue separate intsructions for
>>     each half.
>>
>> 3) Register pairs, where wider operand is created by pairing two
>>     existing registers within EEW so avoiding bitcrossing.  But this
>>     splits a single element's storage across two architectural
>>     registers, which doesn't support load/store to in-memory application
>>     formats without pack/unpack bitcrossing operations or additional
>>     load/store instructions (also effectively pack/unpack instructions).
>>
>> 4) EDIV-style, where mixed operations are handled by dividing an
>>     element width into subelements and accumulating multiple subelements
>>     into parent element size (e.g., 4 8b*8b multiplies accumulated into
>>     32b accumulator).  There provide mixed-width operations while
>>     avoiding bitcrossing.  However, they impose restrictions on
>>     application input and/or output data layouts to achieve high
>>     efficiency.
>>
>> With RVV we are trying to support mixed-width operations without
>> adding specialized registers, or splitting an element across
>> architectural registers, or requiring implicit or explicit bitcrossing
>> beyond min(DPW,SLEN) on ALU operands (bitcrossing for memory
>> load/stores cannot be avoided).  We're also trying to support vector
>> units with current implementation targets ranging from VLEN=32 to
>> VLEN=16384.
>>
>> ----------------------------------------------------------------------
>>
>> The SLEN parameter allows implementations to optimize their wire
>> length.  For narrower DPW (<=128b) regardless of VLEN, the SLEN=VLEN
>> layout where in-register format matches in-memory works OK as
>> bitcrossing cannot be outside DPW.  For wider DPW (>128b), SLEN<VLEN
>> layouts minimize bitcrossing.  When codes want to pun bytes between
>> different element widths, the SLEN<ELEN requires cast operations that
>> will shuffle bytes around (become simple register moves on SLEN=VLEN
>> machines).
>>
>> I see two separate problems with the SLEN parameter.
>>
>> 1) SLEN=VLEN layout can be profitably exploited in some code,
>> encouraging programmers to ignore compatibility and drop cast
>> instructions.
>>
>> 2) Correct code varying in SLEN cannot be migrated between machines
>> with same VLEN.  This I view as a quite serious issue, not just for
>> migration but also verification and any case where we're working
>> across two implementations.
>>
>> We've worked through many alternatives, but at this point, I'm back to
>> proposing that SLEN=VLEN is an extension, and that this extension is
>> required for application processors (i.e., this is in "V").  The mode
>> bit idea (software indicates if SLEN=VLEN layout is needed) doesn't
>> solve thread migration, but we could add that to SLEN<VLEN profile
>> somehow instead of casting, in a way such that SLEN=VLEN could ignore
>> it and didn't have to implement the null cast instructions.
>>
>> For systems with DPW <=128b, this is simple to implement in all kinds
>> of system.
>>
>> For wider datapaths DPW>=256b, the SLEN<VLEN layout would be
>> preferable.
>>
>> For more complex microarchitectures that want wider DPW, SLEN=VLEN can
>> be software view, with internal layout hidden by the
>> microarchitectural tricks we previously discussed.  Any access using
>> the wrong EEW can shuffle the bytes microarchitecturally.  This does
>> add complexity, but assuming that casting is relatively rare, these
>> machines would benefit from fact the ISA not does not require bit
>> crossing for general mixed-width code (not just for EDIV).  There is
>> clearly some complexity here, but I think the shuffling is thankfully
>> contained with a DPW-bit element group.
>>
>> ----------------------------------------------------------------------
>>
>> A different direction would be to say that SLEN=VLEN layout is
>> mandatory but bitcrossing instructions are an extension.  But we'd
>> still need to define something for mixed-width arithmetic (EDIV is
>> probably least objectionable choice out of above list of 0-4 options).
>>
>> ----------------------------------------------------------------------
>>
>> Finally, I think for the crypto extensions there is actually no need
>> to limit ELEN.  We can instead just limit bitcrossing arithmetic
>> instructions to ELEN<=128.  ELEN >128 need only be supported by a few
>> operations such as crypto.  We can use wider ELEN with EDIV, where
>> EDIV cuts size of sub-element to supported arithmetic EEW (e.g.,
>> ELEN=256 with EDIV=8 for 8*32b floats).
>>
>>
>> Krste
>>
>>