Hi all,

Recently the TSC established a lightweight "fast track" architecture extension process that small, straightforward, relatively uncontentious arch extension proposals can utilize.  Andrew and I discussed trying to help move a couple of Privileged architecture related small extensions - that a number of people/companies have expressed desire for over the past year - through this process sooner than later (especially since this entails much more than simply proposing and reviewing a spec).  For this one Siqi and I are co-sponsors.  The following starts with an intro for context, and then provides the draft spec.

=================================================================

The current Privileged specification only defines a hardware mechanism for generating machine-mode timer interrupts (based on the mtime and mtimecmp registers).  With the resultant requirement that timer services for S-mode/HS-mode (and for VS-mode) have to all be provided by M-mode - via SBI calls from S/HS-mode up to M-mode (or VS-mode calls to HS-mode and then to M-mode).  M-mode software then multiplexes these multiple logical timers onto its one physical M-mode timer facility, and the M-mode timer interrupt handler passes timer interrupts back down to the appropriate lower privilege mode.

Over the past year or more, there have been a number of requests to provide supervisor mode with its own CSR-based timer interrupt facility that it can directly manage to provide its own timer service (in the form of having its own stimecmp register) - thus eliminating the large overheads for emulating S/HS-mode timers and timer interrupt generation up in M-mode.  This proposal, using the new fast-track architecture extension process, serves to accomplish exactly this.  Further, this proposal lays the groundwork for adding a similar facility to the Hypervisor extension for VS-mode.   (ARMv8, for example, has separate timer registers for its equivalent of M, S/HS, and VS privilege levels.)

Below is a one-page draft spec of the proposal - which sticks to addressing the basic well-understood need that has been requested.  The proposed extension name is "Zstc" ('Zs' for privileged Supervisor arch extensions, and 'tc' for timecmp).

Various further pieces of related architectural functionality have previously been suggested, debated, and questioned - without clear conclusion and agreement as to whether these are truly justified.  These more contentious enhancements are left to be sorted out and pursued by a follow-on effort if desired and justified by sufficient people.

The suggestion to also introduce an mtimecmp CSR has been considered, but ruled out for now as not sufficiently justified over the current definition of mtimecmp as a memory-mapped register (which would have to remain for backward compatibility in any case - resulting in two aliases for this register).  But if there is sufficient interest, a separate email thread can be started to discuss the possible addition of an mtimecmp CSR (as part of this or a separate fast-track extension).  For now I'll leave out details about how the mtimecmp memory-mapped register and mtimecmp CSR could inter-relate (it doesn't have to be complicated in practice).

This extension adds a single CSR - the stimecmp CSR.  This extension also describes what the corresponding addition to the Hypervisor extension would be - the vstimecmp CSR.  The former is the official part of this fast-track arch extension.  The latter is solely informative and reflective of what would then likely be added to the current H-extension draft spec as part of its virtualization of S-mode.

The draft spec below provides all the details.  Note that this extension very specifically strives to maintain maximal consistency with many little details in the existing Privileged architecture and Hypervisor extension (and has already been reviewed by the two lead Privileged and Hypervisor architects to ensure this).

This posting to this email list starts an initial review period (over the next few weeks) for people to provide feedback, questions, comments, etc.

Thanks,

Siqi and Greg

=========================================================================

=======================  Supervisor Extension Additions  ========================

Supervisor Timer Register (stimecmp)

The stimecmp CSR is a 64-bit register and has 64-bit precision on all RV32 and RV64 systems.  In RV32 only, reads of the stimecmp CSR return the low 32 bits, while reads of the stimecmph CSR return the high 32 bits of stimecmp.

The proposed CSR numbers for stimecmp / stimecmph are 0x108 / 0x109 (within the Supervisor Trap Setup block of CSRs).

A supervisor timer interrupt becomes pending - as reflected in the STIP bit in the mip and sip registers - whenever mtime contains a value greater than or equal to stimecmp, treating the values as unsigned integers.  Writes to stimecmp are guaranteed to be reflected in STIP eventually, but not necessarily immediately.  The interrupt remains posted until stimecmp becomes greater than mtime - typically as a result of writing stimecmp. The interrupt will be taken based on the standard interrupt enable and delegation rules.

----------------------------  Non-Normative Text    ----------------------------

A spurious timer interrupt might occur if an interrupt handler advances stimecmp then immediately returns, because STIP might not yet have fallen in the interim. All software should be written to assume this event is possible, but most software should assume this event is extremely unlikely. It is almost always more performant to incur an occasional spurious timer interrupt than to poll STIP until it falls.  

----------------------------------------------------------------------------------------

----------------------------  Non-Normative Text    ----------------------------

In systems in which a supervisor execution environment (SEE) provides timer facilities via an SBI function call, this SBI call will continue to support requests to schedule a timer interrupt.  The SEE will simply make use of stimecmp, changing its value as appropriate.  This ensures compatibility with existing S-mode software that uses this SEE facility, while new S-mode software takes advantage of stimecmp directly.)

----------------------------------------------------------------------------------------

Machine Interrupt Registers (mip and mie)

This extension modifies the description of the STIP/STIE bits in these registers as follows:

If supervisor mode is implemented, its mip.STIP and mie.STIE are the interrupt-pending and interrupt-enable bits for supervisor-level timer interrupts.  If the stimecmp register is not implemented, STIP is writable in mip, and may be written by M-mode software to deliver timer interrupts to S-mode.  If the stimecmp (supervisor-mode timer compare) register is implemented, STIP is read-only in mip and reflects the supervisor-level timer interrupt signal resulting from stimecmp.  This timer interrupt signal is cleared by writing stimecmp with a value greater than the current mtime value.

Supervisor Interrupt Registers (sip and sie)

This extension modifies the description of the STIP/STIE bits in these registers as follows:

 

Bits sip.STIP and sie.STIE are the interrupt-pending and interrupt-enable bits for supervisor level timer interrupts.  If implemented, STIP is read-only in sip, and is either set and cleared by the execution environment (if stimecmp is not implemented), or reflects the timer interrupt signal resulting from stimecmp (if stimecmp is implemented).  The sip.STIP bit, in response to timer interrupts generated by stimecmp, is set and cleared by writing stimecmp with a value that respectively is less than or equal to, or greater than, the current mtime value.

Machine Counter-Enable Register (mcounteren)

This extension adds to the description of the TM bit in this register as follows:

In addition, when the TM bit in the mcounteren register is clear, attempts to read the stimecmp register while executing in S-mode will cause an illegal instruction exception. When this bit is set, access to the stimecmp register (if implemented) is permitted in S-mode.

=========================================================================

=======================  Hypervisor Extension Additions  ========================

Virtual Supervisor Timer Register (vstimecmp)

The vstimecmp CSR is a 64-bit register and has 64-bit precision on all RV32 and RV64 systems.  In RV32 only, reads of the vstimecmp CSR return the low 32 bits, while reads of the vstimecmph CSR return the high 32 bits of vstimecmp.

The proposed CSR numbers for vstimecmp / vstimecmph are 0x208 / 0x209 (within the Virtual Supervisor Registers block of CSRs, and mirroring the CSR numbers for stimecmp/stimecmph).

A virtual supervisor timer interrupt becomes pending - as reflected in the VSTIP bit in the hip register - whenever (mtime + htimedelta) contains a value greater than or equal to vstimecmp, treating the values as unsigned integers.  Writes to vstimecmp and htimedelta are guaranteed to be reflected in VSTIP eventually, but not necessarily immediately.  The interrupt remains posted until vstimecmp becomes greater than (mtime + htimedelta) - typically as a result of writing vstimecmp. The interrupt will be taken based on the standard interrupt enable and delegation rules while V=1.

----------------------------  Non-Normative Text    ----------------------------

In systems in which a supervisor execution environment (SEE) implemented by an HS-mode hypervisor provides timer facilities via an SBI function call, this SBI call will continue to support requests to schedule a timer interrupt.  The SEE will simply make use of vstimecmp, changing its value as appropriate.  This ensures compatibility with existing guest VS-mode software that uses this SEE facility, while new VS-mode software takes advantage of vstimecmp directly.)

----------------------------------------------------------------------------------------

Hypervisor Interrupt Registers (hvip, hip, and hie)h

This extension modifies the description of the VSTIP/VSTIE bits in the hip/hie registers as follows:

Bits hip.VSTIP and hie.VSTIE are the interrupt-pending and interrupt-enable bits for VS-level timer interrupts. VSTIP is read-only in hip, and is the logical-OR of hvip.VSTIP and the timer interrupt signal resulting from vstimecmp (if vstimecmp is implemented).  The hip.VSTIP bit, in response to timer interrupts generated by vstimecmp, is set and cleared by writing vstimecmp with a value that respectively is less than or equal to, or greater than, the current (mtime + htimedelta) value.  The hip.VSTIP bit remains defined while V=0 as well as V=1.

Hypervisor Counter-Enable Register (hcounteren)

This extension adds to the description of the TM bit in this register as follows:

In addition, when the TM bit in the hcounteren register is clear, attempts to read the vstimecmp register while executing in VS-mode will cause a virtual illegal instruction exception if the same bit in mcounteren is 1. When this bit is set, access to the vstimecmp register (if implemented) is permitted in VS-mode.

On Tue, Dec 1, 2020 at 5:21 PM Allen Baum <allen.baum@...> wrote:

So to know if M-mode can fake an Smode counter interrupt or not,(which it can currently do), 

Mmode SW needs to attempt to write STMECMP and see if it traps? OR attempt to set the STIP bit and see if it sticks? 

You might want some non-normative text about discovery.

This text takes the same approach as much of the rest of the Supervisor chapter in the Priv spec, i.e. the existing and coming new discovery methods apply equally to these new CSRs as to all existing and new s*/h*/vs* CSRs that may be accessible in S/HS modes.  This extension doesn't try to establish anything different.  Or, put differently, the discovery issue is a general issue (for which this extension does not represent any form of special case).

Whether for this Zstc extension or the coming (for example) Zsn and Zsa extensions, there is trapping behavior as one (ugly) form of discovery method.  And then there will be the new discovery method being developed by tech-config which will be the more reasonable way to discover many things about optional extensions, and optional or WARL features within extensions.

So any new non-normative text about CSR discovery really should be added to the Priv/Unpriv specs as a general matter (separate from this and other in-progress Supervisor extensions).  (And each extension innovating its own "better" discovery method would be an undesirable way to go.)

 

IS there some reason for removing the existing behavior of allowing STIP to be writable in MIP?

Yes.  This was first pointed out by John, and Andrew and I agreed.  The gist is that when a hart implements stimecmp, for consistency we should require STIP to be read-only - just like MTIP.  (Especially since there is no longer a need for a software-writable bit behind STIP when stimecmp exists.)

Greg

Hi,

Was any consideration given to the possibility of defining stimecmp as a memory-mapped register (like mtimecmp) rather than a CSR?
Assuming that the timer will actually be implemented in a block outside of the CPU (e.g. the ACLINT), a memory-mapped register would be preferable for interfacing the CPU to the timer block.  Keep in mind that in most high-end (especially multi-processor) systems will have the timer in a separate clock domain from the CPU and CSRs (which seems to have been the key motivation for defining mtime/mtimecmp as memory-mapped registers rather than CSRs).

Also, just curious about the overall status of this extension.  It is mentioned in the RVA22 profile, which would suggest that the Sstc definition should be finalized soon if it isn't already.

Thanks,
Phil McCoy

On Mon, Sep 13, 2021 at 12:49 PM Phil McCoy <pnm@...> wrote:

Was any consideration given to the possibility of defining stimecmp as a memory-mapped register (like mtimecmp) rather than a CSR?
Assuming that the timer will actually be implemented in a block outside of the CPU (e.g. the ACLINT), a memory-mapped register would be preferable for interfacing the CPU to the timer block.  Keep in mind that in most high-end (especially multi-processor) systems will have the timer in a separate clock domain from the CPU and CSRs (which seems to have been the key motivation for defining mtime/mtimecmp as memory-mapped registers rather than CSRs).

Paul's comment is correct.  The time, stimecmp, and vstimecmp registers are all CSRs within a hart; separate from the mtime register located typically somewhere that may not be nearby any or every hart.  Also keep in mind that the vstimecmp CSR will be context switched by a hypervisor.  And in some applications even the stimecmp CSR may be context switched some.  Whereas the memory-mapped mtimecmp would generally never be context switched.

If anything, during discussions about this extension, the question was raised in the other direction about also adding an mtimecmp CSR for consistency.  But that was a whole other discussion to be had - given that there is existing mtimecmp functionality and not a strong argument for changing that.

 

Also, just curious about the overall status of this extension.  It is mentioned in the RVA22 profile, which would suggest that the Sstc definition should be finalized soon if it isn't already.

Keep in mind that the Profile specs are very much still under development and won't be frozen til November.  While the Sstc extension is driving to get ratified by mid-November.

Greg