Date   

Re: Proposal: Delegating Exceptions from VS-mode or VU-mode to U-mode

Andrew Waterman
 

The N extension is effectively deprecated. We don’t see sufficient demand for user-level interrupts in managed/Unix-like environments to pursue that approach at this time.

We do see demand for this feature in embedded systems with two levels of privilege. In this case, the proposal is to run the trusted piece in M-mode and the less-trusted piece in “bare S-mode”: i.e., S-mode without virtual memory.

This approach covers the interesting use cases without additional architectural complication, and in particular, the hypervisor architecture need not care.

On Fri, Sep 18, 2020 at 1:03 AM Yifei Jiang via lists.riscv.org <jiangyifei=huawei.com@...> wrote:
















Hi all,



 



In this proposal, we extended N extension and applied it to H extension for improving the performance of virtual I/O devices in the virtualization scenario. We proposed a new mechanism to delegate exceptions from VS-mode/VU-mode

to U-mode. A solution is also implemented based on QEMU simulator and KVM virtualization architecture. Evaluation results show that our proposal achieves nearly 2x faster synchronous I/O processing speed than the orginal system.



 



The attachment is the detailed proposal. Any comments are welcome.



 



Regards,



Yifei


















Re: Proposal: Delegating Exceptions from VS-mode or VU-mode to U-mode

Anup Patel
 

Hi,

 

The context switching overhead for VirtIO interrupts that this proposal is trying to solve is already solved across architectures by:

  1. KVM in-kernel VirtIO backends emulation (i.e. VHost) (Already available)
  2. KVM in-kernel interrupt-controller emulation and KVM irqfd (Available on other architectures but not currently available for KVM RISC-V)

 

The point2 is intentionally not done for KVM RISC-V because we don’t want in-kernel PLIC emulation due to lack of MSI support and Virtualization support in PLIC. Also, the in-kernel PLIC emulation will become redundant once new interrupt controller spec is available. We will go for in-kernel interrupt controller emulation once new interrupt controller spec is available.

 

Is there any other advantage of this proposal over VHost + In-kernel interrupt-controller ??

 

Regards,

Anup

 

From: tech-privileged@... <tech-privileged@...> On Behalf Of Yifei Jiang via lists.riscv.org
Sent: 18 September 2020 13:33
To: tech-privileged@...
Subject: [RISC-V] [tech-privileged] Proposal: Delegating Exceptions from VS-mode or VU-mode to U-mode

 

Hi all,

 

In this proposal, we extended N extension and applied it to H extension for improving the performance of virtual I/O devices in the virtualization scenario. We proposed a new mechanism to delegate exceptions from VS-mode/VU-mode to U-mode. A solution is also implemented based on QEMU simulator and KVM virtualization architecture. Evaluation results show that our proposal achieves nearly 2x faster synchronous I/O processing speed than the orginal system.

 

The attachment is the detailed proposal. Any comments are welcome.

 

Regards,

Yifei


Re: Proposal: Delegating Exceptions from VS-mode or VU-mode to U-mode

Phil McCoy
 

If I understand correctly, there is a security issue around the URET instruction.  All the controls for URET are accessible from User mode.  This is OK in the scenario you describe where U-Mode is trusted (the U-Mode code is the UART emulation, which is effectively just a privilege-reduced portion of the hypervisor).

There does not appear to be any way for the hypervisor to prevent untrusted user-mode code from URET'ing into any arbitrary location in the virtualized guest (VS/VU) software by programming uepc and ustatus.


Proposal: Accelerating Handling User-level Interrupts

Yifei Jiang
 

Hi all,

 

When we applied user-level interrupts in N extension to the Unix-like OS, there is no way to directly handle user-level interrupts in userspace bypassing the kernel. This makes little use of the user-level interrupt mechanism. In this proposal, we provide a hardware-assisted context switch mechanism to accelerate handling interrupts in the U-mode of Unix-like OS. Our proposal is implemented on the QEMU simulator and KVM virtualization architecture. Evaluation results show that our proposal improves the block performance by 2.63% to 7.56%. Also, more benefits can be gained by combining this proposal with some other optimizations.

 

The attachment is the detailed proposal. Any comments are welcome.

 

Regards,

Yifei


Proposal: Delegating Exceptions from VS-mode or VU-mode to U-mode

Yifei Jiang
 

Hi all,

 

In this proposal, we extended N extension and applied it to H extension for improving the performance of virtual I/O devices in the virtualization scenario. We proposed a new mechanism to delegate exceptions from VS-mode/VU-mode to U-mode. A solution is also implemented based on QEMU simulator and KVM virtualization architecture. Evaluation results show that our proposal achieves nearly 2x faster synchronous I/O processing speed than the orginal system.

 

The attachment is the detailed proposal. Any comments are welcome.

 

Regards,

Yifei


Re: platform specific interrupt control

Andrew Waterman
 

Yes, the behavior you'd like follows implicitly from the definition of mideleg.


On Tue, Sep 15, 2020 at 9:23 PM Sanjay Patel <spatel@...> wrote:
Hi,

The definition of the interrupt pending and enable registers, example mip and mie, define bits 16 and above as available for platform or custom use.

Section 3.1.9 Machine Interrupt Registers (mip and mie

Bits 15:0 are allocated to standard interrupt causes only, while bits 16 and above are available for platform or custom use.

The delegation registers have no similar explicit definition unless I missed it. I will assume that bits 16 and above of mideleg for example are similarly available for platform or custom use.
To be otherwise wouldn't make sense because there is a bit to bit corresponding between all 3 registers. Can these bits be safely implemented in the manner described?

Sanjay




platform specific interrupt control

Sanjay Patel <spatel@...>
 

Hi,

The definition of the interrupt pending and enable registers, example mip and mie, define bits 16 and above as available for platform or custom use.

Section 3.1.9 Machine Interrupt Registers (mip and mie

Bits 15:0 are allocated to standard interrupt causes only, while bits 16 and above are available for platform or custom use.

The delegation registers have no similar explicit definition unless I missed it. I will assume that bits 16 and above of mideleg for example are similarly available for platform or custom use.
To be otherwise wouldn't make sense because there is a bit to bit corresponding between all 3 registers. Can these bits be safely implemented in the manner described?

Sanjay




Re: Disabling and re-enabling extensions

Allen Baum
 

Emulating systems that don’t support some extension either has SW that doesn’t use the extension, in which case it does t matter if the extension is enabled or disable, or does use it, in which case the more capable system will execute it natively. 

If the less capable system is non-conforming - using those opcodes for something else, then you have to 
 -guarantee that disabled extensions trap
 - guarantee that all the custom ops used by the less-capable implementation are a subset of the disabled extension.

This getting pretty far out into left field, IMHO.


-Allen

On Sep 11, 2020, at 4:17 PM, Andrew Waterman <andrew@...> wrote:



On Thu, Sep 10, 2020 at 7:14 PM Allen Baum <allen.baum@...> wrote:
OK - taking a step back, what about requiring: any extension spec that doesn't specifically discuss enabling and disabling should be prohibited from enabling or disabling..
I can't see any good use cases - even for power, given the F/D method of dealing with that via the XS bits.

Emulating less-capable systems.
 
That is a very explicit mechanism, specifically architected for enabling and disabling, with all the side effects and corner cases defined (one hopes).
 

On Thu, Sep 10, 2020 at 4:13 PM Paul Donahue <pdonahue@...> wrote:
The hypervisor spec specifically says that implementations should have writable misa so this isn't theoretical:
"The hypervisor extension is enabled by setting bit 7 in the misa CSR, which corresponds to the letter H. When misa[7] is clear, the hart behaves as though this extension were not implemented, and attempts to use hypervisor CSRs or instructions raise an illegal instruction exception. Implementations that include the hypervisor extension are encouraged not to hardwire misa[7], so that the extension may be disabled. "
And, of course, the misa register says that extensions may be disabled, that both E and I may be implemented and that misa is the way to select between them, etc.  I don't know if anybody has actually done those things, though.


Thanks,

-Paul


On Thu, Sep 10, 2020 at 4:05 PM Allen Baum <allen.baum@...> wrote:
I thought there was already a speced requirement that, at reset, all extensions are enabled.:
    "At reset, the Extensions field shall contain the maximal set of supported extensions, and I shall be selected over E if both are available."

And then there are the corner cases: if there are no misa bits for subsets (e.g Zbb, Zv<whatever>), then they can't be enabled or disabled.
So, why should misa. bits for the full extensions allow enabling/disabling?
What does it mean for misa.G to be enabled or disabled? misa.I ?(if misa.E isn't also present)?

Has anyone identified use cases for turning extensions on and off? (as opposed to FP-like disabled state to save power).
Has anyone ever implemented RW misa bits?
IF not: why not just require those MISA bits to be RdOnly?

On Thu, Sep 10, 2020 at 10:44 AM Greg Favor <gfavor@...> wrote:
A potential second general requirement for all Priv extensions:

- The hart reset state must be fully specified and should generally be UNSPECIFIED except for any key control/status bits that need to be in a specific state.  (This minimizes hardware cost and leaves probably the large majority of state to be initialized by software.)

- The reset state should also specify one of:
    a) The extension is "disabled" (and most, if not all, other extension state is don't care).

    b) The extension is "enabled" and any key control/status bits are initialized to specific values as necessary (with all the rest UNSPECIFIED and left to be initialized by software).

Greg


Re: Disabling and re-enabling extensions

Andrew Waterman
 



On Thu, Sep 10, 2020 at 7:14 PM Allen Baum <allen.baum@...> wrote:
OK - taking a step back, what about requiring: any extension spec that doesn't specifically discuss enabling and disabling should be prohibited from enabling or disabling..
I can't see any good use cases - even for power, given the F/D method of dealing with that via the XS bits.

Emulating less-capable systems.
 
That is a very explicit mechanism, specifically architected for enabling and disabling, with all the side effects and corner cases defined (one hopes).
 

On Thu, Sep 10, 2020 at 4:13 PM Paul Donahue <pdonahue@...> wrote:
The hypervisor spec specifically says that implementations should have writable misa so this isn't theoretical:
"The hypervisor extension is enabled by setting bit 7 in the misa CSR, which corresponds to the letter H. When misa[7] is clear, the hart behaves as though this extension were not implemented, and attempts to use hypervisor CSRs or instructions raise an illegal instruction exception. Implementations that include the hypervisor extension are encouraged not to hardwire misa[7], so that the extension may be disabled. "
And, of course, the misa register says that extensions may be disabled, that both E and I may be implemented and that misa is the way to select between them, etc.  I don't know if anybody has actually done those things, though.


Thanks,

-Paul


On Thu, Sep 10, 2020 at 4:05 PM Allen Baum <allen.baum@...> wrote:
I thought there was already a speced requirement that, at reset, all extensions are enabled.:
    "At reset, the Extensions field shall contain the maximal set of supported extensions, and I shall be selected over E if both are available."

And then there are the corner cases: if there are no misa bits for subsets (e.g Zbb, Zv<whatever>), then they can't be enabled or disabled.
So, why should misa. bits for the full extensions allow enabling/disabling?
What does it mean for misa.G to be enabled or disabled? misa.I ?(if misa.E isn't also present)?

Has anyone identified use cases for turning extensions on and off? (as opposed to FP-like disabled state to save power).
Has anyone ever implemented RW misa bits?
IF not: why not just require those MISA bits to be RdOnly?

On Thu, Sep 10, 2020 at 10:44 AM Greg Favor <gfavor@...> wrote:
A potential second general requirement for all Priv extensions:

- The hart reset state must be fully specified and should generally be UNSPECIFIED except for any key control/status bits that need to be in a specific state.  (This minimizes hardware cost and leaves probably the large majority of state to be initialized by software.)

- The reset state should also specify one of:
    a) The extension is "disabled" (and most, if not all, other extension state is don't care).

    b) The extension is "enabled" and any key control/status bits are initialized to specific values as necessary (with all the rest UNSPECIFIED and left to be initialized by software).

Greg


Re: Proposal: Supervisor Timer CSR and Virtual Supervisor Timer CSR

John Hauser
 

zhaosiqi (Siqi) wrote:
So VGTI is purely for the prompt handling of the vstimecmp interrupt
when the vCPU is still bound to a hart but not executing.
I believe a hypervisor can get the same effect by saving and clearing
bit 6 of hideleg on entry to a trap handler in HS mode. On trap exit,
restore the saved value of bit 6 of hideleg. (Usually, it should be
possible just to save and restore all of hideleg.) While bit 6 of
hideleg is zero, a VS-level timer interrupt will trap to HS mode,
assuming bit 6 (VSTIE) of hie is also set to enable the trap.

- John Hauser


Re: Proposal: Supervisor Timer CSR and Virtual Supervisor Timer CSR

Greg Favor
 

Siqi,

Thanks.  That narrows down where we are differing.  

If the hypervisor has set hideleg bit 6 (so that VS-level timer interrupts can be received by VS-mode while V=1), then any new pending VS-level timer interrupt won't be taken until the hypervisor returns back into the VM.  But at that moment the hypervisor is busy servicing whatever caused the exit from the VM and into the hypervisor.  Even if the hypervisor got interrupted by the pending VS-level interrupt, it would defer doing anything about it until it reached a point where it could consider returning to this VM.

Then, at junctures like that, the hypervisor may decide to context switch to another VM instead of returning into this one.  But the hypervisor code that considers whether to switch away to another VM or not would be the natural place for it to check (via hip.VSTIP) whether the current VM has a pending VS-level timer interrupt and decide whether it should not switch to another VM.  This check should not be appearing in many places in hypervisor code.  I believe it would be in the one or few pieces of code that handle deciding whether to return to the current VM or to context switch to a different VM (i.e this decision-making process should not be spread across many places in the code).

Conversely, if you had some form of VGTI, then when the hypervisor receives the VS-level timer interrupt, it would generally have to defer doing something about it until it reaches a suitable juncture where it can consider whether to return to this (or another) VM.  At which point the code at that juncture could simply poll hip.VSTIP instead of having to remember that a VGTI was received earlier.

I'm guessing this may not get us to a point of agreement yet :), but hopefully we're narrowing in.

Greg

P.S. If necessary, we can put this to the people doing two of the RISC-V hypervisor ports.


On Thu, Sep 10, 2020 at 8:04 PM zhaosiqi (A) via lists.riscv.org <zhaosiqi3=huawei.com@...> wrote:
Hi Greg,

I understand your concern. In fact, the goal is not to provide more than one vstimecmp CSR.

You are right that when a VM is context switched out, it's vstimecmp CSR is also switched out. However, there exists time when the VM, or a vCPU, is not executing, however, its CSR context is still 'bound' to a hart. In other words, the vs- CSRs still contain values for that vCPU (including vstimecmp), however, the hart is executing hypervisor code. For example, when the hypervisor is handling certain exception caused by the VM such as a guest page fault, there is no need to switch the vCPU context out.

During this time, since vstimecmp still contains the value for that vCPU, it can fire interrupts. When this interrupt is received by the hart, then obviously it should be the hypervisor that handles it, thus the VGTI. Without VGTI, this interrupt will be delayed until the hypervisor returns to the vCPU. This delay can be long because the hypervisor may not immediately return to the vCPU, i.e. it may well decide that the vCPU needs to yield and schedule something else. Of course, the hypervisor can check at strategic points the value in vstimecmp and make proper decision, but that makes software more complex because those checks might be tricky to be inserted.

So VGTI is purely for the prompt handling of the vstimecmp interrupt when the vCPU is still bound to a hart but not executing.

Lastly, when the hypervisor finally decides to switch a vCPU out, then vstimecmp gets saved and hypervisor uses its own timer to track the timer set by that vCPU, which goes to stimecmp. In this way the only vstimecmp is multiplexed among vCPUs, there's no need for more than one vstimecmp.

Regards,
Siqi


Re: Proposal: Supervisor Timer CSR and Virtual Supervisor Timer CSR

Siqi Zhao
 

Hi Greg,

I understand your concern. In fact, the goal is not to provide more than one vstimecmp CSR.

You are right that when a VM is context switched out, it's vstimecmp CSR is also switched out. However, there exists time when the VM, or a vCPU, is not executing, however, its CSR context is still 'bound' to a hart. In other words, the vs- CSRs still contain values for that vCPU (including vstimecmp), however, the hart is executing hypervisor code. For example, when the hypervisor is handling certain exception caused by the VM such as a guest page fault, there is no need to switch the vCPU context out.

During this time, since vstimecmp still contains the value for that vCPU, it can fire interrupts. When this interrupt is received by the hart, then obviously it should be the hypervisor that handles it, thus the VGTI. Without VGTI, this interrupt will be delayed until the hypervisor returns to the vCPU. This delay can be long because the hypervisor may not immediately return to the vCPU, i.e. it may well decide that the vCPU needs to yield and schedule something else. Of course, the hypervisor can check at strategic points the value in vstimecmp and make proper decision, but that makes software more complex because those checks might be tricky to be inserted.

So VGTI is purely for the prompt handling of the vstimecmp interrupt when the vCPU is still bound to a hart but not executing.

Lastly, when the hypervisor finally decides to switch a vCPU out, then vstimecmp gets saved and hypervisor uses its own timer to track the timer set by that vCPU, which goes to stimecmp. In this way the only vstimecmp is multiplexed among vCPUs, there's no need for more than one vstimecmp.

Regards,
Siqi


Re: Proposal: Supervisor Timer CSR and Virtual Supervisor Timer CSR

Greg Favor
 

Siqi,

Thanks for your clarification.  If I understand correctly, your goal (analogous to SGEI as you mention) is to enable a pending virtual timer interrupt, even for a VM not currently context switched in, to cause an interrupt to the hypervisor so that it can context switch in that VM.  (As you recognize, for virtual external interrupts this is based on the hgeip / hgeie registers and hip.SGEIP / hie.SGEIE register bits.)

In contrast, for the VM currently switched in, a virtual timer interrupt can be directly reflected - based on hideleg - in either hip.VSTIP or vsip.STIP, i.e. the hypervisor can choose whether the pending interrupt goes to itself or directly to the guest.  (In the latter case the hypervisor can also poll hip.VSTIP.)  There's no need for an SGEI-like mechanism here.

The problem is that while a VM is context switched out, its vstimecmp is also switched out.  Or, put differently, if there are N VM's currently assigned to a hart and one wants hardware to inform the hypervisor when a VM's virtual timer "fires", then there would need to be N vstimecmp registers, N associated htimedelta registers, N time comparators, and one would need the equivalent of the hgeip/hgeie registers (i.e. hgtip/hgtie), and hip.SGTIP and hie.SGTIE bits.  In essence, the full analogue of what the H-extension provides for virtual external interrupts.

This obviously is a lot more expensive that the analogous support for virtual external interrupts.  One can also observe that other architectures (using ARMv8 as an example) don't provide hardware support for sending virtual timer interrupts to the hypervisor for VM's not currently context-switched in.  Instead the hypervisor can keep track of the stimecmp's for VM's, recognize when one of them "fires", and context switch in that VM.  (I can't say specifically how ARMv8 hypervisors deal with this, but one could imagine a scheme similar to what RISC-V imagines for multiplexing OS-level timers onto the one hardware M-mode timer.)

Am I addressing your intended goal or do you have in mind a different goal behind your suggested arch changes?

Greg

On Thu, Sep 10, 2020 at 12:14 AM zhaosiqi (A) via lists.riscv.org <zhaosiqi3=huawei.com@...> wrote:
Hi Greg,

Thanks for the comments.

It seems the proposal is not explicit enough about the type of interrupt. So to answer your question, with vstimecmp, there is in fact a new type of interrupt types. The new type is triggered by vstimecmp but received when the hart is at V==0, might be called SGTI (Supervisor Guest Timer Interrupt, in the same spirit as SGEI). The proposal didn't really distinguish this interrupt with VSTI. With this new interrupt type, this is how things conceptually work: the HS-mode code first receives a SGTI triggered by vstimecmp, consequently, a VSTI is generated by the HS-mode code for VS-mode to handle.

With the current specs, hip.VSTIP is used to represent pending state for both VSTI and SGTI, which can be made to work as shown in the document. Since once SGTI is pending, then the next step is naturally to make a pending VSTI for the guest. What causes the issue is that the enable bits are also shared in existing specs. I believe your comments was caused by another shared aspect which is the delegation bits. The current specs only provide bit 2 in hideleg, which can't be used to control delegation for two types of interrupt.

A better solution might be to introduce a new interrupt type called SGTI and the corresponding pending bits and delegation bits. M-mode delegate SGTI to HS-mode. HS-mode still delegates VSTI to VS-mode. When an interrupt is triggered by vstimecmp, the hart sets SGTI to be pending and generates a trap. The hypervisor then sets VSTI to be pending, and executes the vCPU.

Hope this explains.

Regards,
Siqi


Re: Disabling and re-enabling extensions

Allen Baum
 

OK - taking a step back, what about requiring: any extension spec that doesn't specifically discuss enabling and disabling should be prohibited from enabling or disabling..
I can't see any good use cases - even for power, given the F/D method of dealing with that via the XS bits.
That is a very explicit mechanism, specifically architected for enabling and disabling, with all the side effects and corner cases defined (one hopes).
 

On Thu, Sep 10, 2020 at 4:13 PM Paul Donahue <pdonahue@...> wrote:
The hypervisor spec specifically says that implementations should have writable misa so this isn't theoretical:
"The hypervisor extension is enabled by setting bit 7 in the misa CSR, which corresponds to the letter H. When misa[7] is clear, the hart behaves as though this extension were not implemented, and attempts to use hypervisor CSRs or instructions raise an illegal instruction exception. Implementations that include the hypervisor extension are encouraged not to hardwire misa[7], so that the extension may be disabled. "
And, of course, the misa register says that extensions may be disabled, that both E and I may be implemented and that misa is the way to select between them, etc.  I don't know if anybody has actually done those things, though.


Thanks,

-Paul


On Thu, Sep 10, 2020 at 4:05 PM Allen Baum <allen.baum@...> wrote:
I thought there was already a speced requirement that, at reset, all extensions are enabled.:
    "At reset, the Extensions field shall contain the maximal set of supported extensions, and I shall be selected over E if both are available."

And then there are the corner cases: if there are no misa bits for subsets (e.g Zbb, Zv<whatever>), then they can't be enabled or disabled.
So, why should misa. bits for the full extensions allow enabling/disabling?
What does it mean for misa.G to be enabled or disabled? misa.I ?(if misa.E isn't also present)?

Has anyone identified use cases for turning extensions on and off? (as opposed to FP-like disabled state to save power).
Has anyone ever implemented RW misa bits?
IF not: why not just require those MISA bits to be RdOnly?

On Thu, Sep 10, 2020 at 10:44 AM Greg Favor <gfavor@...> wrote:
A potential second general requirement for all Priv extensions:

- The hart reset state must be fully specified and should generally be UNSPECIFIED except for any key control/status bits that need to be in a specific state.  (This minimizes hardware cost and leaves probably the large majority of state to be initialized by software.)

- The reset state should also specify one of:
    a) The extension is "disabled" (and most, if not all, other extension state is don't care).

    b) The extension is "enabled" and any key control/status bits are initialized to specific values as necessary (with all the rest UNSPECIFIED and left to be initialized by software).

Greg


Re: Disabling and re-enabling extensions

Paul Donahue
 

The hypervisor spec specifically says that implementations should have writable misa so this isn't theoretical:
"The hypervisor extension is enabled by setting bit 7 in the misa CSR, which corresponds to the letter H. When misa[7] is clear, the hart behaves as though this extension were not implemented, and attempts to use hypervisor CSRs or instructions raise an illegal instruction exception. Implementations that include the hypervisor extension are encouraged not to hardwire misa[7], so that the extension may be disabled. "
And, of course, the misa register says that extensions may be disabled, that both E and I may be implemented and that misa is the way to select between them, etc.  I don't know if anybody has actually done those things, though.


Thanks,

-Paul


On Thu, Sep 10, 2020 at 4:05 PM Allen Baum <allen.baum@...> wrote:
I thought there was already a speced requirement that, at reset, all extensions are enabled.:
    "At reset, the Extensions field shall contain the maximal set of supported extensions, and I shall be selected over E if both are available."

And then there are the corner cases: if there are no misa bits for subsets (e.g Zbb, Zv<whatever>), then they can't be enabled or disabled.
So, why should misa. bits for the full extensions allow enabling/disabling?
What does it mean for misa.G to be enabled or disabled? misa.I ?(if misa.E isn't also present)?

Has anyone identified use cases for turning extensions on and off? (as opposed to FP-like disabled state to save power).
Has anyone ever implemented RW misa bits?
IF not: why not just require those MISA bits to be RdOnly?

On Thu, Sep 10, 2020 at 10:44 AM Greg Favor <gfavor@...> wrote:
A potential second general requirement for all Priv extensions:

- The hart reset state must be fully specified and should generally be UNSPECIFIED except for any key control/status bits that need to be in a specific state.  (This minimizes hardware cost and leaves probably the large majority of state to be initialized by software.)

- The reset state should also specify one of:
    a) The extension is "disabled" (and most, if not all, other extension state is don't care).

    b) The extension is "enabled" and any key control/status bits are initialized to specific values as necessary (with all the rest UNSPECIFIED and left to be initialized by software).

Greg


Re: Disabling and re-enabling extensions

Allen Baum
 

I thought there was already a speced requirement that, at reset, all extensions are enabled.:
    "At reset, the Extensions field shall contain the maximal set of supported extensions, and I shall be selected over E if both are available."

And then there are the corner cases: if there are no misa bits for subsets (e.g Zbb, Zv<whatever>), then they can't be enabled or disabled.
So, why should misa. bits for the full extensions allow enabling/disabling?
What does it mean for misa.G to be enabled or disabled? misa.I ?(if misa.E isn't also present)?

Has anyone identified use cases for turning extensions on and off? (as opposed to FP-like disabled state to save power).
Has anyone ever implemented RW misa bits?
IF not: why not just require those MISA bits to be RdOnly?


On Thu, Sep 10, 2020 at 10:44 AM Greg Favor <gfavor@...> wrote:
A potential second general requirement for all Priv extensions:

- The hart reset state must be fully specified and should generally be UNSPECIFIED except for any key control/status bits that need to be in a specific state.  (This minimizes hardware cost and leaves probably the large majority of state to be initialized by software.)

- The reset state should also specify one of:
    a) The extension is "disabled" (and most, if not all, other extension state is don't care).

    b) The extension is "enabled" and any key control/status bits are initialized to specific values as necessary (with all the rest UNSPECIFIED and left to be initialized by software).

Greg


Re: Disabling and re-enabling extensions

Paul Donahue
 

On Wed, Sep 9, 2020 at 8:33 PM Andrew Waterman <andrew@...> wrote:

On Wed, Sep 9, 2020 at 12:14 PM Paul Donahue <pdonahue@...> wrote:


I propose that there should be a sentence in the misa description that says that, unless otherwise specified, when software enables an extension that was previously disabled then all state uniquely associated with that extension is UNSPECIFIED.


I don’t object, but I’m wondering: would different implementations actually be motivated to have different behavior w.r.t. hypervisor state when H is disabled then re-enabled? Put another way, do we have any reason to believe that making the behavior fully specified would unduly burden certain styles of implementation?

I think that the general statement for misa should definitely be that it's UNSPECIFIED.  For instance, implementations may want to power gate a vector unit when V is disabled and that becomes more complicated (though certainly not impossible) if there's a requirement to preserve state.  But the "unless otherwise specified" is important.  That allows for the mepc[1] behavior and, to your question, it would allow H to tighten up the requirements.

As for tightening up H requirements, I agree with Greg that it doesn't seem to add value.  If software wants to preserve the state across a temporary disable then it can context switch the state.  I also don't think that compliance tests would be burdened since they can (if they even have a need for a single test that disables H then reenables H then goes on to use H state) live with the same context switch requirements.  The formal model doesn't need to implement all possible behaviors because no proper software can ever observe the behavior.


Thanks,

-Paul


Re: Disabling and re-enabling extensions

Greg Favor
 

A potential second general requirement for all Priv extensions:

- The hart reset state must be fully specified and should generally be UNSPECIFIED except for any key control/status bits that need to be in a specific state.  (This minimizes hardware cost and leaves probably the large majority of state to be initialized by software.)

- The reset state should also specify one of:
    a) The extension is "disabled" (and most, if not all, other extension state is don't care).

    b) The extension is "enabled" and any key control/status bits are initialized to specific values as necessary (with all the rest UNSPECIFIED and left to be initialized by software).

Greg


Re: Disabling and re-enabling extensions

Greg Favor
 

On Thu, Sep 10, 2020 at 6:47 AM mark <markhimelstein@...> wrote:
Is there a checklist of requirements for priv specifications?
if not should there be?

I'm not sure, off-hand, if there is a many item list to be created or not.  (Andrew probably has better thoughts on this.)  But on the particular topic of this email thread, my leaning would be to say (to all extension TG's) the following:

- An extension spec must be "complete", i.e. it fully specifies all behaviors, including for all corner cases.

- It is alright for some behaviors to be specified as UNSPECIFIED, but it is preferable for specific behavior(s) to be specified - unless this would be burdensome on a worthwhile subset of reasonable implementations.  Further, in the latter case, specifying a bounded set of two allowed behaviors is acceptable if that would be of value to software (or compliance testing or SAIL modeling) instead of just specifying UNSPECIFIED.
 

Regarding the specific question of the H-specific state when disabling then re-enabling the H-extension (or more generally disabling then re-enabling any Priv extension), I suspect the following would be a reasonable specific behavior to specify (in the spirit of the preceding general philosophy):

When misa.H is cleared, the values of all state that become either inaccessible or masked (e.g. treated as _effectively_ a fixed value) are preserved such that these values become visible again when misa.H is set and this state becomes accessible and unmasked.

In other words, flop state is left untouched and any values that must now _appear_ as a fixed value are implemented via physical masking of the flop value (instead of changing the flop state).

Although I would try and argue - for the H-extension - that this provides no value to software (in the seemingly rare cases of this scenario happening).  In which case specifying UNSPECIFIED would seem suitable for the sake of avoiding unnecessary over-specification of behavior.

Lastly, it would seem desirable that all Priv extensions should, by default, have the same specification for what happens when the extension is disabled and then re-enabled.  In which case my own leaning would be towards the "UNSPECIFIED" option as the general default across all priv arch extensions.  Or otherwise towards the "preserve underlying state" option if Andrew or others felt this as being more appropriate.  (Or is there a better default option?)

Greg


Re: Disabling and re-enabling extensions

mark
 

Is there a checklist of requirements for priv specifications?

if not should there be?

If there is, would this be a requirement in that list to specify to some level of clarity (maybe indicating what is allowed to be left unspecified or not for example) what happens when the extension is turned off and on. and maybe provide overall guidance for future extensions.  I suggest it might be best not to just leave this up to reviews to catch these in an ad hoc manner.

Paul brought a broader set of items to light which makes me want to ask that once we answer paul and john's questions are there any others lurking out there from the past or in flight? Do we need to do a little audit?

am I misunderstanding something?

mark


On Wed, Sep 9, 2020 at 12:14 PM Paul Donahue <pdonahue@...> wrote:

In a belated followup to https://github.com/riscv/riscv-isa-manual/issues/332#issuecomment-466581845, I have the same question as John Hauser.  One thing that has changed is that now the architecture has an explicit goal of either describing behavior or using the term UNSPECIFIED.


To restate the question: Various extensions that add architectural state can be enabled/disabled via misa.  What happens to that architectural state if the extension is disabled and later re-enabled?


In particular, what happens to the following state if the associated misa bit is cleared and then set:

  • misa.F: FP registers and fcsr

  • misa.D: upper part of FP registers (when misa.D=0 and misa.F=1)

  • misa.Q: upper part of FP registers (when misa.Q=0 and misa.F=1)

  • misa.S: supervisor CSR state

  • misa.H: hypervisor CSR state

  • misa.V: vector state


The spec is clear about the treatment of mepc[1] and sepc[1] when misa.C is disabled and subsequently re-enabled.  That bit will contain the last written value even if the write comes while C is disabled.


In a different approach, the spec is also clear that if mstatus.FS is set to Off that the FP state is preserved.  However, this S-mode operation is different from the M-mode misa.F control.  And the question remains about disabling portions of the FP registers where implementations may continue to do single-precision operations that write (at least) the lower part of the register.


Implementations that support H are encouraged to make misa.H writable.  While support for disabling D without disabling F is potentially academic, most implementations are expected to allow software to disable and re-enable H.


I propose that there should be a sentence in the misa description that says that, unless otherwise specified, when software enables an extension that was previously disabled then all state uniquely associated with that extension is UNSPECIFIED.



Thanks,


-Paul


881 - 900 of 1227