Date   

Re: Hypervisor interrupt enables

Scott Johnson
 

Perhaps you could say that if V=0 then hideleg is ignored, so that the virtualized interrupts could only go to HS-mode (there is no S-mode when misa.H=1). But I don't believe the spec says this.

There is also the original question of HS-mode interrupts when in VS-mode. Should sstatus.SIE apply here? It depends on if HS-mode is considered more privileged, less privileged, or the same privilege as VS-mode. John Hauser says that VS-mode is less privileged than HS-mode, but I don't believe the spec says this. Spike considers them to be the same privilege.



On Wed, Mar 10, 2021 at 09:12 AM, Allen Baum wrote:
Exactly the right question - I mean the latter, when V=1. IF that's the case, 
 - an interrupt taken with a hart in U-mode will be handled by S mode.
 - an interrupt taken with a hart in VU-mode will be handled by VS mode.
So the original question "Are interrupts destined for VS-mode globally enabled when in U-mode?" can't actually happen; because if it's in U-mode, the interrupt would go not to VS, but to S.
 
I am perfectly happy to be told that I'm not understanding how this works, of course.
 
 

On Tue, Mar 9, 2021 at 3:07 PM Jonathan Behrens <behrensj@...> wrote:
 

On Tue, Mar 9, 2021 at 5:30 PM Allen Baum via lists.riscv.org <allen.baum=esperantotech.com@...> wrote:
This is a stupid question: if VS mode is enabled - can a core ever be in Umode, or must it be in VU mode?
IF the answer is: when virtualization is on, the core cannot be in S mode or U mode, but only in VS and modes (and M...) then the question is moot.
 
What does "VS mode is enabled" mean? If you are referring to misa.H=1, then yes a core can be in U-mode while VS-mode is enabled. However, if by VS-mode enabled you mean V=1 (recalling that V stands for the current virtualization mode) then it isn't possible because the only modes allowed when V=1 are VU-mode and VS-mode.
 
Jonathan


Re: Hypervisor interrupt enables

Allen Baum
 

Exactly the right question - I mean the latter, when V=1. IF that's the case, 
 - an interrupt taken with a hart in U-mode will be handled by S mode.
 - an interrupt taken with a hart in VU-mode will be handled by VS mode.
So the original question "Are interrupts destined for VS-mode globally enabled when in U-mode?" can't actually happen; because if it's in U-mode, the interrupt would go not to VS, but to S.

I am perfectly happy to be told that I'm not understanding how this works, of course.

On Tue, Mar 9, 2021 at 3:07 PM Jonathan Behrens <behrensj@...> wrote:

On Tue, Mar 9, 2021 at 5:30 PM Allen Baum via lists.riscv.org <allen.baum=esperantotech.com@...> wrote:
This is a stupid question: if VS mode is enabled - can a core ever be in Umode, or must it be in VU mode?
IF the answer is: when virtualization is on, the core cannot be in S mode or U mode, but only in VS and modes (and M...) then the question is moot.

What does "VS mode is enabled" mean? If you are referring to misa.H=1, then yes a core can be in U-mode while VS-mode is enabled. However, if by VS-mode enabled you mean V=1 (recalling that V stands for the current virtualization mode) then it isn't possible because the only modes allowed when V=1 are VU-mode and VS-mode.

Jonathan


Re: Correspondence between hedeleg and medeleg writeable bits?

James Robinson
 

Pinging to see if any of the architects can chime in to clarify whether H-extension cores are supposed to have bits which are writable in hedeleg but not medeleg, as per the current spec?


Re: Hypervisor interrupt enables

Jonathan Behrens <behrensj@...>
 


On Tue, Mar 9, 2021 at 5:30 PM Allen Baum via lists.riscv.org <allen.baum=esperantotech.com@...> wrote:
This is a stupid question: if VS mode is enabled - can a core ever be in Umode, or must it be in VU mode?
IF the answer is: when virtualization is on, the core cannot be in S mode or U mode, but only in VS and modes (and M...) then the question is moot.

What does "VS mode is enabled" mean? If you are referring to misa.H=1, then yes a core can be in U-mode while VS-mode is enabled. However, if by VS-mode enabled you mean V=1 (recalling that V stands for the current virtualization mode) then it isn't possible because the only modes allowed when V=1 are VU-mode and VS-mode.

Jonathan


Re: Hypervisor interrupt enables

Paolo Bonzini
 

There are both "HU" and VU-modes indeed.

There were very old discussions on the public isa-dev mailing list about the advantages of this method (shared with s390 and x86 among others) to the "pure H-mode" used by ARM (especially before the introduction of VHE) and POWER. H-mode was reserved by early version of the RISC-V privileged specification, but those discuss ultimately led to the removal of all mentiond of H-mode and the introduction of HS-mode in its stead.


Thanks,

Paolo


Il mar 9 mar 2021, 23:33 Allen Baum <allen.baum@...> ha scritto:
This is a stupid question: if VS mode is enabled - can a core ever be in Umode, or must it be in VU mode?
IF the answer is: when virtualization is on, the core cannot be in S mode or U mode, but only in VS and modes (and M...) then the question is moot.


Re: Hypervisor interrupt enables

Scott Johnson
 

When hypervisor is enabled, S-mode becomes HS-mode, and VS-mode and VU-mode are newly available.

U-mode is still available.


On Tue, Mar 9, 2021 at 02:29 PM, Allen Baum wrote:
This is a stupid question: if VS mode is enabled - can a core ever be in Umode, or must it be in VU mode?
IF the answer is: when virtualization is on, the core cannot be in S mode or U mode, but only in VS and modes (and M...) then the question is moot.

On Tue, Mar 9, 2021 at 7:43 AM Scott Johnson <scott.johnson@...> wrote:
The question is about VS-mode interrupts, relevant even without the N extension.

Are interrupts destined for VS-mode globally enabled when in U-mode?

If U-mode is less privileged than VS-mode, then yes.
If U-mode is the same privilege as VS-mode, then vsstatus.SIE applies.
If U-mode is more privileged than VS-mode, then they are disabled. (Priv spec 3.1.6.1 says "Interrupts for lower-privilege modes, w<x, are always globally disabled regardless of the setting of any global wIE bit for the lower-privilege mode.)

So the relative privilege of U vs VS is important, and apparently unspecified.

The graph of privileges is how I had pictured it in my head before this issue came up. But it doesn't address this question.

 

 


Re: Hypervisor interrupt enables

Allen Baum
 

This is a stupid question: if VS mode is enabled - can a core ever be in Umode, or must it be in VU mode?
IF the answer is: when virtualization is on, the core cannot be in S mode or U mode, but only in VS and modes (and M...) then the question is moot.

On Tue, Mar 9, 2021 at 7:43 AM Scott Johnson <scott.johnson@...> wrote:
The question is about VS-mode interrupts, relevant even without the N extension.

Are interrupts destined for VS-mode globally enabled when in U-mode?

If U-mode is less privileged than VS-mode, then yes.
If U-mode is the same privilege as VS-mode, then vsstatus.SIE applies.
If U-mode is more privileged than VS-mode, then they are disabled. (Priv spec 3.1.6.1 says "Interrupts for lower-privilege modes, w<x, are always globally disabled regardless of the setting of any global wIE bit for the lower-privilege mode.)

So the relative privilege of U vs VS is important, and apparently unspecified.

The graph of privileges is how I had pictured it in my head before this issue came up. But it doesn't address this question.


Re: Hypervisor interrupt enables

Scott Johnson
 

The question is about VS-mode interrupts, relevant even without the N extension.

Are interrupts destined for VS-mode globally enabled when in U-mode?

If U-mode is less privileged than VS-mode, then yes.
If U-mode is the same privilege as VS-mode, then vsstatus.SIE applies.
If U-mode is more privileged than VS-mode, then they are disabled. (Priv spec 3.1.6.1 says "Interrupts for lower-privilege modes, w<x, are always globally disabled regardless of the setting of any global wIE bit for the lower-privilege mode.)

So the relative privilege of U vs VS is important, and apparently unspecified.

The graph of privileges is how I had pictured it in my head before this issue came up. But it doesn't address this question.


Re: Correspondence between hedeleg and medeleg writeable bits?

Scott Johnson
 

Funny, I was just asking the same question. https://github.com/riscv/riscv-isa-sim/issues/668


Correspondence between hedeleg and medeleg writeable bits?

James Robinson
 

Riscv-privileged_1.12-draft, Table 6.2 gives a list of hedeleg bits which must be writable, and it is stated:

"Requiring that certain bits of hedeleg be writable reduces some of the burden on a hypervisor to handle variations of implementation."

I see that the list of hedeleg writable bits includes bits which are not writable in medeleg in the spike implementation (Illegal Instruction, Load/Store address misaligned, Load/Store/Inst Access Fault).

Is it intended that these bits should be writable even if they are not writable in medeleg, and that the writability the corresponding bits in medeleg remains an implementation choice?

In which case, some exceptions, although writable in hedeleg, will never actually be delegated to VS-mode by the hardware?

Is machine mode software expected (under certain circumstances) to emulate delegation of the exception by setting up the correct state and executing mret?

Thanks,
James


Re: Hypervisor interrupt enables

John Hauser
 

Scott Johnson wrote:
What about U-mode? Are VS-mode interrupts enabled when in U-mode?

I would guess not...so the hierarchy is M > S > U > VS > VU.
The intended hierarchy is what mathematicians call a partial order:

M > HS > U and HS > VS > VU

In other words, HS has two children in the graph.

Interrupts are globally enabled for VS level if the current operating
mode has less privilege than VS or if the current operating mode is
VS and vsstatus.SIE = 1. Because U and VS modes are unordered in the
privilege hierarchy, U does not have less privilege than VS, or vice
versa.

This would become painfully relevant if the N extension for user-level
interrupts were ever made fully compatible with the hypervisor
extension. The original intent was that U-level interrupts not be able
to interrupt VS mode, and VS-level interrupts not be able to interrupt
U mode.

However, as it currently stands, the incomplete N extension has not
been updated to be fully compatible with the hypervisor extension, and
what's more, there's a movement to discourage the N extension, or at
the very least, to discourage it when supervisor mode is implemented.
If the N extension and S mode were forever mutually exclusive, we could
more easily adopt the full order you propose, with U > VS.

- John Hauser


Re: Hypervisor interrupt enables

Jonathan Behrens <behrensj@...>
 

Oh, good point. I'd forgotten about the hstatus.HU bit which gives U-mode the ability to do loads and stores in the VS-mode address space. U-mode still doesn't have access to the VS-mode CSRs (or virtual memory fence instructions AFAIK), but this does complicate the story.

On Mon, Mar 8, 2021 at 12:01 PM Scott Johnson via lists.riscv.org <scott.johnson=arilinc.com@...> wrote:
There is talk in the spec of running portions of the hypervisor code in U-mode.

On Mon, Mar 8, 2021 at 08:50 AM, Jonathan Behrens wrote:
I don't think calling U-mode more privileged than VS-mode is quite right either. U-mode software cannot access VS-mode CSRs for instance.
 
A different way of looking at it would be that VS-mode only exists when virtualization is enabled, so if you are running in U-mode (meaning virtualization is disabled) then VS-mode software cannot receive interrupts because there is no trap handler or other software configured for that mode.
 
Jonathan

On Mon, Mar 8, 2021 at 11:32 AM Scott Johnson via lists.riscv.org <scott.johnson=arilinc.com@...> wrote:
Ugh, correction: M > HS > U > VS > VU.

On Mon, Mar 8, 2021 at 08:27 AM, Scott Johnson wrote:

I would guess not...so the hierarchy is M > S > U > VS > VU.

 

 


Re: Hypervisor interrupt enables

Scott Johnson
 

There is talk in the spec of running portions of the hypervisor code in U-mode.


On Mon, Mar 8, 2021 at 08:50 AM, Jonathan Behrens wrote:
I don't think calling U-mode more privileged than VS-mode is quite right either. U-mode software cannot access VS-mode CSRs for instance.
 
A different way of looking at it would be that VS-mode only exists when virtualization is enabled, so if you are running in U-mode (meaning virtualization is disabled) then VS-mode software cannot receive interrupts because there is no trap handler or other software configured for that mode.
 
Jonathan

On Mon, Mar 8, 2021 at 11:32 AM Scott Johnson via lists.riscv.org <scott.johnson=arilinc.com@...> wrote:
Ugh, correction: M > HS > U > VS > VU.

On Mon, Mar 8, 2021 at 08:27 AM, Scott Johnson wrote:

I would guess not...so the hierarchy is M > S > U > VS > VU.

 

 


Re: Hypervisor interrupt enables

Jonathan Behrens <behrensj@...>
 

I don't think calling U-mode more privileged than VS-mode is quite right either. U-mode software cannot access VS-mode CSRs for instance.

A different way of looking at it would be that VS-mode only exists when virtualization is enabled, so if you are running in U-mode (meaning virtualization is disabled) then VS-mode software cannot receive interrupts because there is no trap handler or other software configured for that mode.

Jonathan


On Mon, Mar 8, 2021 at 11:32 AM Scott Johnson via lists.riscv.org <scott.johnson=arilinc.com@...> wrote:
Ugh, correction: M > HS > U > VS > VU.

On Mon, Mar 8, 2021 at 08:27 AM, Scott Johnson wrote:

I would guess not...so the hierarchy is M > S > U > VS > VU.


Re: Hypervisor interrupt enables

Scott Johnson
 

Ugh, correction: M > HS > U > VS > VU.


On Mon, Mar 8, 2021 at 08:27 AM, Scott Johnson wrote:

I would guess not...so the hierarchy is M > S > U > VS > VU.


Re: Hypervisor interrupt enables

Scott Johnson
 

I think it makes more sense this way, though I concur that a spec clarification is needed.

I'm not sure I understand Andrew's explanation though. How could VS-mode code block HS-mode interrupts, since VS-mode has no control over sstatus.SIE?

What about U-mode? Are VS-mode interrupts enabled when in U-mode?

I would guess not...so the hierarchy is M > S > U > VS > VU.

When explained this way, my confusion about sstatus vs vsstatus disappears, since when V=1, HS-mode interrupts are globally enabled regardless of any *status.SIE bit.


On Sun, Mar 7, 2021 at 12:14 PM, John Hauser wrote:
I wrote:
Interrupts are always
enabled/disabled globally for HS level by the HS-level sstatus.SIE.
Correction: Interrupts are globally enabled for HS level if the
current privilege mode has less privilege than HS (U, VS, or VU mode)
or if the current privilege mode is HS and the HS-level sstatus.SIE
= 1.

- John Hauser

 
d-        state.mip = set_field(state.mip, MIP_MSIP, bytes[0]);


Re: Hypervisor interrupt enables

Sandro Pinto
 


Scott Johnson wrote:
> I expect, and have observed in Spike, that `sstatus.SIE` will apply
> to interrupts delegated to HS-mode, and `vsstatus.SIE` will apply to
> interrupts delegated to VS-mode.
>
> But I cannot find this explained anywhere in the Hypervisor 0.6.1
> spec. Am I missing it somewhere? There is this statement
> ( https://github.com/riscv/riscv-isa-manual/blob/5ca937811c13ae142f783e69f61f6a402694ebe0/src/hypervisor.tex#L1154-L1156
> ) about `vsstatus` substituting for `sstatus` when V=1, but that
> seems incomplete because `sstatus.SIE` apparently still applies to
> some interrupts even when in VS-mode.

Your expectation about the roles of sstatus.SIE and vsstatus.SIE
is correct, and you're probably also right to say that this isn't
explained clearly in the current Privileged Architecture document.

Definitely the Privileged Architecture document needs additional
clarification with regard to this topic.

A few months ago we had a similar problem with QEMU.
We needed to cross-check with Andrew and provide a fix to QEMU:

Sandro


Re: Hypervisor interrupt enables

John Hauser
 

I wrote:
Interrupts are always
enabled/disabled globally for HS level by the HS-level sstatus.SIE.
Correction: Interrupts are globally enabled for HS level if the
current privilege mode has less privilege than HS (U, VS, or VU mode)
or if the current privilege mode is HS and the HS-level sstatus.SIE
= 1.

- John Hauser


Re: Hypervisor interrupt enables

John Hauser
 

Scott Johnson wrote:
I expect, and have observed in Spike, that `sstatus.SIE` will apply
to interrupts delegated to HS-mode, and `vsstatus.SIE` will apply to
interrupts delegated to VS-mode.

But I cannot find this explained anywhere in the Hypervisor 0.6.1
spec. Am I missing it somewhere? There is this statement
( https://github.com/riscv/riscv-isa-manual/blob/5ca937811c13ae142f783e69f61f6a402694ebe0/src/hypervisor.tex#L1154-L1156
) about `vsstatus` substituting for `sstatus` when V=1, but that
seems incomplete because `sstatus.SIE` apparently still applies to
some interrupts even when in VS-mode.
Your expectation about the roles of sstatus.SIE and vsstatus.SIE
is correct, and you're probably also right to say that this isn't
explained clearly in the current Privileged Architecture document.

Related question: it seems that (in Spike, at least) interrupts
delegated to HS-mode are blocked by `sstatus.SIE` even from VS-mode.
I suppose that is spec-compliant based on this statement
( https://github.com/riscv/riscv-isa-manual/blob/5ca937811c13ae142f783e69f61f6a402694ebe0/src/supervisor.tex#L372-L374
) since the current privilege mode when in VS-mode is not less than S
(right?).
I'm afraid I don't understand what you're asking here, since to my eyes
it seems to be a repeat of your first point. Interrupts are always
enabled/disabled globally for HS level by the HS-level sstatus.SIE.

If VS-mode was considered less privileged than HS-mode then my first
question above would be moot.
Whether or not the manual explicitly says so, it definitely intends
that VS mode is less privileged than HS mode.

- John Hauser


Hypervisor interrupt enables

Scott Johnson
 

I expect, and have observed in Spike, that `sstatus.SIE` will apply to interrupts delegated to HS-mode, and `vsstatus.SIE` will apply to interrupts delegated to VS-mode.
 
But I cannot find this explained anywhere in the Hypervisor 0.6.1 spec. Am I missing it somewhere? There is this statement about `vsstatus` substituting for `sstatus` when V=1, but that seems incomplete because `sstatus.SIE` apparently still applies to some interrupts even when in VS-mode.
 
Related question: it seems that (in Spike, at least) interrupts delegated to HS-mode are blocked by `sstatus.SIE` even from VS-mode. I suppose that is spec-compliant based on this statement since the current privilege mode when in VS-mode is not less than S (right?).
 
But is that really the intended/desired behavior?
 
If VS-mode was considered less privileged than HS-mode then my first question above would be moot.
 
Scott
 

741 - 760 of 1274