Date   

Re: G-stage address translation

John Hauser
 

Paul Donahue wrote:
I think that there are a few problems in section 5.5.1 of the hypervisor
spec. It says that G-stage translation follows the same algorithm as
section 4.3.2 with a few changes. However: [... some things are wrong]
It seems the algorithm in section 4.3.2 was edited since section 5.5.1
was written. I'm looking at updating section 5.5.1 to adapt to the
changes.

- John H.


G-stage address translation

Paul Donahue
 

I think that there are a few problems in section 5.5.1 of the hypervisor spec.  It says that G-stage translation follows the same algorithm as section 4.3.2 with a few changes.  However:
1. the first bullet refers to step 1 of section 4.3.2 but should refer to step 2
2. it should specify that VALEN is 34/41/50 for Sv32x4/Sv39x4/Sv48x4
3. it should say that step 1 of 4.3.2 checks for zero extension rather than sign extension


Thanks,

-Paul


Re: Proposal: Delegating Exceptions from VS-mode or VU-mode to U-mode

Anup Patel
 

Hi Yifei,

 

Both full emulated I/O devices (e.g. UART, RTC, PCIe devices, etc) and paravirtual I/O devices (e.g. VirtIO devices, XenPV devices etc) have MMIO registers which are trap-n-emulated by Hypervisors. The paravirtual I/O devices are carefully designed devices where MMIO register programming is minimum so that MMIO trap-n-emulate overhead is minimum across hypervisors.

 

In KVM hypervisor, the full emulated I/O devices are usually emulated in KVM user-space whereas paravirtual I/O devices and pass-through devices are usually provided from KVM kernel-space (Refer, VHost, IRQFD, VFIO, etc). The interrupt-controller and timer are the only critical full emulated I/O devices so for most architectures we have KVM in-kernel emulation of interrupt-controller and timer. In case of KVM RISC-V, the timer emulation is totally in-kernel whereas we are waiting for new interrupt-controller spec for KVM RISC-V in-kernel interrupt-controller emulation.

 

Over decades, the focus across architectures (x86, ARM, PowerPC, etc) and across hypervisors (KVM, Xen, HyperV, VMWare, etc) have been to maximize use of paravirtual I/O devices (i.e. VirtIO, XenPV, etc) and pass-through I/O devices for performance. Modern Guest/VMs usually have mix of pass-through I/O devices, paravirtual I/O devices and full emulated I/O devices where the full emulated I/O devices are only used for less critical things such as UART, RTC, etc. Due to this, none of the architectures have focused on accelerating user-space emulation full emulate I/O devices for KVM (Type2) hypervisors.

 

If the focus of this proposal is to only improve trap-n-emulate performance of less-critical full emulated I/O devices in KVM user-space then it won’t have any significant impact because:

  1. Full emulated I/O devices in KVM user-space are generally in-frequently accessed and less critical devices so overall performance improvement will be less.
  2. We will be having critical full emulated I/O devices in KVM kernel-space (e.g. interrupt-controller) so routing all Guest/VM MMIO traps to KVM user-space will not allow full emulated I/O devices in KVM kernel-space.
  3. This proposal is too focused on user-space MMIO emulation of KVM hypervisor and in my opinion won’t benefit Type1 hypervisor (e.g. Xen, VMWare ESX, HyperV, Xvisor, etc) because these hypervisors do lot of MMIO emulation in their respective kernel spaces.

 

Regards,

Anup

 

From: tech-privileged@... <tech-privileged@...> On Behalf Of Yifei Jiang via lists.riscv.org
Sent: 21 September 2020 19:21
To: tech-privileged@...
Subject: Re: [RISC-V] [tech-privileged] Proposal: Delegating Exceptions from VS-mode or VU-mode to U-mode

 

Hi Anup,

 

Sorry for the confusing description of the background in this proposal. To be clear, we here divide the implementation of trap-and-emulated I/O devices into full emulated I/O devices and paravirtual I/O devices. This proposal only focuses on improving the performance of full emulated I/O devices implemented in userspace, such as UART. Besides, we believe that other mmio emulation devices which require Guest OS to trap and exit to userspace, such as rtc, can also benefit from our proposal.

 

Paravirtual I/O devices, which require the GuestOS to trap and send in-kernel interrupts to interact with VirtIO backend, i.e., vHost you mentioned, are not considered in this proposal. 

 

Regards,

Yifei


Re: Proposal: Accelerating Handling User-level Interrupts

mark
 

i remember using DEC VAX ancillary control processes and writing knob and trackball device drivers in user land in the early 80s. I tried to find something describing this but I think it is lost to antiquity.

does anyone know how they did it? i don't remember ever losing an interrupt or affecting the rest of the system  but then again everything is relative.

mark

On Fri, Sep 18, 2020 at 1:07 AM Yifei Jiang via lists.riscv.org <jiangyifei=huawei.com@...> wrote:
















Hi all,



 



When we applied user-level interrupts in N extension to the Unix-like OS, there is no way to directly handle user-level interrupts in userspace bypassing the kernel. This makes little use of the user-level interrupt mechanism.

In this proposal, we provide a hardware-assisted context switch mechanism to accelerate handling interrupts in the U-mode of Unix-like OS. Our proposal is implemented on the QEMU simulator and KVM virtualization architecture. Evaluation results show that our

proposal improves the block performance by 2.63% to 7.56%. Also, more benefits can be gained by combining this proposal with some other optimizations.



 



The attachment is the detailed proposal. Any comments are welcome.



 



Regards,



Yifei

















--
Mark I Himelstein
CTO RISC-V International
+1-408-250-6611
twitter @mark_riscv


Re: Proposal: Delegating Exceptions from VS-mode or VU-mode to U-mode

Yifei Jiang
 

Hi Anup,
 
Sorry for the confusing description of the background in this proposal. To be clear, we here divide the implementation of trap-and-emulated I/O devices into full emulated I/O devices and paravirtual I/O devices. This proposal only focuses on improving the performance of full emulated I/O devices implemented in userspace, such as UART. Besides, we believe that other mmio emulation devices which require Guest OS to trap and exit to userspace, such as rtc, can also benefit from our proposal.
 
Paravirtual I/O devices, which require the GuestOS to trap and send in-kernel interrupts to interact with VirtIO backend, i.e., vHost you mentioned, are not considered in this proposal. 
 
Regards,
Yifei


Re: Proposal: Delegating Exceptions from VS-mode or VU-mode to U-mode

Yifei Jiang
 

Hi John,
 
Yes, we believe that the new interrupt architecture and an IOMMU, which is regarded as a pass-through method as we knew, can indeed minimize the number of times of GuestOS exiting. However, this proposal targets to optimize user-level virtual devices that require to be implemented using the trap-and-emulated paradigm, such as rtc and UART. 
 
When accessing these devices, I/O path overheads caused by context switches between Guest OS and Host OS always dominate in the whole trap-and-emulated overheads.
Therefore, our proposal can improve the performance by kernel-bypassing I/O paths. According to our experiments performed on the RISC-V QEMU emulator and KVM hypervisor, the performance of UART implemented using our solution becomes 2X faster than the one in the original system. 
 
Regards,
Yifei


Re: Proposal: Delegating Exceptions from VS-mode or VU-mode to U-mode

Yifei Jiang
 

Hi,
 
Thanks for your comment.
 
To solve the security problem about the URET instruction, we further add a field, called HUR, in hstatus to control the behavior of URET instruction. When the hstatus.HUR=1, the privilege mode can switch back to VS-mode/VU-mode by executing the URET instruction. Otherwise, the execution of URET instruction causes an illegal instruction trap. The idea is similar to the field hstatus.HU.
 
The hstatus.HUR is set by the hypervisor only when the vCPU is loaded, and it is cleared only when the vCPU is put. In this case, the vCPU is regarded as a trusted task. So, untrusted user-level tasks can not switch to VS-mode/VU-mode by the URET instruction.
 
Regards,
Yifei


Re: Disabling and re-enabling extensions

Allen Baum
 

I was arguing something different. 
If an extension can be disabled (and is)-
what happens when you execute a disabled instruction? Trap or unspecified?

Not all extensions are instruction related, of course, but instead affect the interpretation of existing instructions (or do both, like H-extension ).

-Allen

On Sep 19, 2020, at 8:29 PM, Jonathan Behrens <behrensj@...> wrote:

Errr... disregard that second part, I misread the previous email

On Sat, Sep 19, 2020 at 11:27 PM Jonathan Behrens <behrensj@...> wrote:
Perhaps “all state no longer associated with any active extension is UNSPECIFIED”?

But it also might be slightly cleaner to talk about the state being unspecified right after an extension is reactivated. I kind of think about extension state as ceasing to exist when disabled which isn’t conceptually quite the same as having a specific but unspecified value.

Jonathan

On Sat, Sep 19, 2020 at 11:13 PM Greg Favor via lists.riscv.org <gfavor=ventanamicro.com@...> wrote:
Given the comments excerpted down below that argue for specifying that extension state is UNSPECIFIED after being disabled and then re-enabled, and given that no particular arguments or use cases have been raised for having a defined behavior like all such state being preserved, can we agree on adding something along the lines of the following statement to the Priv spec's description of the misa CSR.  Note that this still allows an extension to specify something different if it so chooses.

When software enables an extension that was previously disabled, then all state uniquely associated with that extension is UNSPECIFIED (unless otherwise specified by that extension).

Greg

On Thu, Sep 10, 2020 at 1:51 PM Paul Donahue <pdonahue@...> wrote:
On Wed, Sep 9, 2020 at 8:33 PM Andrew Waterman <andrew@...> wrote:

On Wed, Sep 9, 2020 at 12:14 PM Paul Donahue <pdonahue@...> wrote:


I propose that there should be a sentence in the misa description that says that, unless otherwise specified, when software enables an extension that was previously disabled then all state uniquely associated with that extension is UNSPECIFIED.


I don’t object, but I’m wondering: would different implementations actually be motivated to have different behavior w.r.t. hypervisor state when H is disabled then re-enabled? Put another way, do we have any reason to believe that making the behavior fully specified would unduly burden certain styles of implementation?

I think that the general statement for misa should definitely be that it's UNSPECIFIED.  For instance, implementations may want to power gate a vector unit when V is disabled and that becomes more complicated (though certainly not impossible) if there's a requirement to preserve state.  But the "unless otherwise specified" is important.  That allows for the mepc[1] behavior and, to your question, it would allow H to tighten up the requirements.

As for tightening up H requirements, I agree with Greg that it doesn't seem to add value.  If software wants to preserve the state across a temporary disable then it can context switch the state.  I also don't think that compliance tests would be burdened since they can (if they even have a need for a single test that disables H then reenables H then goes on to use H state) live with the same context switch requirements.  The formal model doesn't need to implement all possible behaviors because no proper software can ever observe the behavior.


Thanks,

-Paul





















Re: Disabling and re-enabling extensions

Jonathan Behrens <behrensj@...>
 

Errr... disregard that second part, I misread the previous email

On Sat, Sep 19, 2020 at 11:27 PM Jonathan Behrens <behrensj@...> wrote:
Perhaps “all state no longer associated with any active extension is UNSPECIFIED”?

But it also might be slightly cleaner to talk about the state being unspecified right after an extension is reactivated. I kind of think about extension state as ceasing to exist when disabled which isn’t conceptually quite the same as having a specific but unspecified value.

Jonathan

On Sat, Sep 19, 2020 at 11:13 PM Greg Favor via lists.riscv.org <gfavor=ventanamicro.com@...> wrote:
Given the comments excerpted down below that argue for specifying that extension state is UNSPECIFIED after being disabled and then re-enabled, and given that no particular arguments or use cases have been raised for having a defined behavior like all such state being preserved, can we agree on adding something along the lines of the following statement to the Priv spec's description of the misa CSR.  Note that this still allows an extension to specify something different if it so chooses.

When software enables an extension that was previously disabled, then all state uniquely associated with that extension is UNSPECIFIED (unless otherwise specified by that extension).

Greg

On Thu, Sep 10, 2020 at 1:51 PM Paul Donahue <pdonahue@...> wrote:
On Wed, Sep 9, 2020 at 8:33 PM Andrew Waterman <andrew@...> wrote:

On Wed, Sep 9, 2020 at 12:14 PM Paul Donahue <pdonahue@...> wrote:


I propose that there should be a sentence in the misa description that says that, unless otherwise specified, when software enables an extension that was previously disabled then all state uniquely associated with that extension is UNSPECIFIED.


I don’t object, but I’m wondering: would different implementations actually be motivated to have different behavior w.r.t. hypervisor state when H is disabled then re-enabled? Put another way, do we have any reason to believe that making the behavior fully specified would unduly burden certain styles of implementation?

I think that the general statement for misa should definitely be that it's UNSPECIFIED.  For instance, implementations may want to power gate a vector unit when V is disabled and that becomes more complicated (though certainly not impossible) if there's a requirement to preserve state.  But the "unless otherwise specified" is important.  That allows for the mepc[1] behavior and, to your question, it would allow H to tighten up the requirements.

As for tightening up H requirements, I agree with Greg that it doesn't seem to add value.  If software wants to preserve the state across a temporary disable then it can context switch the state.  I also don't think that compliance tests would be burdened since they can (if they even have a need for a single test that disables H then reenables H then goes on to use H state) live with the same context switch requirements.  The formal model doesn't need to implement all possible behaviors because no proper software can ever observe the behavior.


Thanks,

-Paul





















Re: Disabling and re-enabling extensions

Jonathan Behrens <behrensj@...>
 

Perhaps “all state no longer associated with any active extension is UNSPECIFIED”?

But it also might be slightly cleaner to talk about the state being unspecified right after an extension is reactivated. I kind of think about extension state as ceasing to exist when disabled which isn’t conceptually quite the same as having a specific but unspecified value.

Jonathan

On Sat, Sep 19, 2020 at 11:13 PM Greg Favor via lists.riscv.org <gfavor=ventanamicro.com@...> wrote:
Given the comments excerpted down below that argue for specifying that extension state is UNSPECIFIED after being disabled and then re-enabled, and given that no particular arguments or use cases have been raised for having a defined behavior like all such state being preserved, can we agree on adding something along the lines of the following statement to the Priv spec's description of the misa CSR.  Note that this still allows an extension to specify something different if it so chooses.

When software enables an extension that was previously disabled, then all state uniquely associated with that extension is UNSPECIFIED (unless otherwise specified by that extension).

Greg

On Thu, Sep 10, 2020 at 1:51 PM Paul Donahue <pdonahue@...> wrote:
On Wed, Sep 9, 2020 at 8:33 PM Andrew Waterman <andrew@...> wrote:

On Wed, Sep 9, 2020 at 12:14 PM Paul Donahue <pdonahue@...> wrote:


I propose that there should be a sentence in the misa description that says that, unless otherwise specified, when software enables an extension that was previously disabled then all state uniquely associated with that extension is UNSPECIFIED.


I don’t object, but I’m wondering: would different implementations actually be motivated to have different behavior w.r.t. hypervisor state when H is disabled then re-enabled? Put another way, do we have any reason to believe that making the behavior fully specified would unduly burden certain styles of implementation?

I think that the general statement for misa should definitely be that it's UNSPECIFIED.  For instance, implementations may want to power gate a vector unit when V is disabled and that becomes more complicated (though certainly not impossible) if there's a requirement to preserve state.  But the "unless otherwise specified" is important.  That allows for the mepc[1] behavior and, to your question, it would allow H to tighten up the requirements.

As for tightening up H requirements, I agree with Greg that it doesn't seem to add value.  If software wants to preserve the state across a temporary disable then it can context switch the state.  I also don't think that compliance tests would be burdened since they can (if they even have a need for a single test that disables H then reenables H then goes on to use H state) live with the same context switch requirements.  The formal model doesn't need to implement all possible behaviors because no proper software can ever observe the behavior.


Thanks,

-Paul



















Re: Disabling and re-enabling extensions

Andrew Waterman
 



On Sat, Sep 19, 2020 at 8:13 PM Greg Favor <gfavor@...> wrote:
Given the comments excerpted down below that argue for specifying that extension state is UNSPECIFIED after being disabled and then re-enabled, and given that no particular arguments or use cases have been raised for having a defined behavior like all such state being preserved, can we agree on adding something along the lines of the following statement to the Priv spec's description of the misa CSR.  Note that this still allows an extension to specify something different if it so chooses.

When software enables an extension that was previously disabled, then all state uniquely associated with that extension is UNSPECIFIED (unless otherwise specified by that extension).



Greg

On Thu, Sep 10, 2020 at 1:51 PM Paul Donahue <pdonahue@...> wrote:
On Wed, Sep 9, 2020 at 8:33 PM Andrew Waterman <andrew@...> wrote:

On Wed, Sep 9, 2020 at 12:14 PM Paul Donahue <pdonahue@...> wrote:


I propose that there should be a sentence in the misa description that says that, unless otherwise specified, when software enables an extension that was previously disabled then all state uniquely associated with that extension is UNSPECIFIED.


I don’t object, but I’m wondering: would different implementations actually be motivated to have different behavior w.r.t. hypervisor state when H is disabled then re-enabled? Put another way, do we have any reason to believe that making the behavior fully specified would unduly burden certain styles of implementation?

I think that the general statement for misa should definitely be that it's UNSPECIFIED.  For instance, implementations may want to power gate a vector unit when V is disabled and that becomes more complicated (though certainly not impossible) if there's a requirement to preserve state.  But the "unless otherwise specified" is important.  That allows for the mepc[1] behavior and, to your question, it would allow H to tighten up the requirements.

As for tightening up H requirements, I agree with Greg that it doesn't seem to add value.  If software wants to preserve the state across a temporary disable then it can context switch the state.  I also don't think that compliance tests would be burdened since they can (if they even have a need for a single test that disables H then reenables H then goes on to use H state) live with the same context switch requirements.  The formal model doesn't need to implement all possible behaviors because no proper software can ever observe the behavior.


Thanks,

-Paul


Re: Disabling and re-enabling extensions

Greg Favor
 

Given the comments excerpted down below that argue for specifying that extension state is UNSPECIFIED after being disabled and then re-enabled, and given that no particular arguments or use cases have been raised for having a defined behavior like all such state being preserved, can we agree on adding something along the lines of the following statement to the Priv spec's description of the misa CSR.  Note that this still allows an extension to specify something different if it so chooses.

When software enables an extension that was previously disabled, then all state uniquely associated with that extension is UNSPECIFIED (unless otherwise specified by that extension).

Greg

On Thu, Sep 10, 2020 at 1:51 PM Paul Donahue <pdonahue@...> wrote:
On Wed, Sep 9, 2020 at 8:33 PM Andrew Waterman <andrew@...> wrote:

On Wed, Sep 9, 2020 at 12:14 PM Paul Donahue <pdonahue@...> wrote:


I propose that there should be a sentence in the misa description that says that, unless otherwise specified, when software enables an extension that was previously disabled then all state uniquely associated with that extension is UNSPECIFIED.


I don’t object, but I’m wondering: would different implementations actually be motivated to have different behavior w.r.t. hypervisor state when H is disabled then re-enabled? Put another way, do we have any reason to believe that making the behavior fully specified would unduly burden certain styles of implementation?

I think that the general statement for misa should definitely be that it's UNSPECIFIED.  For instance, implementations may want to power gate a vector unit when V is disabled and that becomes more complicated (though certainly not impossible) if there's a requirement to preserve state.  But the "unless otherwise specified" is important.  That allows for the mepc[1] behavior and, to your question, it would allow H to tighten up the requirements.

As for tightening up H requirements, I agree with Greg that it doesn't seem to add value.  If software wants to preserve the state across a temporary disable then it can context switch the state.  I also don't think that compliance tests would be burdened since they can (if they even have a need for a single test that disables H then reenables H then goes on to use H state) live with the same context switch requirements.  The formal model doesn't need to implement all possible behaviors because no proper software can ever observe the behavior.


Thanks,

-Paul


Re: Proposal: Delegating Exceptions from VS-mode or VU-mode to U-mode

Gernot <gernot.heiser@...>
 

OK, we had a closer look at the N extension, and it doesn’t do much for us, should have looked closer in the first place :-(

Hence I don’t particularly care about this one – sorry for the unnecessary noise.

Gernot

On 20 Sep 2020, at 06:22, John Hauser <jh.riscv@...> wrote:

Gernot wrote:
User-level drivers are a core properties of (well-designed)
microkernels, and microkernels are pretty much the only choice for
safety- and security-critical systems, and the only kind of OS that
is feasible to prove correct (see seL4).

And if you run drivers in user mode, then being able to route
interrupts directly to the user-level handlers without invoking the
kernel would seem to pretty much eliminate the performance
disadvantage microkernels have compared to Linux. In fact, my
immediate reaction seeing this extension was “yeah!”.
Aside from the fact that doing this requires some version of the
quasi-deprecated N extension be implemented in addition to the
hypervisor extension, the main problem with this idea is how the
hardware decides to send memory access traps to U mode versus HS mode.
The delegation provided by sideleg is too crude to suffice for this
purpose. Instead, the choice must be encoded on a per-page basis
in the G-stage page tables---which is what Huawei's proposal does,
naturally enough.

But if our real goal is for virtual machines to run as fast as
possible, it seems to me the more important subgoal is to minimize
the number of times when memory accesses to a virtual device must be
trapped and emulated, by maximizing the opportunity for a guest OS
to directly control physical devices without emulation. The hardware
components needed for this include the new interrupt architecture that
is being developed, plus a sufficiently capable IOMMU, a proposal for
which is being drafted by a different informal group of interested
parties.

Even with this new hardware (in whatever form it actually becomes
standard for RISC-V), we can expect that some need to trap-and-emulate
for virtual devices will remain. But at the current time, I don't know
how we can predict very well the performance cost of those traps that
remain, or how much improvement we would get from adopting Huawei's
proposal. For all I know, it may be that, once we have the new
interrupt architecture and an IOMMU, the added improvement from
Huawei's proposal is barely noticeable.

Maybe it will be, and maybe it won't. All I'm suggesting is, it would
be better to evaluate that choice after these other essential pieces
are in place, and after all the relevant software has been completed
and optimized, as Anup Patel spoke of.

One counterargument might be to claim that there is sufficient long-
term market interest in supporting the hypervisor extension as best
as possible _without_ the new interrupt architecture and IOMMU. I'll
leave it to others to try to make that case.

- John Hauser





Re: Proposal: Delegating Exceptions from VS-mode or VU-mode to U-mode

John Hauser
 

Gernot wrote:
User-level drivers are a core properties of (well-designed)
microkernels, and microkernels are pretty much the only choice for
safety- and security-critical systems, and the only kind of OS that
is feasible to prove correct (see seL4).

And if you run drivers in user mode, then being able to route
interrupts directly to the user-level handlers without invoking the
kernel would seem to pretty much eliminate the performance
disadvantage microkernels have compared to Linux. In fact, my
immediate reaction seeing this extension was “yeah!”.
Aside from the fact that doing this requires some version of the
quasi-deprecated N extension be implemented in addition to the
hypervisor extension, the main problem with this idea is how the
hardware decides to send memory access traps to U mode versus HS mode.
The delegation provided by sideleg is too crude to suffice for this
purpose. Instead, the choice must be encoded on a per-page basis
in the G-stage page tables---which is what Huawei's proposal does,
naturally enough.

But if our real goal is for virtual machines to run as fast as
possible, it seems to me the more important subgoal is to minimize
the number of times when memory accesses to a virtual device must be
trapped and emulated, by maximizing the opportunity for a guest OS
to directly control physical devices without emulation. The hardware
components needed for this include the new interrupt architecture that
is being developed, plus a sufficiently capable IOMMU, a proposal for
which is being drafted by a different informal group of interested
parties.

Even with this new hardware (in whatever form it actually becomes
standard for RISC-V), we can expect that some need to trap-and-emulate
for virtual devices will remain. But at the current time, I don't know
how we can predict very well the performance cost of those traps that
remain, or how much improvement we would get from adopting Huawei's
proposal. For all I know, it may be that, once we have the new
interrupt architecture and an IOMMU, the added improvement from
Huawei's proposal is barely noticeable.

Maybe it will be, and maybe it won't. All I'm suggesting is, it would
be better to evaluate that choice after these other essential pieces
are in place, and after all the relevant software has been completed
and optimized, as Anup Patel spoke of.

One counterargument might be to claim that there is sufficient long-
term market interest in supporting the hypervisor extension as best
as possible _without_ the new interrupt architecture and IOMMU. I'll
leave it to others to try to make that case.

- John Hauser


Re: Proposal: Delegating Exceptions from VS-mode or VU-mode to U-mode

Jonathan Behrens <behrensj@...>
 


On Sat, Sep 19, 2020 at 5:29 AM Gernot via lists.riscv.org <gernot.heiser=data61.csiro.au@...> wrote:
User-level drivers are a core properties of (well-designed) microkernels, and microkernels are pretty much the only choice for safety- and security-critical systems, and the only kind of OS that is feasible to prove correct (see seL4).

And if you run drivers in user mode, then being able to route interrupts directly to the user-level handlers without invoking the kernel would seem to pretty much eliminate the performance disadvantage microkernels have compared to Linux. In fact, my immediate reaction seeing this extension was “yeah!”.

You can achieve the same thing using the hypervisor extension analogously to how M/U systems can avoid user-level interrupts by switching to M/S/U. Instead of running the kernel in S-mode and drivers in U-mode, run the kernel in HS-mode and the drivers in VS-mode.

Jonathan


Re: Proposal: Delegating Exceptions from VS-mode or VU-mode to U-mode

David Horner
 


On 2020-09-19 5:29 a.m., Gernot wrote:
On 19 Sep 2020, at 15:53, Andrew Waterman <andrew@...> wrote:

The N extension is effectively deprecated. We don’t see sufficient demand for user-level interrupts in managed/Unix-like environments to pursue that approach at this time.

That’s a real concern, and I seem to have missed any discussion about this. I suspect that this also reflects a confusion resulting from the use of “Unix-like environments”, which seems to get applied to any protected-mode OS.

User-level drivers are a core properties of (well-designed) microkernels, and microkernels are pretty much the only choice for safety- and security-critical systems, and the only kind of OS that is feasible to prove correct (see seL4).

And if you run drivers in user mode, then being able to route interrupts directly to the user-level handlers without invoking the kernel would seem to pretty much eliminate the performance disadvantage microkernels have compared to Linux.

I don't know if these developments will address your concerns, but I see the synergy of these two task groups, Fast-interrupts and Code Size Reduction, as potentially reducing the overhead for M-mode to initiate and manage drivers running in U-mode.

I invite you to join these groups to contribute in directing their ability to support this important segment of the ecosystem.

In fact, my immediate reaction seeing this extension was “yeah!”.

As I have aligned aspirations,I'm hoping that your response will, along with mine,  be "yeah!" :

       that running micro-kernel (and other like) components in U-mode will be a first class supported reality obtained through these initiatives.


Gernot


Re: Proposal: Delegating Exceptions from VS-mode or VU-mode to U-mode

Gernot <gernot.heiser@...>
 

On 19 Sep 2020, at 15:53, Andrew Waterman <andrew@...> wrote:

The N extension is effectively deprecated. We don’t see sufficient demand for user-level interrupts in managed/Unix-like environments to pursue that approach at this time.

That’s a real concern, and I seem to have missed any discussion about this. I suspect that this also reflects a confusion resulting from the use of “Unix-like environments”, which seems to get applied to any protected-mode OS.

User-level drivers are a core properties of (well-designed) microkernels, and microkernels are pretty much the only choice for safety- and security-critical systems, and the only kind of OS that is feasible to prove correct (see seL4).

And if you run drivers in user mode, then being able to route interrupts directly to the user-level handlers without invoking the kernel would seem to pretty much eliminate the performance disadvantage microkernels have compared to Linux. In fact, my immediate reaction seeing this extension was “yeah!”.

Gernot


Re: Proposal: Delegating Exceptions from VS-mode or VU-mode to U-mode

Andrew Waterman
 

The N extension is effectively deprecated. We don’t see sufficient demand for user-level interrupts in managed/Unix-like environments to pursue that approach at this time.

We do see demand for this feature in embedded systems with two levels of privilege. In this case, the proposal is to run the trusted piece in M-mode and the less-trusted piece in “bare S-mode”: i.e., S-mode without virtual memory.

This approach covers the interesting use cases without additional architectural complication, and in particular, the hypervisor architecture need not care.

On Fri, Sep 18, 2020 at 1:03 AM Yifei Jiang via lists.riscv.org <jiangyifei=huawei.com@...> wrote:
















Hi all,



 



In this proposal, we extended N extension and applied it to H extension for improving the performance of virtual I/O devices in the virtualization scenario. We proposed a new mechanism to delegate exceptions from VS-mode/VU-mode

to U-mode. A solution is also implemented based on QEMU simulator and KVM virtualization architecture. Evaluation results show that our proposal achieves nearly 2x faster synchronous I/O processing speed than the orginal system.



 



The attachment is the detailed proposal. Any comments are welcome.



 



Regards,



Yifei


















Re: Proposal: Delegating Exceptions from VS-mode or VU-mode to U-mode

Anup Patel
 

Hi,

 

The context switching overhead for VirtIO interrupts that this proposal is trying to solve is already solved across architectures by:

  1. KVM in-kernel VirtIO backends emulation (i.e. VHost) (Already available)
  2. KVM in-kernel interrupt-controller emulation and KVM irqfd (Available on other architectures but not currently available for KVM RISC-V)

 

The point2 is intentionally not done for KVM RISC-V because we don’t want in-kernel PLIC emulation due to lack of MSI support and Virtualization support in PLIC. Also, the in-kernel PLIC emulation will become redundant once new interrupt controller spec is available. We will go for in-kernel interrupt controller emulation once new interrupt controller spec is available.

 

Is there any other advantage of this proposal over VHost + In-kernel interrupt-controller ??

 

Regards,

Anup

 

From: tech-privileged@... <tech-privileged@...> On Behalf Of Yifei Jiang via lists.riscv.org
Sent: 18 September 2020 13:33
To: tech-privileged@...
Subject: [RISC-V] [tech-privileged] Proposal: Delegating Exceptions from VS-mode or VU-mode to U-mode

 

Hi all,

 

In this proposal, we extended N extension and applied it to H extension for improving the performance of virtual I/O devices in the virtualization scenario. We proposed a new mechanism to delegate exceptions from VS-mode/VU-mode to U-mode. A solution is also implemented based on QEMU simulator and KVM virtualization architecture. Evaluation results show that our proposal achieves nearly 2x faster synchronous I/O processing speed than the orginal system.

 

The attachment is the detailed proposal. Any comments are welcome.

 

Regards,

Yifei


Re: Proposal: Delegating Exceptions from VS-mode or VU-mode to U-mode

Phil McCoy
 

If I understand correctly, there is a security issue around the URET instruction.  All the controls for URET are accessible from User mode.  This is OK in the scenario you describe where U-Mode is trusted (the U-Mode code is the UART emulation, which is effectively just a privilege-reduced portion of the hypervisor).

There does not appear to be any way for the hypervisor to prevent untrusted user-mode code from URET'ing into any arbitrary location in the virtualized guest (VS/VU) software by programming uepc and ustatus.

821 - 840 of 1184