Στις 2021-04-19 17:39, Bill Huffman έγραψε:

I would also rather disallow composing sPMP and paging. It seems an extra complexity for very little benefit. The argument I understand for sPMP is a poor man's paging. I don't know an argument for having both active at the same time.
Bill

A scenario we discussed at some point was a trusted hypervisor running on HS mode, with e.g. Linux and a trusted service running on VS mode. The trusted hypervisor is usually very small/simple and may not use paging, so hgatp will be set to bare and it'll fallback to PMP/ePMP as the current hypervisor spec mandates. With sPMP the hypervisor will be able to configure its own regions and also isolate Linux from the trusted service, without going through M-mode using PMP/ePMP, this allows for a much more flexible / clean implementation. In other words we can use sPMP as a poor man's paging for HS mode and still use paging for VS mode, in which case when operating on VS mode both MMU and sPMP will be active. Such scenarios by the way (with small trusted hypervisors) are commonly used in mobile phones.

Regards,
Nick

Στις 2021-04-19 01:43, Andrew Waterman έγραψε:

I'm going to push back against the proposal to allocate new cause values for sPMP faults. Allocating new cause values has knock-on effects and incremental costs throughout the architecture (with respect to virtualization and delegation, in particular), and so the bar for adding new ones needs to be set high.
Many of us do not believe that composing sPMP and paging is an essential use case to support--and, in fact, some of us would prefer the architectural economy of disallowing their composition. Even if we do support it, software has sufficient information to distinguish sPMP faults from page faults. The process is indeed cumbersome, as y'all have discussed. But it need not be a performance sink--first of all, because this isn't a common case, and second, because the checks can be performed almost entirely off the critical path. (The sPMP checks only need to be performed if the paging subsystem determines the fault shouldn't have occurred--similar to Linux's existing handling of spurious page faults due to lazy shootdowns on upgrades.)

Right now a system without MMU or with satp set to bare mode, will raise an access fault regardless of PMP/ePMP being active, and that access fault may be handled by M-mode or delegated to S-mode as-is through medeleg. If sPMP is present the same system will get an access fault if sPMP is disabled and a page fault if sPMP is active, not only this is inconsistent, it also doesn't make sense to return a page fault when paging is disabled or even unavailable. Even if we don't use sPMP together with MMU the proposal of raising page faults on sPMP failures is a bit off. I believe we should either treat sPMP in the same way as PMP/ePMP and trigger access faults since we are dealing with physical memory, or treat access faults from S-mode in the same way regardless of sPMP. If we decide to treat access faults from S-mode as page faults so that we can delegate them all together to S-mode through medeleg, I'd expect to be able to do this regardless of sPMP being active.

Regards,
Nick

On 19/04/21 21:23, Nick Kossifidis wrote:

A scenario we discussed at some point was a trusted hypervisor running on HS mode, with e.g. Linux and a trusted service running on VS mode. The trusted hypervisor is usually very small/simple and may not use paging, so hgatp will be set to bare and it'll fallback to PMP/ePMP as the current hypervisor spec mandates. With sPMP the hypervisor will be able to configure its own regions and also isolate Linux from the trusted service

I don't see much difference between setting up sPMP and a constant page table for the guest. It's going to be a few dozens lines of code that only runs once when the hypervisor starts.

Paolo

Nick,

For security reasons, PMP exceptions will almost always be handled in M-mode. sPMP exceptions typically want to be handled in S-mode. I don't think it ever makes sense to have the two types routed to the same exception code. So that seems out to me.

I don't think it ever makes sense to handle PMP access faults in S-mode. It's the wrong privilege level (and S-mode can't write the PMP either).

If we don't allow sPMP and paging at the same time, then the two can use the same exception codes. Perhaps the only issue is the name. The name "page fault" doesn't seem to fit with an sPMP exception. If that's the issue, then let's see if we can solve that.

Bill

On 19/04/21 22:17, Bill Huffman wrote:

Nick,
For security reasons, PMP exceptions will almost always be handled in
M-mode. sPMP exceptions typically want to be handled in S-mode. I
don't think it ever makes sense to have the two types routed to the
same exception code. So that seems out to me.
I don't think it ever makes sense to handle PMP access faults in
S-mode. It's the wrong privilege level (and S-mode can't write the
PMP either).
If we don't allow sPMP and paging at the same time, then the two can
use the same exception codes. Perhaps the only issue is the name.
The name "page fault" doesn't seem to fit with an sPMP exception. If
that's the issue, then let's see if we can solve that.

It really seems to me that the sPMP is not really a PMP but a simplified page table format (which vaguely reminds me of PowerPC BAT registers)...

Paolo

Perhaps we should rename the sPMP to something that sounds more like the simplified page table idea. Maybe Memory Protection Unit. The fact that it resembles PMP doesn't have to appear in the name.

Bill

On 19/04/21 23:37, Bill Huffman wrote:

Perhaps we should rename the sPMP to something that sounds more like
the simplified page table idea. Maybe Memory Protection Unit. The
fact that it resembles PMP doesn't have to appear in the name.

Or just use page tables. I am not convinced that they're so hard to build.

Paolo

They're not hard to build. The point is eliminating the miss time, reducing hardware cost, and improving the granularity issues by having a few address ranges controllable by S-mode.

Στις 2021-04-19 23:17, Bill Huffman έγραψε:

Nick,
For security reasons, PMP exceptions will almost always be handled in
M-mode. sPMP exceptions typically want to be handled in S-mode. I
don't think it ever makes sense to have the two types routed to the
same exception code. So that seems out to me.
I don't think it ever makes sense to handle PMP access faults in
S-mode. It's the wrong privilege level (and S-mode can't write the
PMP either).

We currently get access-fault exceptions in case anyone tries to access a region protected with PMA or PMP, regardless of privilege mode. All PMA and PMP faults from any mode are already routed to the same exception codes anyway. Those exceptions are typically handled by M-mode, and based on mstatus.MPP, firmware may forward them accordingly. That's what OpenSBI currently does -> https://github.com/riscv/opensbi/blob/master/lib/sbi/sbi_trap.c#L260, and here is Linux triggering a segfault in sw upon receiving an access-fault exception -> https://git.kernel.org/pub/scm/linux/kernel/git/riscv/linux.git/tree/arch/riscv/kernel/traps.c?h=fixes#n102 without going though do_page_fault. I don't see why sPMP faults should be treated any differently, we can always get an access fault with mstatus.MPP = S/U from sPMP on M-mode and forward it to S-mode to handle it, like we forward PMA or PMP faults originating from S/U-mode.

The requirement set by the proposal's authors was to use medeleg to delegate sPMP faults to S-mode directly as page-faults, without going through M-mode's trap handler as PMA/PMP access-faults do. Without MMU support on the S-mode OS this won't work, on Linux for example do_page_fault is only available when MMU support is enabled -> https://git.kernel.org/pub/scm/linux/kernel/git/riscv/linux.git/tree/arch/riscv/mm/Makefile?h=fixes#n13, https://git.kernel.org/pub/scm/linux/kernel/git/riscv/linux.git/tree/arch/riscv/kernel/entry.S?h=fixes#n449. So to be clear, treating sPMP faults as page-faults requires the OS to support MMU and expect page-faults, it's only possible when both sPMP and MMU are used at the same time, and the scenario I mentioned previously is also not covered by this approach (since the hypervisor won't use the MMU). That's why we talked about using another cause value for sPMP. If we drop the requirement to delegate sPMP faults through medeleg, we can just treat them as access faults and be done with it, we won't need to change any code as well, access-faults are expected by the OS in any case.

Regards,
Nick

Στις 2021-04-19 23:01, Paolo Bonzini έγραψε:

On 19/04/21 21:23, Nick Kossifidis wrote:

A scenario we discussed at some point was a trusted hypervisor running on HS mode, with e.g. Linux and a trusted service running on VS mode. The trusted hypervisor is usually very small/simple and may not use paging, so hgatp will be set to bare and it'll fallback to PMP/ePMP as the current hypervisor spec mandates. With sPMP the hypervisor will be able to configure its own regions and also isolate Linux from the trusted service

I don't see much difference between setting up sPMP and a constant
page table for the guest. It's going to be a few dozens lines of code
that only runs once when the hypervisor starts.
Paolo

It's possible for an implementation to only support one-stage translation and hardwire hgatp to zero, in which case it makes sense for the hypervisor to have sPMP available.

Regards,
Nick

Στις 2021-04-19 22:47, Jonathan Behrens έγραψε:

How about sPMP is only used if satp.MODE=BARE or virtualization is enabled and hgatp.MODE=BARE? That would enable the trusted hypervisor case, while disallowing an S-mode operating system from enabling both paging and sPMP at the same time.
Jonathan

I'd be ok with this, but I don't see what we gain from it, in the end the hw will still use sPMP + MMU in case the hypervisor uses sPMP and the guest uses paging, because we still want sPMP to be active when guest is running.

Regards,
Nick

Στις 2021-04-20 03:14, Andrew Waterman έγραψε:

On Mon, Apr 19, 2021 at 5:05 PM Nick Kossifidis <mick@...> wrote:

Στις 2021-04-19 23:17, Bill Huffman έγραψε:

Nick,
For security reasons, PMP exceptions will almost always be handled in
M-mode. sPMP exceptions typically want to be handled in S-mode. I
don't think it ever makes sense to have the two types routed to the
same exception code. So that seems out to me.
I don't think it ever makes sense to handle PMP access faults in
S-mode. It's the wrong privilege level (and S-mode can't write the
PMP either).

We currently get access-fault exceptions in case anyone tries to access
a region protected with PMA or PMP, regardless of privilege mode. All
PMA and PMP faults from any mode are already routed to the same
exception codes anyway. Those exceptions are typically handled by
M-mode, and based on mstatus.MPP, firmware may forward them accordingly.
That's what OpenSBI currently does ->
https://github.com/riscv/opensbi/blob/master/lib/sbi/sbi_trap.c#L260,
and here is Linux triggering a segfault in sw upon receiving an
access-fault exception ->
https://git.kernel.org/pub/scm/linux/kernel/git/riscv/linux.git/tree/arch/riscv/kernel/traps.c?h=fixes#n102
without going though do_page_fault. I don't see why sPMP faults should
be treated any differently, we can always get an access fault with
mstatus.MPP = S/U from sPMP on M-mode and forward it to S-mode to handle
it, like we forward PMA or PMP faults originating from S/U-mode.
The requirement set by the proposal's authors was to use medeleg to
delegate sPMP faults to S-mode directly as page-faults, without going
through M-mode's trap handler as PMA/PMP access-faults do. Without MMU
support on the S-mode OS this won't work, on Linux for example
do_page_fault is only available when MMU support is enabled ->
https://git.kernel.org/pub/scm/linux/kernel/git/riscv/linux.git/tree/arch/riscv/mm/Makefile?h=fixes#n13,
https://git.kernel.org/pub/scm/linux/kernel/git/riscv/linux.git/tree/arch/riscv/kernel/entry.S?h=fixes#n449.
So to be clear, treating sPMP faults as page-faults requires the OS to
support MMU and expect page-faults, it's only possible when both sPMP
and MMU are used at the same time, and the scenario I mentioned
previously is also not covered by this approach (since the hypervisor
won't use the MMU).

That sounds like an arbitrary constraint based upon how our arch-specific Linux code is currently written. There's nothing wrong with changing our arch-specific Linux code to be sPMP-aware in the no-MMU case. That doesn't mean we need to compile in the do_page_fault routine and all of the MMU code that it depends on; we'd instead map the page-fault exceptions to a different subroutine for the no-MMU case.

Indeed we can just use the same code path as access-faults and rename the exception as Bill suggested, my question is why not just raise access-faults from sPMP like we do for PMA/PMP, in which case we don't need to change anything in the spec / code. Why do we need to treat sPMP differently than PMP or PMA ? The answer I have so far is to avoid going through M-mode and use medeleg instead, is this enough to justify such a change ?

Regards,
Nick