On Wed, Jul 20, 2022 at 9:08 PM Ved Shanbhogue <ved@...> wrote:

Guo Hi -

I have a few additional question

On Wed, Jul 20, 2022 at 10:20:07AM +0800, Guo Ren wrote:

1. Treatment of Sv32x4 when mencvcfg.PBMT is set is not mentioned. Is that
intensional? Or is it expected to follow the redefinition?

I've mentioned that, and you missed it.

Sv32 implementations could support Svnapot (Chapter~\ref{svnapot}) and Svpbmt (Chapter~\ref{svpbmt}) with reduced physical address space from 34-bit to 31-bit when {\tt menvcfg}.PBMTE is set.

I see you mentioned Sv32 implementations. I am reading it literally as Sv32 as in
one of the presently defined modes of satp. I dont see anywhere in the text a mention
of hgatp or Sv32x4. If you did intend that then it is extremely subtle. Could it be
made more explicit.

I didn't mention satp or hgatp, because menvcfg.PBMTE would affect all
of them. I agree to make it more explicit:
{\tt satp}.Sv32x4 and {\tt hgatp}.Sv32x4 implementation could support
Svnapot (Chapter~\ref{svnapot}) and Svpbmt (Chapter~\ref{svpbmt}) with
reduced physical address space from 34-bit to 31-bit when {\tt
menvcfg}.PBMTE is set.

Further the behavior that bit 31 of Sv32x4 PTE becomes ignored
when menvcfg.PBMTE is 1 is important to call out.

Okay, add the sentence:
The bit 31-29 of any Sv32x4 PTE becomes ignored from address bits when
menvcfg.PBMTE is 1.

3. If menvcfg.PBMT is set but henvcfg.PBMT is not set is it expected that non-zero
values in bits 30:29 will lead 1) to a page fault OR 2) bits 30:29 being part of
PPN[1]?

Select 2), not 30:29. But GUEST.PTE.PPN[1] is 31:20, and
HOST.PTE.PPN[1] is 28:20.

This should be stated in the proposal. Please see later question about a part that
supports Sv32, Svpbmt but not Svnapot. Selecting 2) with bit 31 included makes the bit 31
not reserved on this part even though part does not support Svnapot. So may be it
should be stated in two parts 1) setting menvcfg.PBMTE makes bits 30:29 usable as PBMT bit
and restricts the physical width to 31 bits. 2) The bit 31 is treated as N bit if the
the Svnapot extension is supported else it is ignore by hardware. Is that right?

1) and 2) are Okay.

4. If menvcfg.PBMT is set but henvcfg.PBMT is not set what is treatment of N bit.
Does it continue to be the N bit, or does bit 31 become part of of PPN[1]?

Currently, Svnapot is also enabled by menvcfg.PBMTE for rv32.

By currently, I think you mean "as currently proposed by this extension". Right?
The change in the meaning of PBMTE bit definition to act as a Svnapot enabled but only
for Sv32 is new. You mentioned "for rv32". Was this intentional that this
would be restricted to rv32 or is a behavior of the Sv32 mode and not restricted to rv32?

Yes, It's Sv32, and rv32 is a typo. When menvcfg/henvcfg.PBMTE is 1,
the bit 31 is treated as N-bit if the Svnapot extension is supported
else it is ignored by hardware.

5. If a hypervisor had a legacy 34-bit VM and hypervisor used say Sv39 and menvcfg.PBMT.
was 1. Then the behavior of those 34-bit VMs changes. henvcfg.PBMT does
not help as it was not defined to have effect on Sv32 (see question 3 and 4). How
can the machine support co-existence of existing 32-bit VMs with new 31-bit restricted
VMs?

Your question is how to support "existing 34-bit VMs with new 31-bit
restricted VMs", right?
Just as questions 3 and 4 were answered, Sv39 has menvcfg.PBMT, so the
guest could support both 34-bit VM and 31-bit VM.

So when menvcfg.PBMTE is 1, then Sv32 behavior changes and becomes 31-bit restricted.

Yes, for both satp.PTE and hgatp.PTE in HS-mode.

The hypervisor needs PBMT - for HS mode execution and so turning on/off menvcfg.PBMTE as part
of VM context switching is not an option.

We need care henvcfg.PBMTE to support different VM (34-bit/31-bit)
during VM context switching. VM could turn on/off menvcfg.PBMTE which
would affect its henvcfg.PBMTE.

How can a 34-bit VM be run since the act of
setting menvcfg.PBMTE forces the Sv32 to work in 31-bit restricted mode.

GPA is the virtual address, we could translate 34-bit GPA into 31-bit PA.

Here is the translation behavior:
If G-stage.PBMT = PMA and VS-stage.PBMT = IO, then the final is IO.
If G-stage.PBMT = IO and VS-stage.PBMT=PMA, then the final is IO.

This is not relevant to what I asked (orthogonally;for completeness that list should also include NC).

Okay, If G-stage.PBMT = NC and VS-stage.PBMT=PMA, then the final is NC.

My question was how can one execute a VM that needs 34-bit GPA on a hypervisor that uses
Sv39 for its S-stage and uses PBMTE. So the hypervisors S-stage page tables are Sv39 and since it
needs to use PBMT, menvcfg.PBMTE is set to 1. This hypervisor when it does an sret to V=1 mode
of execution and if the VS uses Sv32 then due to menvcfg.PBMTE being 1 it will get the
31-bit restricted behavior.

menvcfg.PBMTE being 1 wouldn't let VS=1 Sv32 get the 31-bit
restricted, but the henvcfg does. I think the hypervisor could
naturally support 34-bit VM & 31-bit VM.

So I am not sure how a 34-bit VM guest can be run by this
hypervisor. The same VM can run natively by having menvcfg.PBMTE set to 0.

The menvcfg.PBMTE for V=1, but henvcfg.PBMTE for HS-mode. Different
VMs could have different henvcfg.PBMTE setting.

7. The privilege chapters that discuss the [m|h]envcfg.PBMTE bit need to be updated with
the new behavior is now associated with this bit.

Do you mean add:
When [m|h]envcfg.PBMTE for HS.Sv32/VS.Sv32, which's physical address
would be reduced from 34-bit to 31-bit.

I think there is more than width reduction - there is also the new behavior of the bit
acting as a enable for Svnapot. Further on a part that support Svpbmt but not Svnapot
this bit now has the behavior of making bit 31 of the PTE a ignored bit.

Yes, that means we couldn't enable Svnapot without Svpbmt in Sv32. But
we could support Svpbmt separately and just ignored 31-bit.

regards
ved

--
Best Regards
Guo Ren