1. Tech-Unixplatformspec
  2. Topics

Two alternatives for architecture extension to specify address/page-based memory types

Greg Favor
 

To all the system software people on the tech-unixplatformspec and software email lists:

The tech-virt-mem Task Group is considering two alternative approaches for specifying address/page-based attributes - which are analogous to "memory types" in x86 ( WB/WC/UC) and ARMv8 (Normal cacheable, Normal noncacheable, Device).  This email attempts to get further feedback from system software / device driver people as to the practical useability, pros and cons, and preference for the two approaches (at a gross first-order level, so I won't belabor the details of the two approaches).  Also note that both approaches would override the default attributes that come from RISC-V PMAs - which are very roughly analogous to x86 MTRRs.

Option #1 - VA/Page-Based Attributes
This approach utilizes 2-3 bits in page table entries to specify a memory type for the page (roughly corresponding to the above x86 and ARMv8 memory types).  This approach is similar to the approach supported by x86 and ARMv8, and presumably would directly mesh with existing Linux support for this type of approach.

Option #2 - Physical Address-Based Attributes
This approach uses the top 2-3 bits of a 64-bit PA (physical address) to specify a memory type for that address value.  This approach applies equally whether MMU address translation is On or Off (where Off corresponds to RISC-V M-mode and Bare translation modes).  When translation is On, these bits likely (like above) would occupy extra bits in page table entries.  When translation is Off, the high bits of the 64-bit address calculation result are used.

While there are some small pros and cons with these two approaches, I'll hold off from coloring people's feedback. (and thanks in advance for that feedback).

Greg

Anup Patel
 

Hi Greg,

 

I am inclined towards Option#1 provided the new PTE format is compatible with existing PTE format for Sv32/Sv39/Sv48.

 

Is the Option#1 documented somewhere ?

 

Regards,

Anup

 

From: tech-unixplatformspec@... <tech-unixplatformspec@...> On Behalf Of Greg Favor
Sent: 08 September 2020 03:58
To: tech-unixplatformspec@...; software@...
Cc: tech-virt-mem@...; Greg Favor <gfavor@...>
Subject: [RISC-V] [tech-unixplatformspec] Two alternatives for architecture extension to specify address/page-based memory types

 

To all the system software people on the tech-unixplatformspec and software email lists:

 

The tech-virt-mem Task Group is considering two alternative approaches for specifying address/page-based attributes - which are analogous to "memory types" in x86 ( WB/WC/UC) and ARMv8 (Normal cacheable, Normal noncacheable, Device).  This email attempts to get further feedback from system software / device driver people as to the practical useability, pros and cons, and preference for the two approaches (at a gross first-order level, so I won't belabor the details of the two approaches).  Also note that both approaches would override the default attributes that come from RISC-V PMAs - which are very roughly analogous to x86 MTRRs.

 

Option #1 - VA/Page-Based Attributes

This approach utilizes 2-3 bits in page table entries to specify a memory type for the page (roughly corresponding to the above x86 and ARMv8 memory types).  This approach is similar to the approach supported by x86 and ARMv8, and presumably would directly mesh with existing Linux support for this type of approach.

 

Option #2 - Physical Address-Based Attributes

This approach uses the top 2-3 bits of a 64-bit PA (physical address) to specify a memory type for that address value.  This approach applies equally whether MMU address translation is On or Off (where Off corresponds to RISC-V M-mode and Bare translation modes).  When translation is On, these bits likely (like above) would occupy extra bits in page table entries.  When translation is Off, the high bits of the 64-bit address calculation result are used.

 

While there are some small pros and cons with these two approaches, I'll hold off from coloring people's feedback. (and thanks in advance for that feedback).

 

Greg

 

Philipp Tomsich
 

Greg,

I would also be inclined towards option 1, based purely on orthogonality: without some extra considerations, the same page could be inadvertently (as in "all too easily by mistake") be used with different memory types when option 2 is chosen.

To illustrate this point, let's view both options as having the memory type separate from the PA
  • option 1 will have a full 64bit PA with the PTE in the page-table
  • option 2 will have a 62bit PA concatenated with 2 bits of memory type
With option 2, it is easy to have code use different attributes for accesses into the same page (or even the same memory location).  When using a page-table, software would need to ensure that it does not accidentally translate onto the same 62bit PA with different memory types (i.e. the masked 62bit value would need to be considered to test if we are mapping to the same page).
With option 1, the test will be performed on the full 64bit address... reducing the chance of programmer error.

Note that both options equally enable a user to intentionally set up translations to the same PA using different cacheability attributes.
My concern here is only about the potential for inadvertent and subtle errors.

The only drawback of option 1 seems to be that the page-tables and translation need to be set up early in the system startup (as happens on most ARMv8 systems as well).
However, installing linear 1:1 translations with appropriate attributes does not seem like a very high bar (some others may disagree strongly, based on their use cases and software stacks — I would like to hear these concerns to better understand the various software stacks targeted).

Philipp.

On Tue, Sep 8, 2020 at 1:28 AM Greg Favor <gfavor@...> wrote:
To all the system software people on the tech-unixplatformspec and software email lists:

The tech-virt-mem Task Group is considering two alternative approaches for specifying address/page-based attributes - which are analogous to "memory types" in x86 ( WB/WC/UC) and ARMv8 (Normal cacheable, Normal noncacheable, Device).  This email attempts to get further feedback from system software / device driver people as to the practical useability, pros and cons, and preference for the two approaches (at a gross first-order level, so I won't belabor the details of the two approaches).  Also note that both approaches would override the default attributes that come from RISC-V PMAs - which are very roughly analogous to x86 MTRRs.

Option #1 - VA/Page-Based Attributes
This approach utilizes 2-3 bits in page table entries to specify a memory type for the page (roughly corresponding to the above x86 and ARMv8 memory types).  This approach is similar to the approach supported by x86 and ARMv8, and presumably would directly mesh with existing Linux support for this type of approach.

Option #2 - Physical Address-Based Attributes
This approach uses the top 2-3 bits of a 64-bit PA (physical address) to specify a memory type for that address value.  This approach applies equally whether MMU address translation is On or Off (where Off corresponds to RISC-V M-mode and Bare translation modes).  When translation is On, these bits likely (like above) would occupy extra bits in page table entries.  When translation is Off, the high bits of the 64-bit address calculation result are used.

While there are some small pros and cons with these two approaches, I'll hold off from coloring people's feedback. (and thanks in advance for that feedback).

Greg

Siqi Zhao
 

Hi Greg,

Option 1 looks better considering two-stage address translation. If top 2-3 bits in the PA are used for other purposes, the GPA also needs to behave in the same way, which means reduced physical memory size for guest VMs. This might be acceptable if we are running 64-bit guest VMs, but for 32-bit guests, the GPA space is greatly reduced (was 4GB, now only 1GB or 512MB).

Regards,
Siqi

1 - 4 of 4