1. Tech-Unixplatformspec
  2. Topics

Non-coherent I/O

Greg Favor
 

On Mon, Jun 14, 2021 at 1:04 PM Greg Favor <gfavor@...> wrote:
I have already sent questions to Andrew to get the official view as to the intent of this aspect of the Priv spec and what is the proper way or perspective with which to be reading the ISA specs.  That then may result in the need for clarifying text to be added to the spec.  And once it is clear as to the scope and bounds of the ISA specs and what they require and allow, then it is left to profile and platform specs to specify tighter requirements.
Here's the results of my Q&A with Andrew: 

- The Priv (and Unpriv) ISA specs are just that.  They are CPU architecture specs and should be read with that limited scope in mind.  They may touch on system-level issues, but they are not trying to constrain the flexibility in how these issues are handled across a wide range of system designs.  (I'll personally add on that RVI now makes an official distinction between ISA (Unpriv and Priv) and Non-ISA (aka system-related) arch specs.  The former apply inside of a hart; the latter apply outside of a hart.)

- Per above, PMAs and the PMA coherency attribute are CPU-specific and only apply to memory accesses by harts.  (One can choose to apply these ideas to accesses by other master agents in a system, but that's not officially a Priv spec matter.)

- The PMA coherency attribute only applies to that hart's accesses.  It is up to software to configure the PMAs in all harts to be the same, or not, as desired.  What is done for non-hart accesses (i.e. by I/O devices) is not specified by the Priv spec.  Hence there are no implications on I/O coherency, one way or another, by the Priv spec.

Naturally many if not most system designs will extend these ideas in some manner across the system and to other masters.  And platform specs may choose to specify and mandate some or all of this.  But that's not the business of the ISA specs.

Greg
Josh Scheid
 

On Mon, Jun 14, 2021 at 1:04 PM Greg Favor <gfavor@...> wrote:
I have already sent questions to Andrew to get the official view as to the intent of this aspect of the Priv spec and what is the proper way or perspective with which to be reading the ISA specs.  That then may result in the need for clarifying text to be added to the spec.  And once it is clear as to the scope and bounds of the ISA specs and what they require and allow, then it is left to profile and platform specs to specify tighter requirements.


Re I/O-related coherence and ordering, Daniel Lustig will readily acknowledge (and I'm quoting him) that "the I/O ordering model isn't currently defined as precisely as RVWMO".

And Krste will certainly say (i.e. has said) that RISC-V supports systems with coherent and non-coherent masters, and needs to standardize arch support for software management in such platforms asap.


While potentially a fine goal, it seems that to make this happen in a manner that allows Platform-compliant SW to be portable, more needs to be done
beyond the Zicmobase work, at least in terms of "glue" specification to tie it all together. It's also possible that the goal of generally enabling non-coherent
masters in RISC-V is perhaps outside the scope of OS-A Platform work, in that in the short term things can be done to enable it in implementation-specific
HW+SW systems, but allowing for implementation portable SW (across platform-compliant implementations) will take longer.

-Josh
Greg Favor
 

I have already sent questions to Andrew to get the official view as to the intent of this aspect of the Priv spec and what is the proper way or perspective with which to be reading the ISA specs.  That then may result in the need for clarifying text to be added to the spec.  And once it is clear as to the scope and bounds of the ISA specs and what they require and allow, then it is left to profile and platform specs to specify tighter requirements.


Re I/O-related coherence and ordering, Daniel Lustig will readily acknowledge (and I'm quoting him) that "the I/O ordering model isn't currently defined as precisely as RVWMO".

And Krste will certainly say (i.e. has said) that RISC-V supports systems with coherent and non-coherent masters, and needs to standardize arch support for software management in such platforms asap.


Greg
mark
 

If this is an issue with the priv spec please add it to the priv spec github issues.

thanks
Mark

On Mon, Jun 14, 2021 at 10:44 AM Josh Scheid <jscheid@...> wrote:
Priv:
"""
Accesses by one hart to main memory regions are observable not only by other harts but also
by other devices with the capability to initiate requests in the main memory system (e.g., DMA
engines). Coherent main memory regions always have either the RVWMO or RVTSO memory
model. Incoherent main memory regions have an implementation-defined memory model.
"""

The above is the core normative piece discussion coherent initiators. 

It's confusing because the "observable" statement in the first sentence is indirectly overridden by the consideration of incoherent main memory.

It may be enough to add additional wording in the platform spec that for platforms that behave differently for NoSnoop=1 inbound TLPs (vs ignoring them and treating them as NoSnoop=0) the region of addresses accessed in that manner should be communicated as having "incoherent" PMA generally in the system.

But it also implies that there's no standard memory model for incoherent memory.  Is the use of RVWMO+Zicmobose sufficient, or is more needed to describe a portable memory model in this case?
-Josh
Josh Scheid
 

Priv:
"""
Accesses by one hart to main memory regions are observable not only by other harts but also
by other devices with the capability to initiate requests in the main memory system (e.g., DMA
engines). Coherent main memory regions always have either the RVWMO or RVTSO memory
model. Incoherent main memory regions have an implementation-defined memory model.
"""

The above is the core normative piece discussion coherent initiators. 

It's confusing because the "observable" statement in the first sentence is indirectly overridden by the consideration of incoherent main memory.

It may be enough to add additional wording in the platform spec that for platforms that behave differently for NoSnoop=1 inbound TLPs (vs ignoring them and treating them as NoSnoop=0) the region of addresses accessed in that manner should be communicated as having "incoherent" PMA generally in the system.

But it also implies that there's no standard memory model for incoherent memory.  Is the use of RVWMO+Zicmobose sufficient, or is more needed to describe a portable memory model in this case?
-Josh

1 - 5 of 5