On Fri, 2021-07-09 at 23:04 +0530, Mayuresh Chitale wrote:
This patch adds requirements for PCIe support for the server
Signed-off-by: Mayuresh Chitale <mchitale@...>
Makefile | 18 +++--
pcie-topology.ditaa | 28 ++++++++
riscv-platform-spec.adoc | 149
3 files changed, 187 insertions(+), 8 deletions(-)
create mode 100644 pcie-topology.ditaa
diff --git a/Makefile b/Makefile
index de5e0b0..06796f3 100644
@@ -3,13 +3,19 @@
ASCIIDOCTOR = asciidoctor
+DITAA = ditaa
+IMAGES = pcie-topology.png
PLATFORM_SPEC = riscv-platform-spec
PANDOC = pandoc
PARTS = changelog.adoc contributors.adoc introduction.adoc
profiles.adoc supervisor-level.adoc user-level.adoc
# Build the platform spec in several formats
-all: $(PLATFORM_SPEC).md $(PLATFORM_SPEC).pdf $(PLATFORM_SPEC).html
+all: $(IMAGES) $(PLATFORM_SPEC).md $(PLATFORM_SPEC).pdf
+ rm -f $@
+ $(DITAA) $<
$(PANDOC) -f docbook -t markdown_strict $< -o $@
@@ -17,10 +23,10 @@ $(PLATFORM_SPEC).md: $(PLATFORM_SPEC).xml
$(ASCIIDOCTOR) -d book -b docbook $<
+$(PLATFORM_SPEC).pdf: $(PLATFORM_SPEC).adoc $(IMAGES)
$(ASCIIDOCTOR) -d book -r asciidoctor-pdf -b pdf $<
+$(PLATFORM_SPEC).html: $(PLATFORM_SPEC).adoc $(IMAGES)
$(ASCIIDOCTOR) -d book -b html $<
@@ -31,11 +37,11 @@ clean:
rm -f $(PLATFORM_SPEC).md
rm -f $(PLATFORM_SPEC).pdf
rm -f $(PLATFORM_SPEC).html
+ rm -f $(IMAGES)
# handy shortcuts for installing necessary packages: YMMV
- sudo apt-get install pandoc asciidoctor ruby-asciidoctor-pdf
+ sudo apt-get install pandoc asciidoctor ditaa ruby-
- sudo dnf install pandoc rubygem-asciidoctor rubygem-
+ sudo dnf install ditaa pandoc rubygem-asciidoctor rubygem-
diff --git a/pcie-topology.ditaa b/pcie-topology.ditaa
new file mode 100644
@@ -0,0 +1,28 @@
+ +----------+ +----------+
+ | CPU | | CPU |
+ +-----+----+ +-----+----+
+ | |
+ | |
+ | |
+ +-------------+------------+ +-------------+--------
+ | |
+ | Root Complex | | Root
+ | |
+ | +--------------+ | | +--------------
+ | | Host Bridge | | | | Host Bridge
+ | +------+-------+ | | +------+-------
+ | | | |
+ | | | | | Bus
+ | +-------+------+ | | +-----+-------
+ | | Bus 0 | | | | Root Port
+ | | | | | +-----+-------
+ | +---+---+ +---+---+ | |
+ | | RCiEP | | RCEC | | | | PCIe
+ | +-------+ +-------+ | |
+ | | | Bus 1
+ | | |
+ +--------------------------+ +----------------------
+ RCiEP : Root complex integrated endpoint
+ RCEC : Root complex event collector
diff --git a/riscv-platform-spec.adoc b/riscv-platform-spec.adoc
index 4418788..24be2d2 100644
@@ -47,7 +47,20 @@ include::profiles.adoc
|RVA22 | RISC-V Application 2022
|EE | Execution Environment
|RV32GC | RISC-V 32-bit general purpose ISA described as
-|RV64GC | RISC-V 64-bit general purpose ISA described as
+|RV64GC | RISC-V 64-bit general purpose ISA described as
+|PCIe | PCI Express
+|ECAM | Enhanced Configuration Access Mechanism
+|BAR | Base Address Register
+|AER | Advanced Error Reporting
+|CRS | Configuration Request Retry Status
+|TLP | Transaction Layer Packet
+|RCiEP | Root Complex Integrated Endpoint
+|RCEC | Root Complex Event Collector
+|PME | Power Management Event
+|MSI | Message Signaled Interrupts
+|MSI-X | Enhanced Message Signaled Interrupts
+|INTx | PCIe Legacy Interrupts
+|PMA | Physical Memory Attributes
@@ -363,7 +376,139 @@
** Platforms are required to delegate the supervisor timer interrupt
mode. If the 'H' extension is implemented then the platforms are
delegate the virtual supervisor timer interrupt to 'VS' mode.
+Platforms are required to support at least PCIe Base Specification
+====== PCIe Config Space
+* Platforms shall support access to the PCIe config space via ECAM
+in the PCIe Base specification.
+* The entire config space for a single PCIe domain should be
accessible via a
+single ECAM I/O region.
+* Platform firmware should implement the MCFG table as listed in the
+Description Tables above to allow the operating systems to discover
+PCIe domains and map the ECAM I/O region for each domain.
+* Platform software shall configure ECAM I/O regions such that the
+memory attributes are that of a PMA I/O region (i.e. strongly-
+====== PCIe Memory Space
+Platforms are required to map PCIe address space directly in the
+space and not have any address translation for outbound accesses
from harts or
+for inbound accesses to any component in the system address space
+* PCIe Outbound Memory +
+PCIe devices and bridges/switches frequently implement BARs which
+32-bit addressing or support 64 bit addressing but do not support
+memory. To support mapping of such BARs, platforms are required to
+some space below 4G for each root port present in the system.
+[underline]*_Implementation Note_* +
+Platform software would likely configure these per root port regions
+their effective memory attributes are that of a PMA I/O region (i.e.
+strongly-ordered, non-cacheable, non-idempotent). Platforms would
+reserve some space above 4G to map BARs that support 64 bit
+prefetchable memory which could be configured by the platform
software as either
+I/O or memory.
+* PCIe Inbound Memory +
+For security reasons, platforms must provide a mechanism controlled
+software to restrict inbound PCIe accesses from accessing regions of
+space intended to be accessible only to M-mode software.
+[underline]*_Implementation Note_* +
+Such an access control mechanism could be analogous to the per-hart
+as described in the RISC-V Privileged Architectures specification.
+====== PCIe Interrupts
+* Platforms shall support both INTx and MSI/MSI-x interrupts.
+* Integration with AIA +
+====== PCIe cache coherency
+PCIe transactions that are not marked as No_snoop and access memory
Greg had few suggestion to change this statement.
"PCIe transactions that are marked with a No Snoop bit of zero and
+cacheable by harts, as well as accesses to memory that is
+harts, are I/O Coherent and no software coherency management is
+In contrast, PCIe transactions that are marked as No_snoop and
+that is cacheable by harts, must have coherency managed by software.
+====== PCIe Topology
+Platforms are required to implement at least one of the following
+and the components required in that topology.
+* Host Bridge +
+Following are the requirements for host bridges:
+** Any read or write access by a hart to an ECAM I/O region shall be
+by the host bridge into the corresponding PCIe config read or config
+** Any read or write access by a hart to a PCIe outbound region
+forwarded by the host bridge to a BAR or prefetch/non-prefetch
+if the address falls within the region claimed by the BAR or
+non-prefetch memory window. Otherwise the host bridge shall return
+** Host bridge shall return all 1s in the following cases:
+*** Config read to non existent functions and devices on root bus.
+*** Config reads that receive Unsupported Request response from
+devices on the root bus.
+* Root ports +
+Following are the requirements for root ports.
+** Root ports shall appear as PCI-PCI bridge to software.
+** Root ports shall implement all registers of Type 1 header.
+** Root ports shall implement all capabilities specified in the PCIe
+specification for a root port.
+** Root ports shall forward type 1 configuration access when the bus
+the TLP is greater than the root port's secondary bus number and
less than or
+equal to the root port's subordinate bus number.
+** Root ports shall convert type 1 configuration access to a type 0
+configuration access when bus number in the TLP is equal to the root
+secondary bus number.
+** Root ports shall respond to any type 0 configuration accesses it
+** Root ports shall forward memory accesses targeting its
+memory windows to downstream components. If address of the
transaction does not
+fall within the regions claimed by prefetch/non-prefetch memory
+the root port shall generate a Unsupported Request.
+** Root port requester id or completer id shall be formed using the
bdf of the
+** The root ports shall support the CRS software visibility.
+** The root port shall implement the AER capability.
+** Root ports shall return all 1s in the following cases:
+*** Config read to non existent functions and devices on secondary
+*** Config reads that receive Unsupported Request from downstream
+*** Config read when root port's link is down.
+* RCiEP +
+All the requirements for RCiEP in the PCIe Base specification shall
+In addition the following requirements shall be met:
+** If RCiEP is implemented then RCEC shall be implemented as well.
+requirements for RCEC specified in the PCIe Base specification shall
+implemented. RCEC is required to terminate the AER and PME messages
+** If both the topologies mentioned above are supported then RCiEP
+shall be implemented in a separate PCIe domain and shall be
addressable via a
+separate ECAM I/O region.
+===== PCIe Device Firmware Requirement
+PCI expansion ROM code type 3 (UEFI) image must be provided by PCIe
+OS/A server extension platform according to
ware Specification Revision 3.3]
+if that PCIe device is utilized during UEFI firmware boot process.
+stored in PCI expansion ROM is an UEFI driver that must be compliant
+https://uefi.org/specifications[UEFI specification 2.9] 14.4.2 PCI
+====== PCIe peer to peer transactions +
==== Secure Boot