Re: Proposal v2: Intro to the Spec and Profiles
Hi Allen,
The “riscv64gc” is the default ISA for which Linux, hypervisors, and distros are compiled. Most of the optional features which you mentioned are detected at boot-time/run-time wherever possible. If an optional feature is not detectable at boot-time/run-time then Linux or hypervisors will rely on Device Tree or ACPI to detect such features.
We can explicitly mandate an optional feature in a profile only if target use-case of the profile expects that optional feature to be always present otherwise we don’t need to mention the optional features. In my opinion, we should keep profiles generic as much as possible with minimum explicit dependency on optional features.
Regards,
Anup
From: tech-unixplatformspec@... <tech-unixplatformspec@...> On Behalf Of Allen Baum
Sent: 27 October 2020 21:24
To: Jonathan Behrens <behrensj@...>
Cc: Mark Himelstein <markhimelstein@...>; Sean Anderson <seanga2@...>; Al Stone <ahs3@...>; tech-unixplatformspec@...
Subject: Re: [RISC-V] [tech-unixplatformspec] Proposal v2: Intro to the Spec and Profiles
To say "riscv64gc is standard; pester the OS if you want to know what other extensions are supported" implies that shrink-wrap software
To standardize the hardware/firmware-to-OS interfaces, is it enough to say "riscv64gc is standard "?
This is where I had questions - repeating my email above, which hasn't been answered to my knowledge:
Is saying "riscv64gc" too coarse granularity? Within that umbrella are a whole host of optional features, e.g.
Are range based PMP entries required/optional/unused?
If required how many PMP entries are required/optional/? Which PMP modes are required/optional/unused (napot/range)?
What is the minimum granularity required (e.g. 4K or larger).
Is misaligned Load/Store support required/optional/unused?
Which exception types load xTVAL?
How many ASID bits must be supported?
(this is far, far from an exhaustive list)
None of the above are captured or proposed to be captured in any extension string.
Again, repeating myself, a possible answer is: All of the above are optional; SW will configure itself to handle any optional support (or none). Someone needs to go through all of the optional features and confirm that none of the optional features need to be present -
- but if any of the optional features need to be present (or need some minimum support, e.g. ASID bits or PMP granularity or #entries)
then I have to conclude that simply declaring that riscv64gc is required is inadequate.
I am a hardware guy, so I could be totally off-base here - am I?
On Mon, Oct 26, 2020 at 6:38 PM Jonathan Behrens <behrensj@...> wrote:
On Mon, Oct 26, 2020 at 7:50 PM mark via lists.riscv.org <markhimelstein=riscv.org@...> wrote:
let's recall that there are two end results we'd like to see:
- being able to run the same u mode application binary from implementation to implementation and get the same results.
- being able to run the same OS or hypervisor regarding the ISA (i.e. excluding non isa platform features like busses).
you can have the former without the latter.
In practice I'm not sure how much of a distinction there really is. Any given application binary is pretty much only going to work on one OS, so either the OS works across all implementations and you can run the application anywhere, or the OS only works on specific implementations and those are the only ones where you can run the application.
i wonder if we need to partition the problem into:
- application profiles
- isa non-u mode profiles
- everything else (maybe not a profile at all)
each would need to include all the pieces to achieve the end results.
does this make sense? am i missing something?
My understanding is that the initial goal of this group was to standardize the hardware-to-OS and firmware-to-OS interfaces. It has since expanded to cover the interface between pretty much every pair of software components running in M-mode and/or S-mode (and all of their interfaces with hardware). What you are describing as "application profiles" I think covers the remaining two interfaces: OS-to-application and hardware-to-application. However, I'd argue we should punt on both of those for the time being. The first because it is highly OS dependent and unclear that there is any possibility/likelihood of standardization. And the latter because we can just say "riscv64gc is standard; pester the OS if you want to know what other extensions are supported"
Jonathan
On Sun, Oct 25, 2020 at 9:00 PM Sean Anderson <seanga2@...> wrote:
On 10/25/20 6:23 PM, Al Stone wrote:
> // SPDX-License-Identifier: CC-BY-4.0
> //
> // introduction.adoc: provide an introduction to the document
> //
> // Provide a basic introduction to the overall document, defining some
> // key terms and usages.
> //
> // Changes for v2:
> // -- Tried to clarify the definitions of the interfaces, breaking
> // apart some that had gotten munged together
> // -- Clean up odds and ends of typos
> // -- Tacked on an implied outline for some of the remainder of
> // spec should we use the structure introduced here
> //
>
> # Introduction
>
> This document is the RISC-V Profiles and Platform Specification (RVP2S).
>
> This specification defines the hardware and firmware required to be able
> to install and use one or more operating environments (OEs) on a platform
> built around the RISC-V Instruction Set Architecture (ISA). The intent
> is to capture the constraints necessary for a hardware platform so that
> an operating environment can be assured it is compatible with that platform,
> if the platform is in compliance with this specification.
>
> For the purposes of this specification an _operating environment_ can
> range from a single application running on a bare metal system to a full
> scale server operating system (OS) running a Linux distribution on a
> commercial off-the shelf (COTS) system.
>
> Constraints on the hardware and firmware comprising part of a platform
> are key to this specification. For example, an OS needs to be able to
> make some basic assumptions about the platform in order to boot; if there
> are too many variations possible, it becomes unmanageable for the OS vendor
> and could preclude using the platform at all. We would like to ensure this
> does not happen.
>
> This specification also sets out any necessary constraints on the RISCV-V
> Instruction Set Architecture (ISA) -- at all privilege levels -- that are
> required to provide a consistent and predictable environment for running
> operating systems.
>
> The intent is to provide only the constraints necessary, and not to restrict
> creativity.
>
> ## Platforms
> The term _platform_ has already been used several times. The definition
> used here is that a platform is the sum total of the hardware, firmware,
Could this sentence be reduced to just "A platform is the sum total..."?
> and software comprising a usable computer system. Examples include
> smartphones, laptops, data center servers, single-board computers, tablets,
> and almost anything in between. While this is not terribly precise, this
> is intentional; this specification is not meant to constrain what a
> platform is, but to clarify what software can expect of it.
I'd argue that a specification is only useful insofar as the
constraints it imposes are useful. However, if one does not wish to make
too concrete of a definition, I suggest erring on the side of concision.
> ## Profiles
> The term _profile_ is frequently used in the RISC-V world. It is time
> to provide a reasonably precise definition of the term and how it
> is to be used. We do this since the specification itself sets down
> requirements on the basis of what a profile needs.
Is this introduction needed?
>
> Consider a platform as being built up by connecting together a number of
> different interfaces -- the interfaces between hardware and firmware, or
> firmware and software, or software and the end-user.
> What we would like to do is define just the interfaces between the objects
> that constitute a platform. This allows the implementations to be whatever
> they need to be, as long as they abide by the interface definition. For
> example, we may need a UEFI Runtime Services interface. Whether that is
> implemented via EDK2 source, or is an interface to linuxboot is irrelevant.
> The functionality defined by the interface is the important part.
Perhaps this paragraph should be moved up to the introduction?
Also, you never define what exactly what a "profile" is. In addition, a
definition of an "interface" beyond examples is also missing (this is
touched on a bit more below). I suggest something along the lines of
A platform is a specific implementation of a profile.
A profile is a set of interfaces.
An interface is the protocol and data structure used to
communicate between different components of a platform.
Perhaps "component" should also be defined.
>
> Of particular interest in this specification are:
>
> * The User Interface: how the end-user of a system interacts with it.
> Examples might be various Linux-, BSD- or Windows based systems; they
> might also include a specific application developed for a specific use.
So would a possible portion of the User Interface be e.g. the ELF psABI?
Or just things like the POSIX API (or POSIX-mandated programs).
> * The Firmware Interface: how the User Interface makes use of S-mode. An
> example is UEFI Runtime Services -- a list of the services needed in order
> to allow the User Interface to work properly.
I am confused by your usage of the word "interface" here (and in the
other examples). Usually if I have two components A and B, and they
interact like
+---+ foo() +---+
| | ---------> | |
| A | return 5 | B |
| | <---------- | |
+---+ +---+
Then I would refer to foo() (and the calling-convention, API, etc) as
the "interface". However, you seem to be using "interface" to refer to
both foo() and B itself. To clarify, could the sentence
> The Firmware Interface [is] how the User Interface makes use of S-mode.
be reworded to
> The Firmware Interface [is] how the component which implements the
> User Interface makes use of S-mode.
and keep your intended meaning?
---
> * The Firmware Interface: how the User Interface makes use of S-mode. An
> example is UEFI Runtime Services -- a list of the services needed in order
> to allow the User Interface to work properly.
Can you clarify what you mean by "makes use of S-mode"? I don't see what
UEFI has to do with that.
> * The Supervisor Interface: how the Firmware Interface interacts with RISC-V
> hardware threads (_harts_), providing functionality to the Firmware
> Interface to start and stop them, for example. There will likely be
> cases, too, where the User Interface interacts directly with this
> functionality, such as when it provides mechanims needed that are not
> defined by the Firmware Interface.
Ok, so would the SBI be considered a Supervisor interface? I don't
understand the difference between this interface and the firmware
interface. Can you clarify a bit more?
> * The Hardware Interface: what the previously mentioned interfaces can expect
> to be present; in a way this is the interface to the outside world, but most
> likely it would be a list of the hardware components that are guaranteed
> to be present, and what standards they comply with.
So this is things like which devices are present on the system, except
for the harts themselves? Are devices like the CLIC/CLINT/PLIC included?
Can you clarify what you mean by "the outside world"? The outside from
whose perspective?
> * The Processor Interface: this interface defines what to expect of the
> processor. In the case of RISC-V this would include the parts of the ISA
> that and are not implemented.
I think you are missing a word.
>
> * The Boot Interface: this is what a user would see at a cold start of the
> system -- for the most part these days, this includes U-Boot bootm/booti
> or the UEFI Shell, though it could just as easily be linuxboot or coreboot.
> This defines the functionality needed to boot the system, update the
> firmware and possibly manage the system remotely; this will have to
> interact closely with OS booting protocols.
Ok, so if the firmware looks for a file named "boot.bin" with a specific
format, then that would be considered part of the boot interface? Or is
this specifically describing the user-interface of the bootloader? How
do multiple bootloader staged fit into this? For example, on some sifive
SoCs, there are 3 bootloaders (ZSBL, FSBL, BBL) before Linux. Are all
three part of the boot interface? Since the ZSBL lives in ROM is it part
of the hardware profile?
---
Would it be possible to provide a diagram showcasing how these
interfaces interact? In particular, it seems like it might be possible
for the same component to provide multiple interfaces (e.g. U-Boot could
provide both a boot interface and a firmware interface).
> For the above interfaces, a platform must provide an instance of each one;
> a platform would not be usable without all of them. There are some interfaces
> that could be optional:
Does it have to? I can certainly imagine a platform without a firmware
interface. And virtualized environments likely have no boot interface
either.
> * The Enumeration Interface: how the User Interface determine what hardware
> is present on the platform; this will need to include mechanisms for finding
> self-identifying hardware such as PCIe, and for hardware that does not
> identify itself such as UARTs, the motherboard, or CPU topology. This
> is typically ACPI in general purpose computing platforms, and Device Tree
> in embedded systems. Note that this may include further information such
> as SMBIOS data. At the same time, though, a very specific-use platform
> may already encapsulate all of this knowledge in the software being run
> so neither ACPI nor DT are needed.
Is the enumeration interface intended to describe just the hardware and
processor interfaces, or is it intended to describe all the other
interfaces as well? What interface is used to discover which enumeration
interface is in use? (where does the buck stop?)
>
> * The Secondary Boot Loader: not all OEs require this. For the ones that
> do, it must behave correctly. GRUB2 is expected by all recent Linux
> distributions, for example.
Is this an interface? This seems more like some kind of component. And
Linux does not necessarily expect GRUB2: EFISTUB, coreboot, U-Boot,
syetemd-boot, and others are all perfectly capable of booting Linux.
Further, it should be clarified that a "secondary boot loader" is not
necessarily the *second* bootloader, but rather the last bootloader.
>
> For each of the interfaces listed above, there is a domain of possibilities.
> Let us look at each domain of interfaces as an element in an n-tuple defined
> as follows:
>
> ```
> N = { user, firmware, supervisor, hardware, processor, boot,
> enumeration, secondary-boot-loader }
> ```
>
> Not all of the permutations of this set make sense. Not all of them are
> necessarily useful. However, it does allow us to provide a more precise
> definition of a profile -- a _profile_ is a value in **_N_**, a profile-tuple,
> if you will.
>
> ### An Example Profile
> The test of any formalism is whether or not it is usable. So, let’s define
> a general purpose Linux platform that could be used to run one of the various
> Linux distributions such as RHEL, Fedora, SuSE, Debian, ArchLinux or maybe
> even Gentoo and Slackware. The profile might be:
>
> ```
> LinuxServer = { Linux, UEFI, OpenSBI, rack-mount, RV64GC, UEFI Shell,
What does "rack-mount" mean? How does a hardware form-factor directly
affect the devices on a system?
> ACPI, GRUB2}
> ```
>
> This is just an example of what is possible; it is not definitive. A proper
> profile for Linux server distributions will be provided later in this
> specification. Regardless, this could tell us:
>
> * The User Interface is a Linux system of some flavor.
> * The Firmware Interface is UEFI, built upon SBI.
> * The Supervisor Interface is OpenSBI.
> * The Hardware Interface is for rack-mount server systems -- defining
> this is where we would have to list what devices that implies.
As noted above, this is insufficiently precise. If I replace my Foomatic
rack-mount server with a Barbaz rack-mount server, then the internat
architecture could be *completely different*, with totally different
devices and buses. I would expect this to specify a particular SoC
family.
> * The Processor Interface is RV64GC (U-mode, and S-mode)
> * The Boot Interface is the UEFI Shell.
> * The Enumeration Interface is ACPI, so we now know how to identify
> all of the hardware being used.
> * The Secondary Boot Loader is GRUB2, so Linux now has some idea of
> how it will be booted.
>
> And, as long as the interface definitions are fairly clear, we can now
> determine what would be in a platform that supports the ```LinuxServerDistro```
> profile. It is the purview of this platform specification to (a) define
> the profiles of interest, and then (b) ensure that each of the terms
> in the n-tuple -- the interfaces -- are clearly defined.
>
> ### Other Distros
"Distro" should probably be introduced before being shortened. E.g.
A Linux distribution (distro) such as ...
> While the definition of a ```LinuxServerDistro``` profile used above would
Above you introduced the profile ``LinuxServer`` not
``LinuxServerDistro``.
> seem to preclude non-Linux operating systems, defining a profile as an
> n-tuple gives us a way of handling this. For example, let’s suppose we
> define profiles for Windows and most BSD-based systems:
>
> ```
> WindowsDistro = { Windows, UEFI, OpenSBI, rack-mount, RV64GC, UEFI Shell,
> ACPI, WindowsLoader}
>
> BSDServer = { NetBSD, UEFI, OpenSBI, rack-mount, RV64GC, UEFI Shell,
> ACPI, GPT}
> ```
>
> Again, these are only examples, and not definitive. Given what we know
> about these sorts of systems, we could assert the following:
>
> ```
> LinuxServer = WindowsDistro = BSDServer
> ```
This is an abuse of the equal sign. May I suggest ≈ instead?
>
> That is, all of these are essentially identical in the things that matter
> to this specification. We know that’s not exactly true, of course, but this
> does give us the tool to start reasoning about such things.
What's the point in introducing methematical notation such as ``=`` if
you aren't going to give a more precise definition?
>
> ### Notation
> Whilst set notation is all well and good,
No set notation was used.
> we need something a bit more
> practical. For that, we can borrow from the example provided by GCC.
> A profile tuple would then be:
>
> ```
> N = user-firmware-supervisor-hardware-processor-boot-enumeration-secondary
> ```
Please don't do this. 3- and 4-tuples are already hard to remember (is
it arch-abi-libc-os or arch-libc-abi-os, and which one can I
omit?). Plus, most components only care about a few layers below and
above them. For example, Linux cares about the
firmware-supervisor-hardware-processor (and perhaps enumeration).
OpenSBI cares about supervisor-hardware-processor. U-Boot cares about
hardware-processor-boot-enumeration (and maybe firmware and secondary). This
format is to remember the ordering for (since interfaces
do not correspond to single components), and does not afford grouping
similar components together.
Further, some "tuples" have multiple possible values for different
interfaces. How would one express that a particular Linux kernel can
boot from either devicetree or ACPI? Or that U-Boot could boot Linux or
VxWare? Or that coreboot could load U-Boot, Grub2, or Linux after
booting?
Both "enumeration" and "secondary" are optional. How would one
differentiate between enumeration wihout secondary and vice versa?
>
> Or, for the example profiles listed above:
>
> ```
> linux-server = linux-uefi-opensbi-rackmount-rv64gc-uefi-acpi-grub2
How is one to know why uefi is listed twice?
> windows-distro = windows-uefi-opensbi-rackmount-rv64gc-uefi-acpi-windows
And why is windows twice?
> bsd-server = bsd-uefi-opensbi-rackmount-rv64gc-uefi-acpi-gpt
There's more than once type of bsd ;)
> ```
>
> For the long term, these could be converted into values and captured in
> firmware to be relayed to the OS, perhaps as part of the Chosen node in
> the DT used to pass information to the Linux kernel. In this manner, the
> platform would describe itself, instead of the OS having to make an
> educated guess.
So are the different boot stages each supposed to perform devicetree
fixups adding to the tuple?
>
> ### Revisions
> Over time, the definitions of the elements in this profile-tuple will
> change. It’s simply the nature of the problem -- new hardware will be
> defined, and ultimately will be assumed to be present, or old hardware
> will get deprecated (e.g., CXL, and perhaps some day UARTs, respectively).
> Hence, we will need to be able to note the revision of an element definition
> for a profile definition to be complete. Again, we could have the firmware
> convey this information to the OS as an additional byte string (how is TBD).
> For practical use, we may wish to append the revisions to the tuple notation:
>
> ```
> N = user-firmware-supervisor-hardware-processor-boot-enumeration-secondary \
> :ur-fr-sr-hr-pr-br-er-sr
> ```
>
> Or, for the example profiles listed above:
>
> ```
> linux-server = linux-uefi-opensbi-rackmount-rv64gc-uefi-acpi-grub2: \
> 0-0-0-0-0-0-0-0
> windows-distro = windows-uefi-opensbi-rackmount-rv64gc-uefi-acpi-windows: \
> 0-3-5-5-0-1-0-2
> bsd-server = bsd-uefi-opensbi-rackmount-rv64gc-uefi-acpi-gpt: \
> 1-0-3-4-0-1-0-2
> ```
>
> ### Some Notes
> * Profiles are _only_ a property of platforms, _not_ harts.
> * This section uses example profiles; these are not intended to be definitive.
> Proper profile definitions will appear in following sections of the
> specification.
> * This section presents the current definition of a platform-tuple. This
> may change as we know more.
>
>
> # Profiles
> ## Linux Profiles
> ### Linux Development SBC Profile
> ### Linux Laptop Profile
> ### Linux Desktop Profile
> ### Linux Server Profile
> ```
> linux-server = linux-uefi-opensbi-rackmount-rv64gc-uefi-acpi-grub2
> ```
>
> ## BSD Profiles
>
> ## Windows Profiles
>
> ## Bare Metal SBC Profiles
>
> # Interface Specifications
>
> ## User Interfaces
> ### Linux
> ### Windows
> ### BSD
>
> ## Firmware Interfaces
> ### UEFI
> ### U-Boot
U-Boot does not provide a firmware interface unless you call it only
from one core and only from the mode it called you from (e.g. if it
starts in M-mode you must call it from M-mode).
>
> ## Supervisor Interfaces
> ### SBI
...and BBL, OpenSBI, Xvisor, KVM, RustSBI, etc.
And any non-SBI interfaces which eventually show up.
>
> ## Hardware Interfaces
> ### Console
> ### I/O
Is that slash to be inserted into the tuple? And what in the world does
"Console" or "I/O" mean?
>
> ## Processor Interfaces
>
> ## Boot Interfaces
> ### UEFI Shell
> ### U-Boot bootm/booti
bootm uses a different interface from booti (booti will boot raw Linux
binaries, while bootm uses legacy or fit images, which are much more
versatile). Coreboot should also be here.
>
> ## Enumeration Interfaces
> ### ACPI
> ### Device Tree
> ### SMBIOS
>
> ## Secondary Boot Loaders
> ### GRUB2
>
Overall there are some interesting ideas here, but there is too much
ambiguity to meaningfully separate different interfaces from each other.
The tuple is an awful format and I sincerely suggest that you
reconsider some other representation.
--Sean
--
Mark I Himelstein
CTO RISC-V International
+1-408-250-6611
twitter @mark_riscv