Anup Patel
Hi Greg,
For per-HART edge sensitive interrupts, we can avoid the overflow status bit for each counter by keeping track of last read value and comparing this last read value in overflow interrupt handler.
I am suggesting one edge-sensitive interrupt for each HART routed through PLIC so that we don’t need too many PLIC interrupt lines.
Regards,
Anup
toggle quoted message
Show quoted text
From: Greg Favor <gfavor@...>
Sent: 21 July 2020 11:36
To: Anup Patel <Anup.Patel@...>
Cc: Brian Grayson <brian.grayson@...>; alankao <alankao@...>; tech-privileged@...; andrew@...
Subject: Re: [RISC-V] [tech-privileged] A proposal to enhance RISC-V HPM (Hardware Performance Monitor)
Regarding overflow interrupts as edge-sensitive interrupts:
It seems like this would require that there not be any overflow status bit in mhpmevent CSRs (or any alternative CSR), otherwise this bit would need to be cleared by software - which is equivalent to the "clearing a serviced overflow interrupt"
that is trying to be avoided below. Which seems generally undesirable; as well as how would the overflow interrupt handler figure out which counter(s) overflowed? (Or are you imagining 32 separate per-counter overflow interrupt requests to a PLIC?)
In contrast, both x86 and ARMv8 have explicit counter overflow status bits that are the basis for generating a shared level-sensitive interrupt request and these bits must be cleared by handler software.
Lastly, note that in my proposal the handler (say in S/HS mode) can directly clear the Overflow status bits for the counter overflows that it has serviced - via the hpmoverflow CSR. That avoids the SBI call that you rightly are wanting
to avoid.
Hi All,
The proposed SBI PMU extension is a set of APIs between M-mode and HS-mode (also between HS-mode and VS-mode) such that no RISC-V spec changes are required for existing HPMCOUNTERs.
The high-level view of SBI PMU extension is as follows:
-
The SBI PMU calls are only for discovering and configuring HARDWARE/SOFTWARE counters and events
-
The HARDWARE/SOFTWARE counters will be read directly from S-mode software without any SBI calls
-
There are two suggested approaches for overflow handling:
-
No overflow interrupt. The S-mode software will use periodic timer interrupts to track counter overflows. This will be imprecise software approach for detecting overflow
-
Per-HART edge-triggered interrupt routed through PLIC (or some other platform interrupt controller). The per-HART interrupt will have to be routed through PLIC because we don’t have interrupts defined in MIP/MIE CSRs for counter overflows. Making this interrupts
edge-triggered will not require S-mode software to clear the interrupt using SBI call
The SBI PMU extension does not prevent anyone in defining new RISC-V PMU extension. Although, whatever new RISC-V PMU extension is defined should consider H-extension and also provide
dedicated CSRs for S-mode.
In future, when a new RISC-V PMU extension is available in RISC-V privilege spec, the SBI PMU extension will continue to exist and will be mostly used for SOFTWARE counters provided
by SBI implementation (OpenSBI or Hypervisors) to the S-mode software.
Regards,
Anup
Hi, Alan.
My proposal is still a work in progress, hence has not been shared publicly, but is significantly based on a proven architecture with about 30 years in the field and a few billion
shipping cores, if not more -- the PowerPC performance monitor implementation. I did the in-house Linux kernel patches and tool support for it about two decades ago at Motorola :) so I used to know it quite well, and can see how a similar approach solves some
of the current problems that we all have encountered with the current RISC-V approach. I am fairly new to the RISC-V ecosystem, so I was not aware of the work that you have done in the past; thanks for the pointer to that.
The SBI PMU extensions is more about the API between what perf (or another tool) communicates, and how the M-mode software interprets it, and not about actually changing the hardware
interpretation of mhpmevent bits, at least that was my understanding.
I am glad that so many of us are converging on all the same fundamental needs!
On Mon, Jul 20, 2020 at 7:38 PM alankao <alankao@...> wrote:
Hi Brian,
I have been working on a similar proposal myself, with overflow, interrupts, masking, and delegation. One of the key differences in my proposal is that it unifies each counter's configuration
control into a per-counter register, by using mhpmevent* but with some fields reserved/assigned a meaning. <elaborating>
Thanks for sharing your experience and the elaboration. The overloading-hpmevent idea looks like the one in the SBI PMU extension threads in Unix Platform Spec TG by
Greg. I have a bunch of questions. How was your proposal later? Was it discussed in public? Did you manage to implement your idea into a working HW/S-mode SW/U-mode SW solution? If so, we can compete with each other by real benchmarking the LoC of the
perf patch (assuming you do it on Linux) and the system overhead running a long perf sample.
Another potential discussion point is, does overflow happen at 0x7fffffffffffffff -> 0x8000000000000000, or at 0xffffffffffffffff -> 0x0000000000000000? I have a bias towards the
former so that even after overflow, the count is wholly contained in an XLEN-wide register treated as an unsigned number and accessible via a single read, which makes arithmetic convenient, but I know some people prefer to (or are used to?) have the overflow
bit as a 33rd or 65th bit in a different register.
I have no bias here as long as the HPM interrupt can be triggered. But somehow it seems to me that you assume the HPM registers are XLEN-width but actually they are not (yet?). The spec
says they should be 64-bit width although obviously nobody implements nor remember that.
Lastly, a feature I have enjoyed using in the past (on another ISA) is the concept of a 'marked' bit in the mstatus register. ... This is of course a bit intrusive in the architecture,
as it requires adding a bit to mstatus, but the rest of the kernel just needs to save and restore this bit on context switches, without knowing its purpose.
Which architecture/OS are you referring to here?
Through this discussion, we will understand which idea is the community prefer to: adding CSRs, overloading existing hpmevents, or any balanced compromise. I believe the ultimate goal of this thread should be determining what the RISC-V HPM should really be
like.
Best,
Alan
|
