Date
1 - 3 of 3
[PATCH v4] Add performance monitoring unit extension
|
atishp@...
From: Anup Patel <anup.patel@...>
This patch adds SBI performance monitoring unit (PMU) extension which allows S-mode (or VS-mode) software to configure hardware/software performance counters with help of M-mode (or HS-mode) software. Signed-off-by: Anup Patel <anup.patel@...> Signed-off-by: Atish Patra <atish.patra@...> --- Changes from v3->v4: The new "sscof" extension requires the event info to be 64 bit. This patch also improves 1. the counter start/stop function ids with a appropriate error codes 2. An additional flag parameter to accomodate "sscof extension". This patch also renames a. software counter to firmware counter to avoid ambiguity with kernel software events. b. event_info to event_data to avoid ambiguity with counter info --- riscv-sbi.adoc | 403 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 403 insertions(+) diff --git a/riscv-sbi.adoc b/riscv-sbi.adoc index 090f10abff94..8d0d3630e2fe 100644 --- a/riscv-sbi.adoc +++ b/riscv-sbi.adoc @@ -34,6 +34,7 @@ https://creativecommons.org/licenses/by/4.0/. * Improved SBI introduction secion * Improved documentation of SBI hart state managment extension * Added suspend function to SBI hart state managment extension +* Added SBI performance monitoring unit extension === Version 0.2 @@ -111,6 +112,8 @@ Standard SBI error codes are listed below | SBI_ERR_DENIED | -4 | SBI_ERR_INVALID_ADDRESS | -5 | SBI_ERR_ALREADY_AVAILABLE | -6 +| SBI_ERR_ALREADY_STARTED | -7 +| SBI_ERR_ALREADY_STOPPED | -8 |=== An `ECALL` with an unsupported SBI extension ID (*EID*) or an unsupported SBI @@ -1041,6 +1044,406 @@ upon failure. | sbi_system_reset | 0 | 0x53525354 |=== +== Performance Monitoring Unit Extension (EID #0x504D55 "PMU") + +The RISC-V M-mode hardware performance counters such as MCYCLE, MINSTRET, +and MHPMCOUNTERx CSRs are accessible as read-only from S-mode and U-mode +using CYCLE, INSTRET, and HPMCOUNTERx CSRs. These performance counters can +only be started, stopped, or configured using MCOUNTINHIBIT and MHPMEVENTx +CSRs in M-mode. That's why, SBI implementation may choose to disallow this +extension if MCOUNTINHIBIT is not implemented in the platform. + +A RISC-V implementation generally supports monitoring of various hardware +events using a limited number of hardware performance counters which are +up to 64 bits wide. In addition to the hardware performance counters, a SBI +implementation can provide firmware performance counters which can monitor +firmware events such as number of misaligned load/store instructions, number +of RFENCEs, number of IPIs, etc. The firmware counters are always 64 bits +wide. + +The SBI PMU extension provides: + +1. An interface for S-mode (or VS-mode) firmware to discover and configure + per-HART hardware/firmware counters +2. A typical https://en.wikipedia.org/wiki/Perf_(Linux)[perf] compatible + interface for hardware/firmware performance counters and events +3. Full access to microarchitecture's raw event encodings + +To define SBI PMU extension calls, we first define important entities +`counter_idx`, `event_idx`, and `event_data`. The `counter_idx` is a +logical number assigned to each hardware/firmware counter. The `event_idx` +represents a hardware/firmware event whereas the `event_data` represents +an additional configuration or parameters for the hardware/firmware event. + +The event_idx is a 20bits wide number encoded as follows: +[source, C] +---- + event_idx[19:16] = type + event_idx[15:0] = code +---- + +=== Event: Hardware general events (Type #0) + +The `event_idx.type` (i.e. *event type*) should be `0x0` for all hardware +general events and each hardware general event is identified by an unique +`event_idx.code` (i.e. *event code*) described in the <<table_pmu_hardware_events>> +below. + +[#table_pmu_hardware_events] +.PMU Hardware Events +[cols="3,1,2", width=100%, align="center", options="header"] +|=== +| General Event Name | Event Code | Description +| SBI_PMU_HW_NO_EVENT | 0 | Unused event because + event_idx cannot be zero +| SBI_PMU_HW_CPU_CYCLES | 1 | Event for each CPU cycle +| SBI_PMU_HW_INSTRUCTIONS | 2 | Event for each completed + instruction +| SBI_PMU_HW_CACHE_REFERENCES | 3 | Event for cache hit +| SBI_PMU_HW_CACHE_MISSES | 4 | Event for cache miss +| SBI_PMU_HW_BRANCH_INSTRUCTIONS | 5 | Event for a branch + instruction +| SBI_PMU_HW_BRANCH_MISSES | 6 | Event for a branch + misprediction +| SBI_PMU_HW_BUS_CYCLES | 7 | Event for each BUS cycle +| SBI_PMU_HW_STALLED_CYCLES_FRONTEND | 8 | Event for a stalled cycle + in microarchitecture + frontend +| SBI_PMU_HW_STALLED_CYCLES_BACKEND | 9 | Event for a stalled cycle + in microarchitecture + backend +| SBI_PMU_HW_REF_CPU_CYCLES | 10 | Event for each reference + CPU cycle +|=== + +*NOTE:* the `event_data` (i.e. *event data*) is unused for hardware general +events and all non-zero values of `event_data` are reserved for future use. + +=== Event: Hardware cache events (Type #1) + +The `event_idx.type` (i.e. *event type*) should be `0x1` for all hardware +cache events and each hardware cache event is identified by an unique +`event_idx.code` (i.e. *event code*) which is encoded as follows: + +[source, C] +---- + event_idx.code[15:3] = cache_id + event_idx.code[2:1] = op_id + event_idx.code[0:0] = result_id +---- + +Below tables show possible values of: `event_idx.code.cache_id` (i.e. +*cache event id*), `event_idx.code.op_id` (i.e. *cache operation id*) +and `event_idx.code.result_id` (i.e. *cache result id*). + +[#table_pmu_cache_event_id] +.PMU Cache Event ID +[cols="5,3,4", width=100%, align="center", options="header"] +|=== +| Cache Event Name | Cache Event ID | Description +| SBI_PMU_HW_CACHE_L1D | 0 | Level1 data cache event +| SBI_PMU_HW_CACHE_L1I | 1 | Level1 instruction cache event +| SBI_PMU_HW_CACHE_LL | 2 | Last level cache event +| SBI_PMU_HW_CACHE_DTLB | 3 | Data TLB event +| SBI_PMU_HW_CACHE_ITLB | 4 | Instruction TLB event +| SBI_PMU_HW_CACHE_BPU | 5 | Branch predictor unit event +| SBI_PMU_HW_CACHE_NODE | 6 | NUMA node cache event +|=== + +[#table_pmu_cache_ops_id] +.PMU Cache Operation ID +[cols="5,3,4", width=100%, align="center", options="header"] +|=== +| Cache Operation Name | Cache Operation ID | Description +| SBI_PMU_HW_CACHE_OP_READ | 0 | Read cache line +| SBI_PMU_HW_CACHE_OP_WRITE | 1 | Write cache line +| SBI_PMU_HW_CACHE_OP_PREFETCH | 2 | Prefetch cache line +|=== + +[#table_pmu_cache_result_id] +.PMU Cache Operation Result ID +[cols="5,3,4", width=100%, align="center", options="header"] +|=== +| Cache Result Name | Cache Result ID | Description +| SBI_PMU_HW_CACHE_RESULT_ACCESS | 0 | Cache access +| SBI_PMU_HW_CACHE_RESULT_MISS | 1 | Cache miss +|=== + +*NOTE:* the `event_data` (i.e. *event data*) is unused for hardware cache +events and all non-zero values of `event_data` are reserved for future use. + +=== Event: Hardware raw events (Type #2) + +The `event_idx.type` (i.e. *event type*) should be `0x2` for all hardware +raw events and `event_idx.code` (i.e. *event code*) should be zero. The +`event_data` parameter of `sbi_pmu_counter_config_matching()` call (described +below) should have the non-zero raw event value to be programmed in the +MHPMEVENTx CSR. + +*Note:* +Platform may choose to define the expected value to be written to MHPMEVENTx CSR +for a hardware event. In case of a hardware generic/cache event, platform may +use the `event_idx` as the expected value for simplicity. In case of a +hardware raw event, it can combine `event_data` with the `event_idx` value +to generate the expected value *<xyz>* for MHPMEVENTx where *<xyz>* is encoded +as follows: +.... +xyz[0:19] : event_idx (0x20000) +xyz[20:XLEN] : event_data[0:(XLEN-20)] +.... + +=== Event: Firmware events (Type #15) + +The `event_idx.type` (i.e. *event type*) should be `0xf` for all firmware +events and each firmware event is identified by an unqiue `event_idx.code` +(i.e. *event code*) described in the <<table_pmu_firmware_events>> below. + +[#table_pmu_firmware_events] +.PMU Firmware Events +[cols="3,1,2", width=100%, align="center", options="header"] +|=== +| Firmware Event Name | Event Code | Description +| SBI_PMU_FW_MISALIGNED_LOAD | 0 | Misaligned load trap + event +| SBI_PMU_FW_MISALIGNED_STORE | 1 | Misaligned store trap + event +| SBI_PMU_FW_ACCESS_LOAD | 2 | Load access trap event +| SBI_PMU_FW_ACCESS_STORE | 3 | Store access trap event +| SBI_PMU_FW_ILLEGAL_INSN | 4 | Illegal instruction + trap event +| SBI_PMU_FW_SET_TIMER | 5 | Set timer event +| SBI_PMU_FW_IPI_SENT | 6 | Sent IPI to other + HART event +| SBI_PMU_FW_IPI_RECEIVED | 7 | Received IPI from other + HART event +| SBI_PMU_FW_FENCE_I_SENT | 8 | Sent FENCE.I request to + other HART event +| SBI_PMU_FW_FENCE_I_RECEIVED | 9 | Received FENCE.I request + from other HART event +| SBI_PMU_FW_SFENCE_VMA_SENT | 10 | Sent SFENCE.VMA request + to other HART event +| SBI_PMU_FW_SFENCE_VMA_RECEIVED | 11 | Received SFENCE.VMA + request from other + HART event +| SBI_PMU_FW_SFENCE_VMA_ASID_SENT | 12 | Sent SFENCE.VMA with + ASID request to other + HART event +| SBI_PMU_FW_SFENCE_VMA_ASID_RECEIVED | 13 | Received SFENCE.VMA with + ASID request from other + HART event +| SBI_PMU_FW_HFENCE_GVMA_SENT | 14 | Sent HFENCE.GVMA request + to other HART event +| SBI_PMU_FW_HFENCE_GVMA_RECEIVED | 15 | Received HFENCE.GVMA + request from other + HART event +| SBI_PMU_FW_HFENCE_GVMA_VMID_SENT | 16 | Sent HFENCE.GVMA with + VMID request to other + HART event +| SBI_PMU_FW_HFENCE_GVMA_VMID_RECEIVED | 17 | Received HFENCE.GVMA + with VMID request from + other HART event +| SBI_PMU_FW_HFENCE_VVMA_SENT | 18 | Sent HFENCE.VVMA request + to other HART event +| SBI_PMU_FW_HFENCE_VVMA_RECEIVED | 19 | Received HFENCE.VVMA + request from other + HART event +| SBI_PMU_FW_HFENCE_VVMA_ASID_SENT | 20 | Sent HFENCE.VVMA with + ASID request to other + HART event +| SBI_PMU_FW_HFENCE_VVMA_ASID_RECEIVED | 21 | Received HFENCE.VVMA + with ASID request from + other HART event +|=== + +*NOTE:* the `event_data` (i.e. *event data*) is unused for firmware events +and all non-zero values of `event_data` are reserved for future use. + +=== Function: Get number of counters (FID #0) + +[source, C] +---- +struct sbiret sbi_pmu_num_counters() +---- + +*Returns* the number of counters (both hardware and firmware) in sbiret.value +and always returns `SBI_SUCCESS` in sbiret.error. + +=== Function: Get details of a counter (FID #1) + +[source, C] +---- +struct sbiret sbi_pmu_counter_get_info(unsigned long counter_idx) +---- + +Get details about the specified counter such as underlying CSR number, +width of the counter, type of counter hardware/firmware, etc. + +The `counter_info` returned by this SBI call is encoded as follows: +[source, C] +---- + counter_info[11:0] = CSR (12bit CSR number) + counter_info[17:12] = Width (One less than number of bits in CSR) + counter_info[XLEN-2:18] = Reserved for future use + counter_info[XLEN-1] = Type (0 = hardware and 1 = firmware) +---- + +If `counter_info.type == 0x1` then `counter_info.csr` and `counter_info.width` +should be ignored. + +*Returns* the `counter_info` described above in `sbiret.value`. + +The possible error codes returned in `sbiret.error` are shown in the +<<table_pmu_counter_get_info_errors>> below. + +[#table_pmu_counter_get_info_errors] +.PMU Counter Get Info Errors +[cols="1,2", width=100%, align="center", options="header"] +|=== +| Error code | Description +| SBI_SUCCESS | `counter_info` read successfully. +| SBI_ERR_INVALID_PARAM | `counter_idx` points to an invalid counter. +|=== + +=== Function: Find and configure a matching counter (FID #2) + +[source, C] +---- +struct sbiret sbi_pmu_counter_config_matching(unsigned long counter_idx_base, + unsigned long counter_idx_mask, + unsigned long event_idx, + uint64_t event_data, + unsigned long flag) +---- + +Find and configure a counter from a set of counters which is not started +(or enabled) and can monitor the specified event. The `counter_idx_base` +and `counter_idx_mask` parameters represent the set of counters whereas +the `event_idx` represent the event to be monitored and `event_data` represents +any additional configuration parameter required for that event. +The `flag` parameter is optional. The possible values of the flags are shown +in the <<table_pmu_counter_cfg_match_flags>> below. + +[#table_pmu_counter_cfg_match_flags] +.PMU Counter Config Match Flags +[cols="2,1,1", width=100%, align="center", options="header"] +|=== +| FLAG Name | Bit field | Description +| SBI_PMU_CFG_FLAG_AUTO_START | 0 | Start the counter after config matching +| SBI_PMU_CFG_FLAG_SKIP_MATCH | 1 | Skip the config matching +| SBI_PMU_CFG_FLAG_SET_MINH | 2 | Counting of events in M-mode is prohibited +| SBI_PMU_CFG_FLAG_SET_SINH | 3 | Counting of events in S-mode is prohibited +| SBI_PMU_CFG_FLAG_SET_UINH | 4 | Counting of events in U-mode is prohibited +| SBI_PMU_CFG_FLAG_SET_VSINH | 5 | Counting of events in VS-mode is prohibited +| SBI_PMU_CFG_FLAG_SET_VUINH | 6 | Counting of events in VU-mode is prohibited +|=== + +*Returns* the `counter_idx` in `sbiret.value` upon success. + +In case of failure, the possible error codes returned in `sbiret.error` are +shown in the <<table_pmu_counter_cfg_match_errors>> below. + +[#table_pmu_counter_cfg_match_errors] +.PMU Counter Config Match Errors +[cols="1,2", width=100%, align="center", options="header"] +|=== +| Error code | Description +| SBI_SUCCESS | counter found and configured successfully. +| SBI_ERR_INVALID_PARAM | set of counters has an invalid counter. +| SBI_ERR_NOT_SUPPORTED | none of the counters can monitor specified event. +|=== + +=== Function: Read a firmware counter (FID #3) + +[source, C] +---- +struct sbiret sbi_pmu_counter_fw_read(unsigned long counter_idx) +---- + +Provide the current value of a firmware counter in `sbiret.value`. + +The possible error codes returned in `sbiret.error` are shown in the +<<table_pmu_counter_fw_read_errors>> below. + +[#table_pmu_counter_fw_read_errors] +.PMU Counter Firmware Read Errors +[cols="1,2", width=100%, align="center", options="header"] +|=== +| Error code | Description +| SBI_SUCCESS | firmware counter read successfully. +| SBI_ERR_INVALID_PARAM | `counter_idx` points to a hardware counter or an + invalid counter. +|=== + +=== Function: Start a counter (FID #4) + +[source, C] +---- +struct sbiret sbi_pmu_counter_start(unsigned long counter_idx, + uint64_t initial_value) +---- + +Start or enable the specified counter on the calling HART with the specified +initial value. + +The possible error codes returned in `sbiret.error` are shown in the +<<table_pmu_counter_start_errors>> below. + +[#table_pmu_counter_start_errors] +.PMU Counter Start Errors +[cols="1,2", width=100%, align="center", options="header"] +|=== +| Error code | Description +| SBI_SUCCESS | counter started successfully. +| SBI_ERR_INVALID_PARAM | `counter_idx` points to an invalid counter. +| SBI_ERR_ALREADY_STARTED | counter already started. +|=== + +=== Function: Stop a counter (FID #5) + +[source, C] +---- +struct sbiret sbi_pmu_counter_stop(unsigned long counter_idx, unsigned long flag) +---- + +Stop or disable the specified counter on the calling HART. The possible values +of the flags are shown in the <<table_pmu_counter_stop_flags>> below. + +[#table_pmu_counter_stop_flags] +.PMU Counter Stop Flags +[cols="1,1,1", width=100%, align="center", options="header"] +|=== +| FLAG Name | Bit Field | Description +| SBI_PMU_STOP_FLAG_RESET | 0 | Reset the counter mapping. +|=== + +The possible error codes returned in `sbiret.error` are shown in the +<<table_pmu_counter_stop_errors>> below. + +[#table_pmu_counter_stop_errors] +.PMU Counter Stop Errors +[cols="1,2", width=100%, align="center", options="header"] +|=== +| Error code | Description +| SBI_SUCCESS | counter stopped successfully. +| SBI_ERR_INVALID_PARAM | `counter_idx` points to an invalid counter. +| SBI_ERR_ALREADY_STOPPED | counter already stopped. +|=== + +=== Function Listing + +[#table_pmu_function_list] +.PMU Function List +[cols="3,1,1", width=70%, align="center", options="header"] +|=== +| Function Name | FID | EID +| sbi_pmu_num_counters | 0 | 0x504D55 +| sbi_pmu_counter_get_info | 1 | 0x504D55 +| sbi_pmu_counter_config_matching | 2 | 0x504D55 +| sbi_pmu_counter_fw_read | 3 | 0x504D55 +| sbi_pmu_counter_start | 4 | 0x504D55 +| sbi_pmu_counter_stop | 5 | 0x504D55 +|=== + == Experimental SBI Extension Space (EIDs #0x08000000 - #0x08FFFFFF) No management. -- 2.30.1 |
|
|
|
Hi Atish,
toggle quoted message
Show quoted text
-----Original Message-----First of all thanks for updating the SBI PMU proposal based on your findings and experience in implementing it. I have few comments below. I will address this comments myself and send a v5 patch. ---We have to explicitly state that `event_data` is 64 bits wide so that we can accommodate Sscof extension. +Some additional clarification is required for HW_CPU_CYCLES, HW_REF_CPU_CYCLES, and HW_BUS_CYCLES to avoid confusion. +For raw events, the `event_data` should have 56 bits wide non-zero event value to be programmed in MHPMEVENTx CSRs because upper 8bits of MHPMEVNTx CSR in Sscof extension are filter bits. +We should mention "SBI version" in PMU function list table. == Experimental SBI Extension Space (EIDs #0x08000000 - #0x08FFFFFF)Regards, Anup |
|
|
|
toggle quoted message
Show quoted text
-----Original Message-----The "NOTE" section contradicts with definition of hardware raw events because by definition we expect RAW value in event_data to be written in MHPMEVENTx CSR. Further, "xyz" value in illustrated in "NOTE" cannot be 64 bits wide because when Sscof extension is available the upper 8-bits of MHPMEVENTx CSR are filter bits. I will consider this in v5 patch. + |
|
|