Date   

Re: A proposal to enhance RISC-V HPM (Hardware Performance Monitor)

Bill Huffman
 


On 8/18/20 8:19 AM, Chuanhua Chang wrote:
EXTERNAL MAIL

Hi, Anup,

Regarding the aspect of a "read-only" CSR, in the RISC-V PMP design, a control bit "L" in the pmpcfg CSR, once set, will change the corresponding read/write pmpaddr CSR to a "read-only" CSR.

They're not the same idea.  The PMP L bit changes the meaning of a write.  Rather than becoming a read-only register, it remains read-write, but writes behave differently and do nothing.  Registers with read-only numbers always take an exception on write.  The idea behind this is that exceptions do not depend on data in CSR registers to avoid difficult timing/layout issues.

       Bill


So fundamentally, I do not see any reason why a control bit cannot change a "read-only" CSR to a "read/write" CSR. It should be similar to the above case in terms of implementation complexity.

And regarding the aspect of a "user" CSR, I also do not see any reason why a higher privileged mode such as S-mode cannot write to a user register, once the write permission is allowed by M-mode.

Regards,
Chuanhua


Re: A proposal to enhance RISC-V HPM (Hardware Performance Monitor)

Anup Patel
 

Hi Chuanhua,

 

Even if we ignore the RISC-V CSR range violation in allowing writes to HPMCOUNTER CSR from S-mode, still the “bypass-sbi” DT property is not an acceptable solution.

 

The “bypass-sbi” DT property will make Linux PMU driver do things differently for HS-mode and VS-mode. Further, it is totally unclear how “bypass-sbi” DT property should be used in nested virtualization because here we will have virtual VS-mode (Guest OS) running on virtual HS-mode (Guest Hypervisor) which in-turn runs on real HS-mode (Host Hypervisor). For a clean nested virtualization, both HS-mode (Hypervisor) and VS-mode (Guest) should write HPMCOUNTER CSR in the same way.

 

Further, the “bypass-sbi” DT property cannot be used for existing RISC-V platforms (SiFive Unleashed, Microchip PolarFire, etc) because existing HARDWARE don’t have proposed “write-enable” bit in HPMEVENT CSR. This means Linux PMU driver will again have to do things differently for existing RISC-V platforms.

 

I think if we want to allow S-mode direct writes to HPMCOUNTER CSRs along with clean nested virtualization then it is better to add separate HS-mode and VS-mode CSRs. Although, I am still wondering why we should allow S-mode direct writes to HPMCOUNTER CSRs considering Linux perf tools are only used for debugging and analysis.

 

Regards,

Anup

 

From: tech-privileged@... <tech-privileged@...> On Behalf Of Chuanhua Chang
Sent: 18 August 2020 20:49
To: tech-privileged@...
Subject: Re: [RISC-V] [tech-privileged] A proposal to enhance RISC-V HPM (Hardware Performance Monitor)

 

Hi, Anup,

Regarding the aspect of a "read-only" CSR, in the RISC-V PMP design, a control bit "L" in the pmpcfg CSR, once set, will change the corresponding read/write pmpaddr CSR to a "read-only" CSR.

So fundamentally, I do not see any reason why a control bit cannot change a "read-only" CSR to a "read/write" CSR. It should be similar to the above case in terms of implementation complexity.

And regarding the aspect of a "user" CSR, I also do not see any reason why a higher privileged mode such as S-mode cannot write to a user register, once the write permission is allowed by M-mode.

Regards,
Chuanhua


Re: A proposal to enhance RISC-V HPM (Hardware Performance Monitor)

Chuanhua Chang
 

Hi, Anup,

Regarding the aspect of a "read-only" CSR, in the RISC-V PMP design, a control bit "L" in the pmpcfg CSR, once set, will change the corresponding read/write pmpaddr CSR to a "read-only" CSR.

So fundamentally, I do not see any reason why a control bit cannot change a "read-only" CSR to a "read/write" CSR. It should be similar to the above case in terms of implementation complexity.

And regarding the aspect of a "user" CSR, I also do not see any reason why a higher privileged mode such as S-mode cannot write to a user register, once the write permission is allowed by M-mode.

Regards,
Chuanhua


Re: A proposal to enhance RISC-V HPM (Hardware Performance Monitor)

Anup Patel
 

Hi Chuanhua,

 

The fact that “write-enable” bit in HPMEVENT CSR makes corresponding HPMCOUNTER as writeable violates the RISC-V CSR numbering scheme of RISC-V privilege spec because it allows S-mode writing to a CSR from “User-read-only” range.

 

Regards,

Anup

 

From: tech-privileged@... <tech-privileged@...> On Behalf Of Chuanhua Chang
Sent: 18 August 2020 15:35
To: tech-privileged@...
Subject: Re: [RISC-V] [tech-privileged] A proposal to enhance RISC-V HPM (Hardware Performance Monitor)

 

The idea of adding write enable bit without adding extra CSR in the read/write register range is that the setting of the write enable bit will change the read-only CSR to a read/write CSR.

Chuanhua


Re: A proposal to enhance RISC-V HPM (Hardware Performance Monitor)

Anup Patel
 

Hi Alan,

 

Like mentioned previously, having different mechanism for HS-mode and VS-mode to write HPMCOUNTER CSR is not acceptable. The “bypass-sbi” DT property only means that Linux PMU driver is now aware whether it is running natively or running inside Guest/VM. This is totally hacky and won’t be acceptable.

 

Regards,

Anup

 

From: tech-privileged@... <tech-privileged@...> On Behalf Of alankao
Sent: 18 August 2020 15:17
To: tech-privileged@...
Subject: Re: [RISC-V] [tech-privileged] A proposal to enhance RISC-V HPM (Hardware Performance Monitor)

 

Hi Anup,

We did the experiment based on our own settings and not yet consider the SBI extension proposal.

Please consider the approach in #278 with one additional condition: Any platform that supports configurations more than M-S-U should not provide a PMU with "bypass-sbi" attribute, like QEMU virt.  Neither VS- nor  HS- usage will be affected by this bit.  Then, you ask, what about emulating a platform that aims to only runs on M-S-U machines?  Well, the one who ports the platform to QEMU/other simulators should put some warning message when the attempt to write happens, rather than implement the whole save-restore just for the PMU status.

We don't need to add many CSRs.  Just one bit in hpmevent*.

Regards,
Alan


Re: A proposal to enhance RISC-V HPM (Hardware Performance Monitor)

Chuanhua Chang
 

The idea of adding write enable bit without adding extra CSR in the read/write register range is that the setting of the write enable bit will change the read-only CSR to a read/write CSR.

Chuanhua


Re: A proposal to enhance RISC-V HPM (Hardware Performance Monitor)

alankao
 

Hi Anup,

We did the experiment based on our own settings and not yet consider the SBI extension proposal.

Please consider the approach in #278 with one additional condition: Any platform that supports configurations more than M-S-U should not provide a PMU with "bypass-sbi" attribute, like QEMU virt.  Neither VS- nor  HS- usage will be affected by this bit.  Then, you ask, what about emulating a platform that aims to only runs on M-S-U machines?  Well, the one who ports the platform to QEMU/other simulators should put some warning message when the attempt to write happens, rather than implement the whole save-restore just for the PMU status.

We don't need to add many CSRs.  Just one bit in hpmevent*.

Regards,
Alan


Re: A proposal to enhance RISC-V HPM (Hardware Performance Monitor)

Anup Patel
 

Hi Alan,

 

Thanks for the data points. The proposed SBI_PMU_COUNTER_CONFIG_MATCHING will helps us minimize SBI calls for configuring HPMEVENT CSR. I am not sure if you have considered latest SBI PMU extension proposal for benchmarking.

 

My question in-context of Hypervisors is still unanswered. The existing HPMCOUNTER CSRs are in read-only CSR range. We will need separate CSR range from S-mode Read-Write CSRs. Further, we will also need separate CSRs from HS-mode to save-restore VS-mode HPMCOUNTER CSRs states.

 

A more general question is that is it worth to add so many CSRs and add increase overhead for Hypervisors just to make HPMCOUNTER writeable form HS-mode and VS-mode. We should also consider the fact the Linux perf will be mostly used for debugging and analysis.

 

Regards,

Anup

 

From: tech-privileged@... <tech-privileged@...> On Behalf Of alankao
Sent: 18 August 2020 14:12
To: tech-privileged@...
Subject: Re: [RISC-V] [tech-privileged] A proposal to enhance RISC-V HPM (Hardware Performance Monitor)

 

 

Hi Anup,

 

Fair enough.  Of course, your conclusion is totally correct: "we CAN EASILY set the HPM* ...," but the real question here is in point 1: Are counter updates really that in-frequent so?  So far, nobody provided any convincing pieces of evidence. Yet, I would like to share our findings from an out-of-the-box experiment.

 

The system for testing is a 4.17-based Linux branch which contains andes_pmu patch, running on our A27 design on FPGA.  The testing command was 

 

time perf record -e cache-references ./whestone

 

where "cache-references" is an andes_pmu cache event and the target program is a normal whestone executable.  We tested three different settings,

 

Baseline: all HPM CSRs are written in S-mode using plain csr_write.

SBI1: HPM counter CSRs are written with SBI calls.
SBI2: HPM counter and event CSRs are written with SBI calls.

We ran each of them 20 times and calculate the average of "real" part of the output.  It turns out that, (in seconds)
Baseline:  23.131
SBI1: 24.840 (increasing 7.4% more time from baseline)
SBI2: 25.691 (increasing 11.1% more time from baseline)

Considering the result, does the write-enable bit count as a benefit now?
Also, it's perfectly fine that QEMU virt platform doesn't support PMUs which allow S-mode to write HPM CSRs, so I don't think #279 answers #280 and #278.

Thanks,
Alan


Re: A proposal to enhance RISC-V HPM (Hardware Performance Monitor)

alankao
 


Hi Anup,

 

Fair enough.  Of course, your conclusion is totally correct: "we CAN EASILY set the HPM* ...," but the real question here is in point 1: Are counter updates really that in-frequent so?  So far, nobody provided any convincing pieces of evidence. Yet, I would like to share our findings from an out-of-the-box experiment.

 

The system for testing is a 4.17-based Linux branch which contains andes_pmu patch, running on our A27 design on FPGA.  The testing command was 

 

time perf record -e cache-references ./whestone

 

where "cache-references" is an andes_pmu cache event and the target program is a normal whestone executable.  We tested three different settings,

 

Baseline: all HPM CSRs are written in S-mode using plain csr_write.

SBI1: HPM counter CSRs are written with SBI calls.
SBI2: HPM counter and event CSRs are written with SBI calls.

We ran each of them 20 times and calculate the average of "real" part of the output.  It turns out that, (in seconds)
Baseline:  23.131
SBI1: 24.840 (increasing 7.4% more time from baseline)
SBI2: 25.691 (
increasing 11.1% more time from baseline)

Considering the result, does the write-enable bit count as a benefit now?
Also, it's perfectly fine that QEMU virt platform doesn't support PMUs which allow S-mode to write HPM CSRs, so 
I don't think #279 answers #280 and #278.

Thanks,
Alan


Re: csrrc/csrrs with mip, sip and uip

John Hauser
 

Allen Baum wrote:
When mip is *read* with a CSR instruction, the *value* of the SEIP bit
returned in the rd destination register is the logical-OR of the
software-writable bit and the interrupt signal from the interrupt
controller. However, the *value* *used* in the read-modify-write sequence
of a CSRRS or CSRRC instruction contains only the software-writable SEIP
bit, ignoring the interrupt value from the external interrupt controller.
If you interpret "value used" as the value read and returned in Rd, then
the value read does differ.
If you interpret "value used" as the value written into the
software-writable bit, then it doesn't.
The problem here is how to interpret "used", and I interpreted is as it was
used (sorry) in the previous sentence.
The interpretation is supposed to be the second one.

Normally, I would expect to execute CSRRC to clear the existing pending
bit, but to OR in any incoming interrupt into the SW writeable bit order
not to lose it.
Your interpretation can cause an incoming interrupt to be lost.
Actually, no, for two reasons:

First, supervisor-level interrupts are normally handled in S mode, and
bit SEIP isn't writable in sip. And obviously S mode can't touch mip.
So, a supervisor-level OS can't clear the pending interrupt by clearing
sip.SEIP; it doesn't have that ability.

Second, for M mode, the incoming external interrupt for SEIP is not
affected by a write to mip, only the software-writable bit is. So,
only the software-writable interrupt could possibly be "lost" this way,
not the externally sourced interrupt. I put _lost_ in quotation marks
because the only reason to write to clear mip.SEIP is to intentionally
clear the software-writable bit. If that's the intention, then the
"loss" of that interrupt isn't an accident.

I think the wording here needs to be updated to remove any possible
ambiguity.
I won't disagree.

- John Hauser


Re: csrrc/csrrs with mip, sip and uip

Allen Baum
 

You say "The value that a CSR instruction reads from mip into the instruction's
rd destination is not affected by which CSR instruction does the read,
whether it's CSRR, CSRRW, CSRRC, or CSRRS. "

but the priv spec (from July, anyway) says:

When mip is read with a CSR instruction, the value of the SEIP bit returned in the rd destination register is the logical-OR of the software-writable bit and the interrupt signal from the interrupt controller. However, the value used in the read-modify-write sequence of a CSRRS or CSRRC instruction contains only the software-writable SEIP bit, ignoring the interrupt value from the external interrupt controller. 

If you interpret "value used" as the value read and returned in Rd, then the value read does differ.
If you interpret "value used" as the value written into the software-writable bit, then it doesn't.
The problem here is how to interpret "used", and I interpreted is as it was used (sorry) in the previous sentence.
Normally, I would expect to execute CSRRC to clear the existing pending bit, but to OR in any incoming interrupt into the SW writeable bit order not to lose it.
Your interpretation can cause an incoming interrupt to be lost.

I think the wording here needs to be updated to remove any possible ambiguity.


On Thu, Aug 13, 2020 at 12:14 PM John Hauser <jh.riscv@...> wrote:
Simon Davidmann wrote:
> The Privileged Architecture specification describes special behavior
> for mip.SEIP as follows:
>
> > When mip is read with a CSR instruction, the value of the SEIP bit returned
> > in the rd destination register is the logical-OR of the software-writable
> > bit and the interrupt signal from the interrupt controller. However, the
> > value used in the read-modify-write sequence of a CSRRS or CSRRC instruction
> > contains only the software-writable SEIP bit, ignoring the interrupt value
> > from the external interrupt controller.
>
> > The SEIP field behavior is designed to allow a higher privilege layer to
> > mimic external interrupts cleanly, without losing any real external
> > interrupts. The behavior of the CSR instructions is slightly modified from
> > regular CSR accesses as a result.
>
> I think this description needs improvement, because the intent is not
> fully clear for SEIP , or other bits. In particular:
> 1. What about other set-pending bits that are writable by software?
> For example, if the N extension is implemented, how do mip.UEIP and
> sip.UEIP behave?

The section you are quoting, 3.1.9, "Machine Interrupt Registers (mip
and mie)", says nothing about the N extension.  If you look at the
figures for the mip register in that section, there is no UEIP bit
there.  If the N extension's UEIP bit should have any special behavior,
the place for this to be specified would be the chapter on the
N extension, the same place where UEIP is first introduced.

As far as UEIP is concerned, I think your real complaint is that the
chapter on the N extension has never been completed or maintained.
I believe many of the principal movers are no longer interested in
developing or promoting this extension.  For the N extension chapter to
be completed, somebody has to step up who wants this to be done.  (My
own personal opinion is that we should drop the N extension.)

As to the bits of mip defined in section 3.1.9, I think the fact that
the document spells out a special case explicitly for SEIP and does not
do the same for any other bits in mip should be ample evidence that the
special case applies only to SEIP and not to any other bits defined for
mip in that section.

> 2. For which bits does any externally-asserted interrupt contribute
> to the result seen in rd for csrrc or csrrs ? For example, would the
> external value of mip.SEIP contribute to rd in this case, or is just
> the software-writable bit value seen?

The value that a CSR instruction reads from mip into the instruction's
rd destination is not affected by which CSR instruction does the read,
whether it's CSRR, CSRRW, CSRRC, or CSRRS.  In all cases, externally
asserted interrupts for bits MEIP and SEIP show up in the value written
to the rd destination.

> As a general case, imagine that:
> 1. A system using the N extension is being used;
> 2. All interrupts are delegated to their lowest possible privilege
>    level using mideleg and sideleg ;
> 3. All interrupts are disabled;
> 4. Interrupts MEI , SEI , UEI , MTI , STI and UTI are all asserted
>    externally (so csrr t1, mip returns 0xbb0 ).
> 5. No software pending bits are set for these interrupts.
>
> Given the above set up, what value is observed in t1 in each of these
> cases:  [...]

As this question involves the N extension, I'll leave it others to say
how they think the N extension should be defined.

If you'd like to ask about a different example that doesn't involve the
N extension, I can try to answer.

Regards,

    - John Hauser




Re: csrrc/csrrs with mip, sip and uip

John Hauser
 

Simon Davidmann wrote:
The Privileged Architecture specification describes special behavior
for mip.SEIP as follows:

When mip is read with a CSR instruction, the value of the SEIP bit returned
in the rd destination register is the logical-OR of the software-writable
bit and the interrupt signal from the interrupt controller. However, the
value used in the read-modify-write sequence of a CSRRS or CSRRC instruction
contains only the software-writable SEIP bit, ignoring the interrupt value
from the external interrupt controller.
The SEIP field behavior is designed to allow a higher privilege layer to
mimic external interrupts cleanly, without losing any real external
interrupts. The behavior of the CSR instructions is slightly modified from
regular CSR accesses as a result.
I think this description needs improvement, because the intent is not
fully clear for SEIP , or other bits. In particular:
1. What about other set-pending bits that are writable by software?
For example, if the N extension is implemented, how do mip.UEIP and
sip.UEIP behave?
The section you are quoting, 3.1.9, "Machine Interrupt Registers (mip
and mie)", says nothing about the N extension. If you look at the
figures for the mip register in that section, there is no UEIP bit
there. If the N extension's UEIP bit should have any special behavior,
the place for this to be specified would be the chapter on the
N extension, the same place where UEIP is first introduced.

As far as UEIP is concerned, I think your real complaint is that the
chapter on the N extension has never been completed or maintained.
I believe many of the principal movers are no longer interested in
developing or promoting this extension. For the N extension chapter to
be completed, somebody has to step up who wants this to be done. (My
own personal opinion is that we should drop the N extension.)

As to the bits of mip defined in section 3.1.9, I think the fact that
the document spells out a special case explicitly for SEIP and does not
do the same for any other bits in mip should be ample evidence that the
special case applies only to SEIP and not to any other bits defined for
mip in that section.

2. For which bits does any externally-asserted interrupt contribute
to the result seen in rd for csrrc or csrrs ? For example, would the
external value of mip.SEIP contribute to rd in this case, or is just
the software-writable bit value seen?
The value that a CSR instruction reads from mip into the instruction's
rd destination is not affected by which CSR instruction does the read,
whether it's CSRR, CSRRW, CSRRC, or CSRRS. In all cases, externally
asserted interrupts for bits MEIP and SEIP show up in the value written
to the rd destination.

As a general case, imagine that:
1. A system using the N extension is being used;
2. All interrupts are delegated to their lowest possible privilege
level using mideleg and sideleg ;
3. All interrupts are disabled;
4. Interrupts MEI , SEI , UEI , MTI , STI and UTI are all asserted
externally (so csrr t1, mip returns 0xbb0 ).
5. No software pending bits are set for these interrupts.

Given the above set up, what value is observed in t1 in each of these
cases: [...]
As this question involves the N extension, I'll leave it others to say
how they think the N extension should be defined.

If you'd like to ask about a different example that doesn't involve the
N extension, I can try to answer.

Regards,

- John Hauser


Re: P extension instruction opcode encoding allocation

Allen Baum
 

Ken Dockser wrote a document on instruction encoding guidelines, but not the actual values of the minor/major opcodes, or sub-minor (functX) fields.
Attached
 

On Tue, Aug 11, 2020 at 7:33 AM mark <markhimelstein@...> wrote:
Krste has said (correct me if I am wrong) that the unpriv SC owns the opcode space. I know there is a lot of overlap in the SC members but I suggest we get that SC officially in the loop.

I have CC'ed them here. 

Unpriv SC, do you have a preferred process for proposing new instruction opcode space, format, etc.? If not, I suggest you establish one.

Mark


On Tue, Aug 11, 2020 at 2:30 AM Chuanhua Chang <chchang@...> wrote:

P extension instructions need to allocate opcode encoding space officially in the OP opcode space or other major opcode (such as reserved opcode).

What is the best way to decide on this and officially allocate encoding space for the P extension instructions?

 

Thanks.

 

Chuanhua


Re: P extension instruction opcode encoding allocation

mark
 

Krste has said (correct me if I am wrong) that the unpriv SC owns the opcode space. I know there is a lot of overlap in the SC members but I suggest we get that SC officially in the loop.

I have CC'ed them here. 

Unpriv SC, do you have a preferred process for proposing new instruction opcode space, format, etc.? If not, I suggest you establish one.

Mark


On Tue, Aug 11, 2020 at 2:30 AM Chuanhua Chang <chchang@...> wrote:

P extension instructions need to allocate opcode encoding space officially in the OP opcode space or other major opcode (such as reserved opcode).

What is the best way to decide on this and officially allocate encoding space for the P extension instructions?

 

Thanks.

 

Chuanhua


csrrc/csrrs with mip, sip and uip

Simon Davidmann Imperas
 

We posted this on https://groups.google.com/a/groups.riscv.org/g/isa-dev/
but had no response in 2 weeks - so maybe this is a better place:
Looking forward to a response.
Simon

The Privileged Architecture specification describes special behavior for mip.SEIP as follows:
 
When mip is read with a CSR instruction, the value of the SEIP bit returned in the rd destination register is the logical-OR of the software-writable bit and the interrupt signal from the interrupt controller. However, the value used in the read-modify-write sequence of a CSRRS or CSRRC instruction contains only the software-writable SEIP bit, ignoring the interrupt value from the external interrupt controller.
 
The SEIP field behavior is designed to allow a higher privilege layer to mimic external interrupts cleanly, without losing any real external interrupts. The behavior of the CSR instructions is slightly modified from regular CSR accesses as a result.
 
I think this description needs improvement, because the intent is not fully clear for SEIP, or other bits. In particular:
1. What about other set-pending bits that are writable by software? For example, if the N extension is implemented, how do mip.UEIP and sip.UEIP behave?
2. For which bits does any externally-asserted interrupt contribute to the result seen in rd for csrrc or csrrs? For example, would the external value of mip.SEIP contribute to rd in this case, or is just the software-writable bit value seen?
 
As a general case, imagine that:
1. A system using the N extension is being used;
2. All interrupts are delegated to their lowest possible privilege level using mideleg and sideleg;
3. All interrupts are disabled;
4. Interrupts MEISEIUEIMTISTI and UTI are all asserted externally (so csrr t1, mip returns 0xbb0).
5. No software pending bits are set for these interrupts.
 
Given the above set up, what value is observed in t1 in each of these cases:
 
li      t0, 0
csrrc   t1, mip, t0    // t1 = ???
csrrc   t1, sip, t0    // t1 = ???
csrrc   t1, uip, t0    // t1 = ???
 
(Note that no CSR state is changed by these instructions - only the result in t1 is of interest.)
 
And given the same set up, which (if any) software-writable bits are set by these instructions:
 
li      t0, 0
csrrs   t1, mip, t0
csrrs   t1, sip, t0
 
(In other words, what externally-asserted interrupt signal values are propagated to software-writable bits?)
 
Thanks.
 


P extension instruction opcode encoding allocation

Chuanhua Chang
 

P extension instructions need to allocate opcode encoding space officially in the OP opcode space or other major opcode (such as reserved opcode).

What is the best way to decide on this and officially allocate encoding space for the P extension instructions?

 

Thanks.

 

Chuanhua


Re: Small tweak to Privileged spec regarding PMP management?

Greg Favor
 

One could argue that the current spec and the sentence in question (with or without the suggested modification), is clear in calling out when an sfence.vma is not required.  But I agree that adding a short non-normative note would avoid any chance of misunderstanding.

Greg


On Mon, Aug 10, 2020 at 12:00 AM Allen Baum <allen.baum@...> wrote:
Do you want to add more detail about the page-based virtual memory being disabled case? 
    (that some implementations may require sfence.vma, depending on whether they do XXX with their TLB)?
That would be non-normative, but will alert designers about this corner case.

On Sun, Aug 9, 2020 at 11:45 PM Greg Favor <gfavor@...> wrote:
In section 3.6.2 of the Privileged spec discussing changing PMP settings, it currently says:
"If page-based virtual memory is not implemented, or when it is disabled, memory accesses check the PMP settings synchronously, so no fence is needed."

I would like to suggest removing "or when it is disabled" and just say:
"If page-based virtual memory is not implemented, memory accesses check the PMP settings synchronously, so no fence is needed."

The motivation is that high-performance implementations that support page-based virtual memory have TLBs and want to use them to handle all fetch/load/store memory accesses as they go down load/store execution pipelines during all modes of execution - including while in M-mode.  In the case of M mode, they would effectively just be caching PMA/PMP permission/access control info (as well as identity address mappings).

For designs that implement page-based virtual memory and use their TLBs as described (which is generally true in high-performance designs), not requiring that M-mode software do an sfence.vma after a series of PMP CSR writes means that these CSR writes cannot simply be implemented as CSR writes, but instead each PMP CSR write needs to also perform a heavyweight sfence.vma operation.  This is both heavily redundant (across a series of PMP writes) and is unnatural for an aggressive o-o-o design RISC design in which an sfence.vma operation really is a very strong fencing operation as well as TLB invalidation operation.  (Put differently, a key point of RISC architecture is to simplify hardware in ways that software can easily and efficiently support.)

Given that M-mode software runs a lot of implementation-specific code (including code related with PMA and PMP management), this spec tweak allows for some implementations to simplify their hardware design and include an sfence.vma in their M-mode PMP CSR writing code (while other implementations can choose to not include an sfence.vma in their M-mode code).  But also note that all designs need to at least selectively do an sfence.vma (per section 3.6.2), so this essentially means that the M-mode code would simply always do an sfence.vma after a series of PMP writes.

Lastly note that this change is backward compatible in that software that does do an sfence.vma after PMP changes will run on "old" designs that support page-based virtual memory yet access PMA's and PMP's inline with load/store execution while in M-mode.

Any objections to this simple accomodation for high-performance CPU designs?

Greg


Re: Small tweak to Privileged spec regarding PMP management?

Allen Baum
 

Do you want to add more detail about the page-based virtual memory being disabled case? 
    (that some implementations may require sfence.vma, depending on whether they do XXX with their TLB)?
That would be non-normative, but will alert designers about this corner case.

On Sun, Aug 9, 2020 at 11:45 PM Greg Favor <gfavor@...> wrote:
In section 3.6.2 of the Privileged spec discussing changing PMP settings, it currently says:
"If page-based virtual memory is not implemented, or when it is disabled, memory accesses check the PMP settings synchronously, so no fence is needed."

I would like to suggest removing "or when it is disabled" and just say:
"If page-based virtual memory is not implemented, memory accesses check the PMP settings synchronously, so no fence is needed."

The motivation is that high-performance implementations that support page-based virtual memory have TLBs and want to use them to handle all fetch/load/store memory accesses as they go down load/store execution pipelines during all modes of execution - including while in M-mode.  In the case of M mode, they would effectively just be caching PMA/PMP permission/access control info (as well as identity address mappings).

For designs that implement page-based virtual memory and use their TLBs as described (which is generally true in high-performance designs), not requiring that M-mode software do an sfence.vma after a series of PMP CSR writes means that these CSR writes cannot simply be implemented as CSR writes, but instead each PMP CSR write needs to also perform a heavyweight sfence.vma operation.  This is both heavily redundant (across a series of PMP writes) and is unnatural for an aggressive o-o-o design RISC design in which an sfence.vma operation really is a very strong fencing operation as well as TLB invalidation operation.  (Put differently, a key point of RISC architecture is to simplify hardware in ways that software can easily and efficiently support.)

Given that M-mode software runs a lot of implementation-specific code (including code related with PMA and PMP management), this spec tweak allows for some implementations to simplify their hardware design and include an sfence.vma in their M-mode PMP CSR writing code (while other implementations can choose to not include an sfence.vma in their M-mode code).  But also note that all designs need to at least selectively do an sfence.vma (per section 3.6.2), so this essentially means that the M-mode code would simply always do an sfence.vma after a series of PMP writes.

Lastly note that this change is backward compatible in that software that does do an sfence.vma after PMP changes will run on "old" designs that support page-based virtual memory yet access PMA's and PMP's inline with load/store execution while in M-mode.

Any objections to this simple accomodation for high-performance CPU designs?

Greg


Small tweak to Privileged spec regarding PMP management?

Greg Favor
 

In section 3.6.2 of the Privileged spec discussing changing PMP settings, it currently says:
"If page-based virtual memory is not implemented, or when it is disabled, memory accesses check the PMP settings synchronously, so no fence is needed."

I would like to suggest removing "or when it is disabled" and just say:
"If page-based virtual memory is not implemented, memory accesses check the PMP settings synchronously, so no fence is needed."

The motivation is that high-performance implementations that support page-based virtual memory have TLBs and want to use them to handle all fetch/load/store memory accesses as they go down load/store execution pipelines during all modes of execution - including while in M-mode.  In the case of M mode, they would effectively just be caching PMA/PMP permission/access control info (as well as identity address mappings).

For designs that implement page-based virtual memory and use their TLBs as described (which is generally true in high-performance designs), not requiring that M-mode software do an sfence.vma after a series of PMP CSR writes means that these CSR writes cannot simply be implemented as CSR writes, but instead each PMP CSR write needs to also perform a heavyweight sfence.vma operation.  This is both heavily redundant (across a series of PMP writes) and is unnatural for an aggressive o-o-o design RISC design in which an sfence.vma operation really is a very strong fencing operation as well as TLB invalidation operation.  (Put differently, a key point of RISC architecture is to simplify hardware in ways that software can easily and efficiently support.)

Given that M-mode software runs a lot of implementation-specific code (including code related with PMA and PMP management), this spec tweak allows for some implementations to simplify their hardware design and include an sfence.vma in their M-mode PMP CSR writing code (while other implementations can choose to not include an sfence.vma in their M-mode code).  But also note that all designs need to at least selectively do an sfence.vma (per section 3.6.2), so this essentially means that the M-mode code would simply always do an sfence.vma after a series of PMP writes.

Lastly note that this change is backward compatible in that software that does do an sfence.vma after PMP changes will run on "old" designs that support page-based virtual memory yet access PMA's and PMP's inline with load/store execution while in M-mode.

Any objections to this simple accomodation for high-performance CPU designs?

Greg


Proposed WG: RISC V needs CMOs, and hence a CMO Working Group

Andy Glew Si5
 

RISC V needs CMOs, and hence a CMO Working Group





All successful computer instruction sets have Cache Management Operations (CMOs).

Several RISC-V systems have already defined implementation specific CMO instructions. It is desirable to have standard CMO instructions to facilitate portable software.

CMOs do things like flushing dirty data and invalidating clean data for use cases that include non-coherent DMA I/O, security (e.g. Spectre), power management (flush to battery backed-up DRAM), persistence (flush to NVRAM), and more.

CMOs cut across several problem domains. It is desirable to have a consistent approach, rather than different idiosyncratic instructions for different problem domains. RISC-V therefore needs a CMO working group that will coordinate with any working groups in those overlapping domains.



Administrivia

2020/8/5: Email proposing this will soon be sent to the RISC-V Technical Steering Committee and other mailing lists, seeking approval of the formation of such a CMO working group.

Here linked is a wiki version of the WG proposal RISC V needs CMOs, and hence a CMO Working Group. Also a CMOs WG Draft Proposed Charter - although probably too long.

Assuming the CMO WG is approved:

Please indicate if you are interested by replying to this email (to me, Andy Glew). To faciliate scheduling of meetings, please indicate timezone.

A risc.org mailing list should be set up soon.

We have already set up https://github.com/riscv/riscv-CMOs, and will arrange permissions for working group members as soon as possible.

Here linked is a CMOs WG Draft Proposed Charter.

Proposals:

  • At least one CMO proposal has been developed in some detail. It is linked to from https://github.com/riscv/riscv-CMOs, and may soon be moved to this official place.
  • We welcome: Other proposals, and/or examples of implementation specific CMO extensions already implemented




I look forward to meeting other folks interested in CMOs!


--- Sorry: Typos (Speech-Os?) Writing Errors <= Speech Recognition <= Computeritis

821 - 840 of 1122