Access problem of mtimercmp in a platform with multiple MTIMER devices


Tianyi Xia <tianshi.xty@...>
 

In the SiFive Core-Local Interruptor (CLINT) device , a core can access the mtimcmp register of all cores in the platform.

In the ACLINT spec, If a platform implements multiple MTIMER devices, such as multiple clusters, each cluster implements one MTIMER device, then a core may not be able to access the MTIMER devices of other clusters.

As I understand, it is not necessary for a core to access the mtimecmp of other cores. Is it possible to add a recommended software usage method to the ACLINT spec, for example, it is recommended that the software only use a core to access its own mtimcmp register, but not access mtimcmp of other cores. This can avoid the problem that the software uses a core to access the MTIMER devices of other clusters, but the hardware cannot support it.


Allen Baum
 

The implication of that is that either
 - there is an mmio address that can access different instantiations of mtime/mtimecmp for each requesting hart (depending on the "cluster")
 - each "cluster" can have its own unique mmio address for mtimecmp (which may or may not be accessible to other "clusters")

Is one or either of those a preferred option? The latter sounds like it would be difficult for SW

On Tue, Sep 6, 2022 at 12:16 AM Tianyi Xia via lists.riscv.org <tianshi.xty=alibaba-inc.com@...> wrote:

In the SiFive Core-Local Interruptor (CLINT) device , a core can access the mtimcmp register of all cores in the platform.

In the ACLINT spec, If a platform implements multiple MTIMER devices, such as multiple clusters, each cluster implements one MTIMER device, then a core may not be able to access the MTIMER devices of other clusters.

As I understand, it is not necessary for a core to access the mtimecmp of other cores. Is it possible to add a recommended software usage method to the ACLINT spec, for example, it is recommended that the software only use a core to access its own mtimcmp register, but not access mtimcmp of other cores. This can avoid the problem that the software uses a core to access the MTIMER devices of other clusters, but the hardware cannot support it.


Tianyi Xia <tianshi.xty@...>
 

each "cluster" can have its own unique mmio address for mtimecmp (which may or may not be accessible to other "clusters")

I think this description is better.

 

Assume there are two clusterseach cluster have two coresand each cluster have there own MTIMER device. The mmio address of mtimecmp for each hart may like this:

Cluster0

    Core0: base0+0x0000_0000

    Core1: base0+0x0000_0008

Cluster1

    Core0: base1+0x0000_0000

    Core1: base1+0x0000_0008

Base0 is the MTIER device base address of cluster0, Base1 is the MTIER device base address of cluster0. the mtimecmp of cluster0 core0 may or maynot be accessible to cluster1,  depending on the implementation. If core try to access mtimecmp of other cluster, the action of the access may be write ignore read zero.


The latter sounds like it would be difficult for SW

I think in a platform with multiple MTIMER devices, the mmio address of mtimecmp should be unique to distinguish different MTIMER devices. The hardware can set regular base address to different Mtimer devices. In the above example, assuming base is 0, then base1 may be set to 0x0000_0010. If cluster0 has four cores, then base1 may be set to 0x0000_0020.Then from a software perspective, all mtimecmp registers are addressed consecutively.

 


 


Allen Baum
 

That makes sense, but it does mean that discovery gets more complicated, and (maybe) you need to build separate device trees for each.
But maybe that has to happen anyway? I don't know if DT can be parameterized based on HartID, but that would greatly simplify the work.

On Tue, Sep 6, 2022 at 8:14 PM Tianyi Xia via lists.riscv.org <tianshi.xty=alibaba-inc.com@...> wrote:

each "cluster" can have its own unique mmio address for mtimecmp (which may or may not be accessible to other "clusters")

I think this description is better.

 

Assume there are two clusterseach cluster have two coresand each cluster have there own MTIMER device. The mmio address of mtimecmp for each hart may like this:

Cluster0

    Core0: base0+0x0000_0000

    Core1: base0+0x0000_0008

Cluster1

    Core0: base1+0x0000_0000

    Core1: base1+0x0000_0008

Base0 is the MTIER device base address of cluster0, Base1 is the MTIER device base address of cluster0. the mtimecmp of cluster0 core0 may or maynot be accessible to cluster1,  depending on the implementation. If core try to access mtimecmp of other cluster, the action of the access may be write ignore read zero.


The latter sounds like it would be difficult for SW

I think in a platform with multiple MTIMER devices, the mmio address of mtimecmp should be unique to distinguish different MTIMER devices. The hardware can set regular base address to different Mtimer devices. In the above example, assuming base is 0, then base1 may be set to 0x0000_0010. If cluster0 has four cores, then base1 may be set to 0x0000_0020.Then from a software perspective, all mtimecmp registers are addressed consecutively.

 


 


Anup Patel
 

On Wed, Sep 7, 2022 at 8:44 AM Tianyi Xia via lists.riscv.org
<tianshi.xty=alibaba-inc.com@...> wrote:

each "cluster" can have its own unique mmio address for mtimecmp (which may or may not be accessible to other "clusters")

I think this description is better.



Assume there are two clusters,each cluster have two cores,and each cluster have there own MTIMER device. The mmio address of mtimecmp for each hart may like this:

Cluster0

Core0: base0+0x0000_0000

Core1: base0+0x0000_0008

Cluster1

Core0: base1+0x0000_0000

Core1: base1+0x0000_0008

Base0 is the MTIER device base address of cluster0, Base1 is the MTIER device base address of cluster0. the mtimecmp of cluster0 core0 may or maynot be accessible to cluster1, depending on the implementation. If core try to access mtimecmp of other cluster, the action of the access may be write ignore read zero.


The latter sounds like it would be difficult for SW

I think in a platform with multiple MTIMER devices, the mmio address of mtimecmp should be unique to distinguish different MTIMER devices. The hardware can set regular base address to different Mtimer devices. In the above example, assuming base is 0, then base1 may be set to 0x0000_0010. If cluster0 has four cores, then base1 may be set to 0x0000_0020.Then from a software perspective, all mtimecmp registers are addressed consecutively.
This case is already handled by ACLINT device tree bindings. We
just need two separate MTIMER DT nodes where the mtimecmp
base address will be different in each DT node.

Please refer to the latest OpenSBI sources.

Regards,
Anup