How to retrieve Rocket Core module parameters from Top module in fpga-zynq repository

Question

My goal is to add counters to the rocket class of the Rocket Core module of the fpga-zynq repository. I want to count parameters like ctrl_stalld, id_stall_fpu ... while the chip is running on a FPGA.

I successfully generated Verilog code and a Vivado project for the default rocket configuration (ZynqConfig). I loaded it onto the ZedBoard's FPGA and got it running. I know how to implement counters in the core, but i'm not sure how to retrieve them from outside.

I figured, that a connection between fpga-zynq/rocket-chip/src/main/scala/rocket/rocket.scala and fpga-zynq/common/src/main/scala/Top.scala probably has to be established, since i'm able to access and further connect the Top modules IO-Ports inside Xilinx Vivado 2016.2. I'd assume that the projects hierarchy has to be backtraced from rocket module to Top module and all IO-Ports of all modules in between connected.

However, i do not quite understand the projects hierarchy. I can't find a connection over the many many modules between rocket and Top. I hope the image clarifies what i' trying to say.

The arrows represent IO connections between modules.

The black dots and "?" represent unknown hierarchy (may be more complex).

This is NOT an actual representation of the fpga-zynq projects hierarchy.

Answer 1

As predicted, all the I/O Ports of all the modules between Top and rocket had to be connected. NOTE: The following solution is meant to work only for a Single Core Rocket!

I'm gonna go through the steps necessary to implement a 32-bit counter in the Rocket core and connecting it with the Top module.

Where to find the following classes:

Rocket -- fpga-zynq/rocket-chip/src/main/scala/rocket/rocket.scala

RocketTile -- fpga-zynq/rocket-chip/src/main/scala/rocket/tile.scala

BaseCorePlexModule -- fpga-zynq/rocket-chip/src/main/scala/coreplex/BaseCoreplexModule.scala

BaseTop -- fpga-zynq/rocket-chip/src/main/scala/rocketchip/BaseTop.scala

Top -- fpga-zynq/common/src/main/scala/Top.scala

1. Implementing a counter in the Rocket class

First, we create a new Bundle for all our counters, outside the Rocket class (for this example, it will be just one counter, but you can add more counters and other parameters to the bundle as you go)

class CounterBundle extends Bundle{
   val counter = UInt(OUTPUT, width = 32)
}

We define an instance of this Bundle in the I/O Bundle of the Rocket class.

class Rocket(implicit p: Parameters) extends CoreModule()(p) {
    val io = new Bundle {
      val interrupts = new TileInterrupts().asInput
      val hartid = UInt(INPUT, xLen)
      val imem = new FrontendIO()(p.alterPartial({ case CacheName => "L1I" }))
      val dmem = new HellaCacheIO()(p.alterPartial({ case CacheName => "L1D" }))
      val ptw = new DatapathPTWIO().flip
      val fpu = new FPUIO().flip
      val rocc = new RoCCInterface().flip
      val cBundle = new CounterBundle
    }
// ...
}

Since our "counter" parameter is essentially a wire and we can't assign a wire to itself, we'll use a register to count and transfer its value to our counter. The counter will be incremented if some condition is true and reset back to 0 if reset is active. We put it inside the Rocket class.

val counter_reg = Reg(init = UInt(0, width = 32))
when(/*some condition*/){
  counter_reg := counter_reg + 1.U
}
io.cBundle.counter := counter_reg

That's it for the Rocket class.

2. Connecting the Rocket class CounterBundle with the Top class

2.1 Rocket → RocketTile

A new Rocket class instance is created in the RocketTile class.

Simply add

val cBundle = new CounterBundle

to the RocketTile I/O's. Now let's go ahead and connect the two bundles inside the RocketTile class.

class RocketTile(clockSignal: Clock = null, resetSignal: Bool = null)
(implicit p: Parameters) extends Tile(clockSignal, resetSignal)(p) {
  val buildRocc = p(BuildRoCC)
  val usingRocc = !buildRocc.isEmpty
  val nRocc = buildRocc.size
  val nFPUPorts = buildRocc.filter(_.useFPU).size

  val core = Module(new Rocket)
  io.cBundle := core.io.cBundle
  // ...
}

If you're wondering where the RocketTile I/O's are defined, look for the TileIO class within the same file (tile.scala).

2.2 RocketTile → BaseCorePlexModule

The BaseCorePlexModule holds a set of tiles. But since we are only creating a counter for a single core rocket, we can just connect the Bundle of the first RocketTile in the set.

First, add

val cBundle = new CounterBundle

to the "abstract class BaseCoreplexBundle" right above the BaseCorePlexModule. As you probably figured, the BaseCorePlexBundle holds all the I/O's for the BaseCorePlexModule.

Then, connect this Bundle with the RocketTile Bundle inside the BaseCorePlexModule.

abstract class BaseCoreplexModule[+L <: BaseCoreplex, +B <: BaseCoreplexBundle](
c: CoreplexConfig, l: L, b: => B)(implicit val p: Parameters) extends LazyModuleImp(l) with HasCoreplexParameters {
   val outer: L = l
   val io: B = b

   // Build a set of Tiles
   val tiles = p(BuildTiles) map { _(reset, p) }
   io.cBundle := tiles.head.io.cBundle
   // ...
 }

2.3 BaseCorePlexModule → BaseTop

This is the last connection to be made before going to the Top class, as the Top class holds a parameter "target" that is a child of BaseTop.

Again, add a new instance of the CounterBundle to the I/O's of this class. The "abstract class BaseTopBundle" holds all the I/O's for BaseTop.

Connect the two.

abstract class BaseTopModule[+L <: BaseTop, +B <: BaseTopBundle](
val p: Parameters, l: L, b: => B) extends LazyModuleImp(l) {
  val outer: L = l
  val io: B = b

  val coreplex = p(BuildCoreplex)(outer.c, p)
  io.cBundle := coreplex.io.cBundle
  // ... 
}

2.4 BaseTop → Top

The Top class holds a parameter "target" of type FPGAZynqTop

val target = LazyModule(new FPGAZynqTop(p)).module

At the bottom of the file, FPGAZynqTop can be found as a child of BaseTop. Therefore, we can access the BaseTop I/O's through the "target" parameter.

For the Top I/O, we are not going to add the CounterBundle, but the 32-bit "counter" parameter from the Bundle. This way, the counter can be accessed in Vivado, as the generated Verilog code will hold a 32-bit wire.

Adding the a 32-bit UInt parameter to the Top I/O

val counter = UInt(OUTPUT, width = 32)

Connecting it with the counter from the CounterBundle.

class Top(implicit val p: Parameters) extends Module {
   val io = new Bundle {
     val ps_axi_slave = new NastiIO()(AdapterParams(p)).flip
     val mem_axi = new NastiIO
     val counter = UInt(OUTPUT, width = 32)
   }

   val target = LazyModule(new FPGAZynqTop(p)).module
   val slave = Module(new ZynqAXISlave(1)(AdapterParams(p)))
   io.counter := target.io.cBundle.counter
   // ...
}

Conclusion

Counting Rocket Core parameters while the chip runs on a FPGA is possible with this configuration. I tested it on a ZedBoard.

With this setup you can simply add more counters / parameters to the CounterBundle without having to go through all the modules between Rocket and Top again. Of course, you will still have to also alter the Top module.

EDIT

After adding more and more parameter to the Bundle, i realised that also adding all of these to the Top module every time becomes annoying. You can add the CounterBundle directly to the Top module I/O. In the generated Verilog code, the CounterBundle will be extracted into all it's containing signals.

My new Top module looks like this. Be sure to import the CounterBundle from the Rocket class

import rocket.CounterBundle

class Top(implicit val p: Parameters) extends Module {
   val io = new Bundle {
     val ps_axi_slave = new NastiIO()(AdapterParams(p)).flip
     val mem_axi = new NastiIO
     val cBundle = new CounterBundle
   }

   val target = LazyModule(new FPGAZynqTop(p)).module
   val slave = Module(new ZynqAXISlave(1)(AdapterParams(p)))
   io.cBundle := target.io.cBundle
   // ...
}

The passage in the generated Verilog code for the Top module then looks like this.

module Top(
  input clock,
  input reset,
  // ...
  output [31:0] io_cBundle_counter
);
// ...
endmodule