Building Custom RISC-V SoCs in Chipyard Abraham Gonzalez UC - - PowerPoint PPT Presentation

▶
building custom risc v socs in chipyard
SMART_READER_LITE
LIVE PREVIEW

Building Custom RISC-V SoCs in Chipyard Abraham Gonzalez UC - - PowerPoint PPT Presentation

Jun 27, 2023 934 likes •1.79k views

Building Custom RISC-V SoCs in Chipyard Abraham Gonzalez UC Berkeley abe.gonzalez@berkeley.edu Goals Get the basics of modifying a configuration Create a heterogeneous BOOM and Rocket RISC-V SoC Learn how to generate Verilog for

slide-1
SLIDE 1

Abraham Gonzalez UC Berkeley abe.gonzalez@berkeley.edu

Building Custom RISC-V SoCs in Chipyard

slide-2
SLIDE 2

Goals

  • Get the basics of modifying a configuration
  • Create a heterogeneous BOOM and Rocket RISC-V SoC
  • Learn how to generate Verilog for an SoC
  • Learn how to run Verilator RTL simulations
  • Case Study: Integrating a SHA3 accelerator into a Chipyard SoC!
  • Add a SHA3 accelerator to Chipyard
  • Add the accelerator to a configuration!
  • Get Verilog and run Verilator simulations
slide-3
SLIDE 3

Tutorial Roadmap

Custom SoC Configuration RTL Generators RISC-V Cores Multi-level Caches Custom Verilog Peripherals Accelerators Software RTL Simulation VCS Verilator FireSim FPGA-Accelerated Simulation Simulation Debugging Networking Automated VLSI Flow Hammer Tech- plugins Tool- plugins RTL Build Process FIRRTL Transforms FIRRTL IR Verilog FireMarshal Bare-metal & Linux Custom Workload QEMU & Spike

slide-4
SLIDE 4

Tutorial Roadmap

Custom SoC Configuration RTL Generators RISC-V Cores Multi-level Caches Custom Verilog Peripherals Accelerators Software RTL Simulation VCS Verilator FireSim FPGA-Accelerated Simulation Simulation Debugging Networking Automated VLSI Flow Hammer Tech- plugins Tool- plugins RTL Build Process FIRRTL Transforms FIRRTL IR Verilog FireMarshal Bare-metal & Linux Custom Workload QEMU & Spike

slide-5
SLIDE 5

Getting Started

slide-6
SLIDE 6

Having trouble? Raise a hand and someone will come help you. You can find these slides on https://fires.im/micro-2019-tutorial/

Interactive Section!

slide-7
SLIDE 7

How things will work

Interactive Slide

“Follow Along”

Explanation Slide

“What’s happening?”

slide-8
SLIDE 8

How things will work

Terminal Section Inside-a-File Section

# command 1 > echo “Chipyard Rules!” # command 2 > do_this arg1 arg2

// SOME COMMENT HERE class SmallBoomConfig extends Config( new WithTop ++ new WithBootROM ++ new boom.common.WithSmallBooms ++ new boom.common.WithNBoomCores(1) ++ new freechips.rocketchip.system.BaseConfig)

slide-9
SLIDE 9

Interactive

10

# start a tmux session > tmux new –s soc # return to chipyard > cd ~/chipyard-morning/ > ls

slide-10
SLIDE 10

Directory Structure

chipyard-morning/ generators/ rocket-chip/ sha3/ sims/ verilator/ software/ firemarshal/ tools/ chisel/ firrtl/ build.sbt

11

Our library of Chisel generators Utilities for simulating SoCs Config file enumerating generators and dependencies Chisel/FIRRTL Utilities for building RISC-V software

slide-11
SLIDE 11

Build and simulate a heterogeneous BOOM + Rocket SoC

12

slide-12
SLIDE 12

Default Chipyard Project

  • You need a top-level project that combines all the

generators wanted

  • Want Rocket Chip, BOOM, SHA3 accelerator, etc…
  • In our case we are using the `example` project
  • Located in `generators/example`
  • This holds things such as
  • Test Harness code
  • Top-level module (matches the top-level of the DUT)
  • SoC configurations
  • Most of the work will be done in this project

13

chipyard-morning/ generators/ example/ rocket-chip/ boom/ sha3/ sims/ verilator/ tools/ chisel/ firrtl/ tests/ build.sbt

slide-13
SLIDE 13

BOOM Integration

  • BOOM source code already added
  • Located in `generators/boom`
  • Already has default configurations
  • Located in

`generators/example/src/main/scala/ BoomConfigs.scala`

14

chipyard-morning/ generators/ example/ rocket-chip/ boom/ sha3/ sims/ verilator/ tools/ chisel/ firrtl/ tests/ build.sbt

slide-14
SLIDE 14

Interactive

15

# open up the default boom configurations file > cd generators/example/src/main/scala > less BoomConfigs.scala

chipyard-morning/ generators/ example/ src/main/scala/ BoomConfigs.scala rocket-chip/ boom/ sha3/ sims/ verilator/ tools/ chisel/ firrtl/ tests/ build.sbt

slide-15
SLIDE 15

Interactive

16

# open up the default boom configurations file > cd generators/example/src/main/scala > less BoomConfigs.scala

chipyard-morning/ generators/ example/ src/main/scala/ BoomConfigs.scala rocket-chip/ boom/ sha3/ sims/ verilator/ tools/ chisel/ firrtl/ tests/ build.sbt

Reminder: use q to quit less

slide-16
SLIDE 16

Configuring a SoC with BOOM

17

  • Bottom-to-top (right-to-left) config hierarchy
  • Configs can read/override other configs

BoomTilesKey => Seq() RocketTilesKey => Seq() BankedL2Key => Module(new TLBroadcast) BootROMParams => BootROMParams() BuildTop => None class SmallBoomConfig extends Config( new WithTop ++ new WithBootROM ++ new freechips.rocketchip.subsystem.WithInclusiveCache ++ new boom.common.WithSmallBooms ++ new boom.common.WithNBoomCores(1) ++ new freechips.rocketchip.system.BaseConfig) chipyard-morning/ generators/ example/ src/main/scala/ BoomConfigs.scala rocket-chip/ boom/ sha3/ sims/ verilator/ tools/ chisel/ firrtl/ tests/ build.sbt

slide-17
SLIDE 17

Configuring a SoC with BOOM

18

  • Bottom-to-top (right-to-left) config hierarchy
  • Configs can read/override other configs

BoomTilesKey => Seq() RocketTilesKey => Seq() BankedL2Key => Module(new TLBroadcast) BootROMParams => BootROMParams() BuildTop => None class SmallBoomConfig extends Config( new WithTop ++ new WithBootROM ++ new freechips.rocketchip.subsystem.WithInclusiveCache ++ new boom.common.WithSmallBooms ++ new boom.common.WithNBoomCores(1) ++ new freechips.rocketchip.system.BaseConfig) chipyard-morning/ generators/ example/ src/main/scala/ BoomConfigs.scala rocket-chip/ boom/ sha3/ sims/ verilator/ tools/ chisel/ firrtl/ tests/ build.sbt

slide-18
SLIDE 18

Configuring a SoC with BOOM

19

  • Bottom-to-top (right-to-left) config hierarchy
  • Configs can read/override other configs

BoomTilesKey => Seq(BoomParams()) RocketTilesKey => Seq() BankedL2Key => Module(new TLBroadcast) BootROMParams => BootROMParams() BuildTop => None class SmallBoomConfig extends Config( new WithTop ++ new WithBootROM ++ new freechips.rocketchip.subsystem.WithInclusiveCache ++ new boom.common.WithSmallBooms ++ new boom.common.WithNBoomCores(1) ++ new freechips.rocketchip.system.BaseConfig) chipyard-morning/ generators/ example/ src/main/scala/ BoomConfigs.scala rocket-chip/ boom/ sha3/ sims/ verilator/ tools/ chisel/ firrtl/ tests/ build.sbt

slide-19
SLIDE 19

Configuring a SoC with BOOM

20

  • Bottom-to-top (right-to-left) config hierarchy
  • Configs can read/override other configs

BoomTilesKey => Seq(BoomParams(Small)) RocketTilesKey => Seq() BankedL2Key => Module(new TLBroadcast) BootROMParams => BootROMParams() BuildTop => None class SmallBoomConfig extends Config( new WithTop ++ new WithBootROM ++ new freechips.rocketchip.subsystem.WithInclusiveCache ++ new boom.common.WithSmallBooms ++ new boom.common.WithNBoomCores(1) ++ new freechips.rocketchip.system.BaseConfig) chipyard-morning/ generators/ example/ src/main/scala/ BoomConfigs.scala rocket-chip/ boom/ sha3/ sims/ verilator/ tools/ chisel/ firrtl/ tests/ build.sbt

slide-20
SLIDE 20

Configuring a SoC with BOOM

21

  • Bottom-to-top (right-to-left) config hierarchy
  • Configs can read/override other configs

BoomTilesKey => Seq(BoomParams(Small)) RocketTilesKey => Seq() BankedL2Key => Module(new InclusiveCache) BootROMParams => BootROMParams() BuildTop => None class SmallBoomConfig extends Config( new WithTop ++ new WithBootROM ++ new freechips.rocketchip.subsystem.WithInclusiveCache ++ new boom.common.WithSmallBooms ++ new boom.common.WithNBoomCores(1) ++ new freechips.rocketchip.system.BaseConfig) chipyard-morning/ generators/ example/ src/main/scala/ BoomConfigs.scala rocket-chip/ boom/ sha3/ sims/ verilator/ tools/ chisel/ firrtl/ tests/ build.sbt

slide-21
SLIDE 21

Configuring a SoC with BOOM

22

  • Bottom-to-top (right-to-left) config hierarchy
  • Configs can read/override other configs

BoomTilesKey => Seq(BoomParams(Small)) RocketTilesKey => Seq() BankedL2Key => Module(new InclusiveCache) BootROMParams => BootROMParams(“../bootrom.rv64.img”) BuildTop => None class SmallBoomConfig extends Config( new WithTop ++ new WithBootROM ++ new freechips.rocketchip.subsystem.WithInclusiveCache ++ new boom.common.WithSmallBooms ++ new boom.common.WithNBoomCores(1) ++ new freechips.rocketchip.system.BaseConfig) chipyard-morning/ generators/ example/ src/main/scala/ BoomConfigs.scala rocket-chip/ boom/ sha3/ sims/ verilator/ tools/ chisel/ firrtl/ tests/ build.sbt

slide-22
SLIDE 22

Configuring a SoC with BOOM

23

  • Bottom-to-top (right-to-left) config hierarchy
  • Configs can read/override other configs

BoomTilesKey => Seq(BoomParams(Small)) RocketTilesKey => Seq() BankedL2Key => Module(new InclusiveCache) BootROMParams => BootROMParams(“../bootrom.rv64.img”) BuildTop => Module(new Top) class SmallBoomConfig extends Config( new WithTop ++ new WithBootROM ++ new freechips.rocketchip.subsystem.WithInclusiveCache ++ new boom.common.WithSmallBooms ++ new boom.common.WithNBoomCores(1) ++ new freechips.rocketchip.system.BaseConfig) chipyard-morning/ generators/ example/ src/main/scala/ BoomConfigs.scala rocket-chip/ boom/ sha3/ sims/ verilator/ tools/ chisel/ firrtl/ tests/ build.sbt

slide-23
SLIDE 23

Interactive

24

# open heterogeneous configs file > vim HeteroConfigs.scala

chipyard-morning/ generators/ example/ src/main/scala/ HeteroConfigs.scala rocket-chip/ boom/ sha3/ sims/ verilator/ tools/ chisel/ firrtl/ tests/ build.sbt class TutorialSmallBoomConfig extends Config( new WithTop ++ new WithBootROM ++ new freechips.rocketchip.subsystem.WithInclusiveCache ++ new boom.common.WithSmallBooms ++ new boom.common.WithNBoomCores(1) ++ new freechips.rocketchip.system.BaseConfig)

Feel free to use another editor! These slides assume vim.

slide-24
SLIDE 24

Interactive

25

# open heterogeneous configs file > vim HeteroConfigs.scala

chipyard-morning/ generators/ example/ src/main/scala/ HeteroConfigs.scala rocket-chip/ boom/ sha3/ sims/ verilator/ tools/ chisel/ firrtl/ tests/ build.sbt class TutorialSmallBoomConfig extends Config( new WithTop ++ new WithBootROM ++ new freechips.rocketchip.subsystem.WithInclusiveCache ++ new boom.common.WithSmallBooms ++ new boom.common.WithNBoomCores(1) ++ new freechips.rocketchip.system.BaseConfig)

Modify

slide-25
SLIDE 25

Interactive

26

# open heterogeneous configs file > vim HeteroConfigs.scala

chipyard-morning/ generators/ example/ src/main/scala/ HeteroConfigs.scala rocket-chip/ boom/ sha3/ sims/ verilator/ tools/ chisel/ firrtl/ tests/ build.sbt class TutorialSmallBoomConfig extends Config( new WithTop ++ new WithBootROM ++ new freechips.rocketchip.subsystem.WithInclusiveCache ++ new boom.common.WithSmallBooms ++ new boom.common.WithNBoomCores(1) ++ new freechips.rocketchip.system.BaseConfig)

class SmallBoomAndRocketConfig extends Config( new WithTop ++ new WithBootROM ++ new freechips.rocketchip.subsystem.WithInclusiveCache ++ new boom.common.WithRenumberHarts ++ new boom.common.WithSmallBooms ++ new boom.common.WithNBoomCores(1) ++ new freechips.rocketchip.subsystem.WithNBigCores(1) ++ new freechips.rocketchip.system.BaseConfig)

Modify and add

VIM Reminders: use i to insert text use ESC to exit inserting text

slide-26
SLIDE 26

Interactive

27

# open heterogeneous configs file > vim HeteroConfigs.scala

chipyard-morning/ generators/ example/ src/main/scala/ HeteroConfigs.scala rocket-chip/ boom/ sha3/ sims/ verilator/ tools/ chisel/ firrtl/ tests/ build.sbt class TutorialSmallBoomConfig extends Config( new WithTop ++ new WithBootROM ++ new freechips.rocketchip.subsystem.WithInclusiveCache ++ new boom.common.WithSmallBooms ++ new boom.common.WithNBoomCores(1) ++ new freechips.rocketchip.system.BaseConfig)

class SmallBoomAndRocketConfig extends Config( new WithTop ++ new WithBootROM ++ new freechips.rocketchip.subsystem.WithInclusiveCache ++ new boom.common.WithRenumberHarts ++ new boom.common.WithSmallBooms ++ new boom.common.WithNBoomCores(1) ++ new freechips.rocketchip.subsystem.WithNBigCores(1) ++ new freechips.rocketchip.system.BaseConfig)

Modify and add

VIM Reminders: use :q to quit use :w to save use :wq to save and quit

slide-27
SLIDE 27

Interactive

28

# navigate to the verilator > cd ~/chipyard-morning/sims/verilator # start the verilator rtl simulator build > make CONFIG=SmallBoomAndRocketConfig –j16 # this will take a while!

chipyard-morning/ generators/ example/ rocket-chip/ boom/ sha3/ sims/ verilator/ tools/ chisel/ firrtl/ tests/ build.sbt

slide-28
SLIDE 28

Behind the Scenes: Config

29

chipyard-morning/ generators/ example/ src/main/scala/ HeteroConfigs.scala rocket-chip/ boom/ sha3/ sims/ verilator/ tools/ chisel/ firrtl/ tests/ build.sbt

class SmallBoomAndRocketConfig extends Config( new WithTop ++ new WithBootROM ++ new freechips.rocketchip.subsystem.WithInclusiveCache ++ new boom.common.WithRenumberHarts ++ new boom.common.WithSmallBooms ++ new boom.common.WithNBoomCores(1) ++ new freechips.rocketchip.subsystem.WithNBigCores(1) ++ new freechips.rocketchip.system.BaseConfig)

Adds a “Big” Rocket core to the SoC

slide-29
SLIDE 29

Behind the Scenes: Config

30

chipyard-morning/ generators/ example/ src/main/scala/ HeteroConfigs.scala rocket-chip/ boom/ sha3/ sims/ verilator/ tools/ chisel/ firrtl/ tests/ build.sbt

class SmallBoomAndRocketConfig extends Config( new WithTop ++ new WithBootROM ++ new freechips.rocketchip.subsystem.WithInclusiveCache ++ new boom.common.WithRenumberHarts ++ new boom.common.WithSmallBooms ++ new boom.common.WithNBoomCores(1) ++ new freechips.rocketchip.subsystem.WithNBigCores(1) ++ new freechips.rocketchip.system.BaseConfig)

Renumbers the core ID’s so that there are no duplicates Adds a “Big” Rocket core to the SoC

slide-30
SLIDE 30

Behind the Scenes: Make

  • `make` is the top-level build system to combine everything
  • Invokes the Scala Build Tool (sbt)
  • Used in Chisel, FIRRTL, and other Chipyard tools
  • Invokes all of the simulator builds (VCS and Verilator)
  • Automatically keeps track of file dependencies for you!
  • Powerful `make` commands
  • Can just `make verilog` for Verilog only
  • Can run pre-added or unique tests
  • `make CONFIG=<YOUR CONFIG> BINARY=<YOUR BINARY> run-binary`
  • `make CONFIG=<YOUR CONFIG> run-bmark-tests`
  • Keeps all outputs organized based of a unique name of the SoC
  • `<PROJECT>.<DUT MODULE>.<CONFIG>`
  • Ex. `example.TestHarness.SmallBoomAndRocketConfig`

31

slide-31
SLIDE 31

Building a SW Simulator

32

Rocket BOOM BootROM FIRRTL Verilog Simulator Config L2 Cache

  • 1. Configs parameterize Chisel

generators

  • 2. Chisel elaborates into

FIRRTL

  • 3. FIRRTL elaborates

into Verilog

  • 4. Verilator compiles Verilog

into simulator binary

slide-32
SLIDE 32

Building a SW Simulator

33

Rocket BOOM BootROM Config L2 Cache

  • 1. Configs parameterize Chisel

generators In this case `SmallBoomAndRocketConfig` creates a BOOM + Rocket SoC

slide-33
SLIDE 33

Building a SW Simulator

34

Rocket BOOM BootROM FIRRTL Config L2 Cache

  • 1. Configs parameterize Chisel

generators

  • 2. Chisel elaborates into

FIRRTL Emits a `*.fir` file

slide-34
SLIDE 34

Building a SW Simulator

35

Rocket BOOM BootROM FIRRTL Verilog Config L2 Cache

  • 1. Configs parameterize Chisel

generators

  • 2. Chisel elaborates into

FIRRTL

  • 3. FIRRTL elaborates

into Verilog Emits a `*.top.v` and `*.harness.v` file

slide-35
SLIDE 35

Why `*.top.v` and `*.harness.v`

36

TestHarness Top Tile 0 BOOM L1I$ L1D$

3-w BOOM

SysBus MemBus

BootROM L2 GPIOs

Tile 1 L1I$ L1D$ BOOM

Rocket SimBlockDevice SimAXIMem JTAG

slide-36
SLIDE 36

37

TestHarness Top Tile 0 L1I$ L1D$

3-w BOOM

SysBus MemBus

BootROM L2 GPIOs

Tile 1 L1I$ L1D$

Rocket SimBlockDevice SimAXIMem JTAG

Why `*.top.v` and `*.harness.v`

slide-37
SLIDE 37

38

Top Tile 0 L1I$ L1D$

3-w BOOM

SysBus MemBus

BootROM L2 GPIOs

Tile 1 L1I$ L1D$

Rocket JTAG

Pass to your VLSI flow!

Why `*.top.v` and `*.harness.v`

slide-38
SLIDE 38

Building a SW Simulator

39

Rocket BOOM BootROM FIRRTL Verilog Simulator Config L2 Cache

  • 1. Configs parameterize Chisel

generators

  • 2. Chisel elaborates into

FIRRTL

  • 3. FIRRTL elaborates

into Verilog

  • 4. Verilator compiles Verilog

into simulator binary Emits a `simulator-*` executable that represents the `SmallBoomAndRocketConfig`

slide-39
SLIDE 39

Where is my Verilog!

  • Once completed the build outputs are located here

40

> ls generated-src/*/ generated-src/*/ *.harness.v *.top.v *.dts *.json ...

chipyard-morning/ generators/ example/ rocket-chip/ boom/ sha3/ sims/ verilator/ generated-src/*/ tools/ chisel/ firrtl/ tests/ build.sbt

Verilog used for testing Main Verilog of the design Device Tree used for SW MMIO files (where and what is connected)

slide-40
SLIDE 40

Interactive

41

chipyard-morning/ generators/ example/ rocket-chip/ boom/ sha3/ sims/ verilator/ generated-src/*/ tools/ chisel/ firrtl/ tests/ build.sbt

# navigate to the output sources > cd generated-src/*/ > ls # take a look at the following files > less example.TestHarness.*.top.v > less example.TestHarness.*.harness.v

slide-41
SLIDE 41

How does the Verilog look?

42

chipyard-morning/ generators/ example/ rocket-chip/ boom/ sha3/ sims/ verilator/ generated-src/*/ tools/ chisel/ firrtl/ tests/ build.sbt

// Rocket Tile (includes a Rocket Core + L1$’s) module RocketTile( // @[:example.TestHarness.SmallBoomAndRocketConfig.fir@347233.2] input clock, // @[:example.TestHarness.SmallBoomAndRocketConfig.fir@347234.4] input reset, // @[:example.TestHarness.SmallBoomAndRocketConfig.fir@347235.4] . . . // BOOM Tile (includes a BOOM Core + L1$’s) module BoomTile( // @[:example.TestHarness.SmallBoomAndRocketConfig.fir@293433.2] input clock, // @[:example.TestHarness.SmallBoomAndRocketConfig.fir@293434.4] input reset, // @[:example.TestHarness.SmallBoomAndRocketConfig.fir@293435.4] . . . // Top-level DUT that includes the BOOM and Rocket Tiles module Top( // @[:example.TestHarness.SmallBoomAndRocketConfig.fir@384841.2] input clock, // @[:example.TestHarness.SmallBoomAndRocketConfig.fir@384842.4] input reset, // @[:example.TestHarness.SmallBoomAndRocketConfig.fir@384843.4]

example.TestHarness.SmallBoomAndRocketConfig.top.v

slide-42
SLIDE 42

Interactive

43

chipyard-morning/ generators/ example/ rocket-chip/ boom/ sha3/ sims/ verilator/ tools/ chisel/ firrtl/ tests/ build.sbt # navigate to the output sources > cd ../.. > ls # run a multi-threaded vector-vector add > make CONFIG=SmallBoomAndRocketConfig BINARY=mt-vvadd.riscv run-binary # take a look at the output file > less mt-vvadd.*.out

slide-43
SLIDE 43

Output file

44

using random seed 1570590524 This emulator compiled with JTAG Remote Bitbang client. To enable, use +jtag_rbb_enable=1. Listening on port 38117 C1: 0 [0] pc=[0000004c0c7cadf1] W[r 0=0000000000000080][0] R[r18=eb4b96208b54f69a] R[r28=eb4b96208b54f69a] inst=[0000c34d] c.beqz a4, pc + 162 C1: 1 [0] pc=[0000004c0c7cadf1] W[r 0=0000000000000080][0] R[r18=eb4b96208b54f69a] R[r28=eb4b96208b54f69a] inst=[0000c34d] c.beqz a4, pc + 162 C1: 2 [0] pc=[0000004c0c7cadf1] W[r 0=0000000000000080][0] R[r18=eb4b96208b54f69a] R[r28=eb4b96208b54f69a] inst=[0000c34d] c.beqz a4, pc + 162 C1: 3 [0] pc=[0000004c0c7cadf1] W[r 0=0000000000000080][0] R[r18=eb4b96208b54f69a] R[r28=eb4b96208b54f69a] inst=[0000c34d] c.beqz a4, pc + 162 C1: 4 [0] pc=[0000004c0c7cadf1] W[r 0=0000000000000080][0] R[r18=eb4b96208b54f69a] R[r28=eb4b96208b54f69a] inst=[0000c34d] c.beqz a4, pc + 162 C1: 5 [0] pc=[0000004c0c7cadf1] W[r 0=0000000000000080][0] R[r18=eb4b96208b54f69a] R[r28=eb4b96208b54f69a] inst=[0000c34d] c.beqz a4, pc + 162 . . . C1: 63210 [1] pc=[000000008000010e] W[r 0=0000000000000000][0] R[r10=0000000000000001] R[r11=0000000000000001] inst=[00b57063] bgeu a0, a1, pc + 0 C1: 63211 [0] pc=[000000008000010e] W[r 0=0000000000000000][0] R[r10=0000000000000001] R[r11=0000000000000001] inst=[00b57063] bgeu a0, a1, pc + 0 *** PASSED *** Completed after 107267 cycles

Commit log lines emitted from the Rocket Core Test passed

mt-vvadd.example.TestHarness.SmallBoomAndRocketConfig.top.v

slide-44
SLIDE 44

Case Study: Create and test a SHA3-accelerated SoC!

45

slide-45
SLIDE 45

Why do the case study?

  • “We are in a golden age of computer architecture”
  • Domain Specific Accelerators are extremely popular
  • Vector Accelerators
  • ML Accelerators
  • Training
  • Inference
  • Compression Accelerators
  • More!
  • Goal is to answer: “But how do I add my own accelerator?”

46

slide-46
SLIDE 46

SHA3 accelerator?

  • SHA3 accelerator is a pre-implemented Chisel accelerator
  • Implements the Secure Hash Algorithm 3 (SHA3)
  • Rough specification in 2012, released in late 2015
  • Uses variable length messages with a sponge function
  • Designed to be more efficient when implemented in HW!
  • Want to improve the hashes/sec and hashes/Watt

47

Let’s begin adding it!

slide-47
SLIDE 47

Interactive

48

chipyard-morning/ generators/ example/ rocket-chip/ boom/ sha3/ sims/ verilator/ tools/ chisel/ firrtl/ tests/ build.sbt

# add sha3 to chipyard > cd ~/chipyard-morning/generators > git submodule add https://github.com/ucb-bar/sha3.git # open the dependencies file > cd .. > vim build.sbt

slide-48
SLIDE 48

Interactive

49

//lazy val sha3 = (project in file("generators/sha3")) // .dependsOn(rocketchip, chisel_testers) // .settings(commonSettings) lazy val example = (project in file("generators/example")) .dependsOn(boom, hwacha, sifive_blocks, sifive_cache, utilities) .settings(commonSettings) chipyard-morning/ generators/ example/ rocket-chip/ boom/ sha3/ sims/ verilator/ tools/ chisel/ firrtl/ tests/ build.sbt

slide-49
SLIDE 49

Interactive

50

//lazy val sha3 = (project in file("generators/sha3")) // .dependsOn(rocketchip, chisel_testers) // .settings(commonSettings) . . . lazy val example = (project in file("generators/example")) .dependsOn(boom, hwacha, sifive_blocks, sifive_cache, utilities) .settings(commonSettings) lazy val sha3 = (project in file("generators/sha3")) .dependsOn(rocketchip, chisel_testers) .settings(commonSettings) . . . lazy val example = (project in file("generators/example")) .dependsOn(sha3, boom, hwacha, sifive_blocks, sifive_cache, utilities) .settings(commonSettings)

Uncomment

chipyard-morning/ generators/ example/ rocket-chip/ boom/ sha3/ sims/ verilator/ tools/ chisel/ firrtl/ tests/ build.sbt

Add sha3

slide-50
SLIDE 50

What happened?

  • There are three steps to add an accelerator to Chipyard
  • 1. Add the source code to `generators`
  • 2. Add the accelerator project to the build system in `build.sbt`
  • Need to tell the build system it exists
  • Need to add it to a top-level project (in our case `example`)
  • 3. Create a new configuration that adds it!

51

slide-51
SLIDE 51

What happened?

  • There are three steps to add an accelerator to Chipyard
  • 1. Add the source code to `generators`
  • 2. Add the accelerator project to the build system in `build.sbt`
  • Need to tell the build system it exists
  • Need to add it to a top-level project (in our case `example`)
  • 3. Create a new configuration that adds it!

52

slide-52
SLIDE 52

What happened?

  • There are three steps to add an accelerator to Chipyard
  • 1. Add the source code to `generators`
  • 2. Add the accelerator project to the build system in `build.sbt`
  • Need to tell the build system it exists
  • Need to add it to a top-level project (in our case `example`)
  • 3. Create a new configuration that adds it!

53

First a quick-background on RoCC accelerators

slide-53
SLIDE 53

RoCC Accelerators

  • RoCC: Rocket Chip Coprocessor
  • Execute custom RISC-V instructions

for a custom extension

  • Examples of RoCC accelerators
  • Hwacha vector accelerators
  • Memcpy accelerator
  • Machine-learning accelerators
  • Java GC accelerator

Tile BOOM/Rocket L1I$ L1D$ PTW TLBs Decoupled RoCC Accelerator (SHA3) L2 SystemBus Core Complex Peripherals

cmd req cmd resp

slide-54
SLIDE 54

chipyard-morning/ generators/ example/ rocket-chip/ sha3/ src/main/scala/ sha3.scala sims/ verilator/ tools/ chisel/ firrtl/ tests/ build.sbt

Understanding the SHA3 Accelerator

  • SHA3 is a minimal example of a RoCC-based

accelerator

  • Executes custom “sha3” instructions sent by the

Rocket or BOOM core

  • sha3.scala
  • Note the `WithSha3Accel` mixin, which plugs into

the Rocket Chip config system

  • `Sha3AccelImp` implements the Chisel-based

accelerator

55

slide-55
SLIDE 55

class WithSha3Accel extends Config ((site, here, up) => { case Sha3WidthP => 64 case Sha3Stages => 1 case Sha3FastMem => true case Sha3BufferSram => false case BuildRoCC => Seq( (p: Parameters) => { val sha3 = LazyModule.apply( new Sha3Accel(OpcodeSet.custom2)(p) ) sha3 } ) })

The SHA3 Accelerator

  • The SHA3 mixin

56

chipyard-morning/ generators/ example/ rocket-chip/ sha3/ src/main/scala/ sha3.scala sims/ verilator/ tools/ chisel/ firrtl/ tests/ build.sbt

Rocket Chip uses the “BuildRoCC” key to figure out which accelerator to build SHA3 Parameters

slide-56
SLIDE 56

The SHA3 Accelerator

  • RoCC Command Stream

57

Tile BOOM/Rocket SHA3: Decoupled RoCC Accelerator

cmd.valid cmd.bits.inst.funct[1:0] cmd.bits.rs1[63:0] cmd.bits.rs2[63:0] cmd.bits.inst.rd[4:0] busy cmd.ready mem.req.ready mem.req.addr[63:0] mem.req.cmd[2:0] mem.req.size[1:0] mem.req.valid

L1DCache

mem.req.data[63:0] mem.resp.valid mem.resp.data[63:0]

chipyard-morning/ generators/ example/ rocket-chip/ sha3/ src/main/scala/ sha3.scala sims/ verilator/ tools/ chisel/ firrtl/ tests/ build.sbt

slide-57
SLIDE 57

Interactive

58

# open rocket configs file > cd generators/example/src/main/scala > vim RocketConfigs.scala

chipyard-morning/ generators/ example/ src/main/scala/ RocketConfigs.scala rocket-chip/ boom/ sha3/ sims/ verilator/ tools/ chisel/ firrtl/ tests/ build.sbt class TutorialRocketConfig extends Config( new WithTop ++ new WithBootROM ++ new freechips.rocketchip.subsystem.WithInclusiveCache ++ new freechips.rocketchip.subsystem.WithNBigCores(1) ++ new freechips.rocketchip.system.BaseConfig)

slide-58
SLIDE 58

Interactive

59

# open rocket configs file > cd generators/example/src/main/scala > vim RocketConfigs.scala

chipyard-morning/ generators/ example/ src/main/scala/ RocketConfigs.scala rocket-chip/ boom/ sha3/ sims/ verilator/ tools/ chisel/ firrtl/ tests/ build.sbt class TutorialRocketConfig extends Config( new WithTop ++ new WithBootROM ++ new freechips.rocketchip.subsystem.WithInclusiveCache ++ new freechips.rocketchip.subsystem.WithNBigCores(1) ++ new freechips.rocketchip.system.BaseConfig)

Modify

slide-59
SLIDE 59

Interactive

60

# open rocket configs file > cd generators/example/src/main/scala > vim RocketConfigs.scala

chipyard-morning/ generators/ example/ src/main/scala/ RocketConfigs.scala rocket-chip/ boom/ sha3/ sims/ verilator/ tools/ chisel/ firrtl/ tests/ build.sbt class TutorialRocketConfig extends Config( new WithTop ++ new WithBootROM ++ new freechips.rocketchip.subsystem.WithInclusiveCache ++ new freechips.rocketchip.subsystem.WithNBigCores(1) ++ new freechips.rocketchip.system.BaseConfig) class Sha3RocketConfig extends Config( new WithTop ++ new WithBootROM ++ new freechips.rocketchip.subsystem.WithInclusiveCache ++ new sha3.WithSha3Accel ++ new freechips.rocketchip.subsystem.WithNBigCores(1) ++ new freechips.rocketchip.system.BaseConfig)

Modify and add

slide-60
SLIDE 60

Interactive

61

# navigate to the verilator > cd ~/chipyard-morning/sims/verilator # start the verilator rtl simulator build > make CONFIG=Sha3RocketConfig debug –j16 # this will take a while!

chipyard-morning/ generators/ example/ rocket-chip/ boom/ sha3/ sims/ verilator/ tools/ chisel/ firrtl/ tests/ build.sbt

slide-61
SLIDE 61

Reminder: Where is my Verilog!

  • Once completed the build outputs are located here

62

> ls generated-src/example.TestHarness.Sha3RocketConfig/ generated-src/example.TestHarness.Sha3RocketConfig/ *.harness.v *.top.v *.dts *.json ...

chipyard-morning/ generators/ example/ rocket-chip/ boom/ sha3/ sims/ verilator/ generated-src/*/ tools/ chisel/ firrtl/ tests/ build.sbt

Verilog used for testing Main Verilog of the design Device Tree used for SW MMIO files (where and what is connected)

slide-62
SLIDE 62

Coffee Break until 11am

Coming up…

Running customized software on the SHA3 accelerated SoC

63

https://fires.im/micro-2019-tutorial/

Slides here:

slide-63
SLIDE 63

Interactive

64

# re-enter the tmux session (only if you exited it) > tmux attach –t soc # navigate to firemarshal > cd ~/chipyard-morning/software/firemarshal # view sha3 accelerated program

> vim workloads/sha3/benchmarks/src/sha3-rocc.c

chipyard-morning/ generators/ example/ rocket-chip/ boom/ sha3/ sims/ verilator/ software/ firemarshal/ tools/ chisel/ firrtl/ tests/ build.sbt

slide-64
SLIDE 64

Accelerator Software

65

printf("Start basic test 1.\n"); // BASIC TEST 1 - 150 zero bytes start = rdcycle(); asm volatile ("fence"); // setup accelerator with addresses of input and output ROCC_INSTRUCTION_SS(2, &input, &output, 0); // Set length and compute hash ROCC_INSTRUCTION_S(2, sizeof(input), 1); asm volatile ("fence" ::: "memory"); end = rdcycle();

sha3-rocc.c

chipyard-morning/ generators/ example/ rocket-chip/ boom/ sha3/ sims/ verilator/ software/ firemarshal/ workloads/sha3/benchmarks/src/ sha3-rocc.c tools/ chisel/ firrtl/ tests/ build.sbt

slide-65
SLIDE 65

Accelerator Software

66

printf("Start basic test 1.\n"); // BASIC TEST 1 - 150 zero bytes start = rdcycle(); asm volatile ("fence"); // setup accelerator with addresses of input and output ROCC_INSTRUCTION_SS(2, &input, &output, 0); // Set length and compute hash ROCC_INSTRUCTION_S(2, sizeof(input), 1); asm volatile ("fence" ::: "memory"); end = rdcycle();

sha3-rocc.c

  • pcode

rd rs1 rs2 funct asm volatile ("custom2 x0, %[msg_addr], %[hash_addr], 0" :: [msg_addr] "r" (&input), [hash_addr] "r" (&output));

Expanded C macro

slide-66
SLIDE 66

Accelerator Software

67

printf("Start basic test 1.\n"); // BASIC TEST 1 - 150 zero bytes start = rdcycle(); asm volatile ("fence"); // setup accelerator with addresses of input and output ROCC_INSTRUCTION_SS(2, &input, &output, 0); // Set length and compute hash ROCC_INSTRUCTION_S(2, sizeof(input), 1); asm volatile ("fence" ::: "memory"); end = rdcycle();

sha3-rocc.c

  • pcode

rd rs1 rs2 funct asm volatile ("custom2 x0, %[msg_addr], %[hash_addr], 0" :: [msg_addr] "r" (&input), [hash_addr] "r" (&output));

Expanded C macro Setup and run the accelerator

slide-67
SLIDE 67

Understanding SHA3 Software

  • Send necessary information to the accelerator
  • Source(s), Destination, and what function to run
  • Accelerator returns
  • Busy… a.k.a. “Is the accelerator done?”

68

  • pcode

rd rs1 rs2 funct asm volatile ("custom2 x0, %[msg_addr], %[hash_addr], 0" :: [msg_addr] "r" (&input), [hash_addr] "r" (&output));

BOOM/Rocket

cmd.valid cmd.bits.inst.funct[1:0] cmd.bits.rs1[63:0] cmd.bits.rs2[63:0] cmd.bits.inst.rd[4:0] busy cmd.ready

SHA3 Ctrl

slide-68
SLIDE 68

Interactive

69

# view sw implementation of SHA3 program

> vim workloads/sha3/benchmarks/src/sha3-sw.c

chipyard-morning/ generators/ example/ rocket-chip/ boom/ sha3/ sims/ verilator/ software/ firemarshal/ tools/ chisel/ firrtl/ tests/ build.sbt

slide-69
SLIDE 69

Software SHA3

70

printf("Start basic test 1.\n"); // BASIC TEST 1 - 150 zero bytes start = rdcycle(); // run sw to compute the SHA3 hash sha3ONE(input, sizeof(input), output); end = rdcycle();

sha3-rocc.c

chipyard-morning/ generators/ example/ rocket-chip/ boom/ sha3/ sims/ verilator/ software/ firemarshal/ workloads/sha3/benchmarks/src/ sha3-sw.c tools/ chisel/ firrtl/ tests/ build.sbt

slide-70
SLIDE 70

Software SHA3

71

printf("Start basic test 1.\n"); // BASIC TEST 1 - 150 zero bytes start = rdcycle(); // run sw to compute the SHA3 hash sha3ONE(input, sizeof(input), output); end = rdcycle();

sha3-rocc.c

chipyard-morning/ generators/ example/ rocket-chip/ boom/ sha3/ sims/ verilator/ software/ firemarshal/ workloads/sha3/benchmarks/src/ sha3-sw.c tools/ chisel/ firrtl/ tests/ build.sbt

SW implementation

  • f SHA3
slide-71
SLIDE 71

Interactive

72

# make sure you are in firemarshal directory > cd ~/chipyard-morning/software/firemarshal # build both binaries > ./marshal build workloads/sha3-bare-*.json

chipyard-morning/ generators/ example/ rocket-chip/ boom/ sha3/ sims/ verilator/ software/ firemarshal/ tools/ chisel/ firrtl/ tests/ build.sbt

slide-72
SLIDE 72

Building SHA3 Software

  • Used the FireMarshal utility to build the binaries
  • Tool that takes in `.json` description of build and emits the `.riscv` binary
  • More in-depth view after lunch
  • What was done… built two binaries
  • `sha3-sw.riscv` - software version of SHA3 computation
  • `sha3-rocc.riscv` - sends SHA3 computation to the accelerator
  • Both binaries created in
  • `workloads/sha3/benchmarks/bare/sha3-*.riscv`

73

slide-73
SLIDE 73

Interactive

74

# navigate to the verilator directory > cd ~/chipyard-morning/sims/verilator # run accelerated program > make CONFIG=Sha3RocketConfig run-binary-debug BINARY=../../software/firemarshal/workloads/sha3/benc hmarks/bare/sha3-rocc.riscv # run non-accelerated program > make CONFIG=Sha3RocketConfig run-binary-debug BINARY=../../software/firemarshal/workloads/sha3/benc hmarks/bare/sha3-sw.riscv

chipyard-morning/ generators/ example/ rocket-chip/ boom/ sha3/ sims/ verilator/ software/ firemarshal/ tools/ chisel/ firrtl/ tests/ build.sbt

The non-accelerated version takes longer

slide-74
SLIDE 74

Outputs

  • You should see the following after both tests

75

# you just ran: “make CONFIG=… BINARY=… run-binary-debug” # this is the output: Start basic test 1.

  • utput[0]:221 ==? results[0]:221

. . .

  • utput[31]:238 ==? results[31]:238

Success! SHA execution took M cycles

Compare the cycle time of both runs. Which is faster?

chipyard-morning/ generators/ example/ rocket-chip/ boom/ sha3/ sims/ verilator/ software/ firemarshal/ tools/ chisel/ firrtl/ tests/ build.sbt

slide-75
SLIDE 75

All Outputs

  • Once completed the build outputs are located here

76

> ls verilator/ sha3-sw.*.out sha3-sw.*.vcd sha3-rocc.*.out sha3-rocc.*.vcd simulator-example-Sha3RocketConfig-debug . . .

Rocket Core Commit log

chipyard-morning/ generators/ example/ rocket-chip/ boom/ sha3/ sims/ verilator/ software/ firemarshal/ tools/ chisel/ firrtl/ tests/ build.sbt

Verilator RTL Simulator Waveform file (used with GTKWave, etc)

slide-76
SLIDE 76

Interactive

77

# exit the terminal CTRL-D

Delete the tmux session

slide-77
SLIDE 77

More Information

  • https://chipyard.readthedocs.io/en/latest
  • Talks about heterogeneous SoCs
  • Talks about adding Verilog IP
  • Talks about adding accelerators
  • . . .

78

Covers what we did in this talk and more!

slide-78
SLIDE 78

That’s it!

Coming up…

Using Hammer to Speed Up the VLSI Flow

79

slide-79
SLIDE 79

Backup Slides

80

slide-80
SLIDE 80

Exploring the Module Hierarchy

ex example. e.Sha3Rocket etConfig.(h (harnes ess/t /top). ).v:

  • Generated Verilog for Top and TestHarness

ex example. e.Sha3Rocket etConfig.core. e.config:

  • Configuration information for core

ex example. e.Sha3Rocket etConfig.dts:

  • Device tree string

ex example. e.Sha3Rocket etConfig.fir:

  • FIRRTL IR

ex example. e.Sha3Rocket etConfig.graphml:

  • Diplomacy graph

81

chipyard-morning/ generators/ example/ rocket-chip/ sha3/ sims/ verilator/ generated-src/ example.Sha3RocketConfig/ tools/ chisel/ firrtl/ tests/ build.sbt

slide-81
SLIDE 81

TileLink PWM Custom Module

  • Custom PWM module driving a top-

level IO pin

  • Changes to base SoC
  • Additional top-level pin for PWM out
  • Additional memory-mapped register for

configuring PWM

  • Additional top-level module to drive PWM

82

chipyard-morning/ generators/ example/ PWM.scala RocketConfigs.scala Top.scala rocket-chip/ sha3/ sims/ verilator/ tools/ chisel/ firrtl/ tests/ build.sbt

slide-82
SLIDE 82

Instantiating a PWM Top

PWMModule:

  • Module which instantiates the PWM

functionality PWMTL:

  • Module which creates a TileLink node, and

inherits functionality of the PWMModule HasPeripheryPWMTL:

  • Connects PWM TileLink node to

Diplomacy graph of the BaseSubsystem HasPeripheryPWMTLModuleImp

  • Adds additional IO pin for PWMOut, and

drives it with the PWMModule

83

chipyard-morning/ generators/ example/ PWM.scala RocketConfigs.scala Top.scala rocket-chip/ sha3/ sims/ verilator/ tools/ chisel/ firrtl/ tests/ build.sbt

slide-83
SLIDE 83

PWM Custom Components

TopWithPWMTL:

  • Adds the HasPeripheryPWMTL trait to

add the PWM node to the Top’s Diplomacy graph

  • Instantiates a TopWithPWMTLModule

TopWithPWMTLModule:

  • Adds the

HasPeripheryPWMTLModuleImp trait to get the additional PWM IO pin and PWM Module

84

chipyard-morning/ generators/ example/ PWM.scala RocketConfigs.scala Top.scala rocket-chip/ sha3/ sims/ verilator/ tools/ chisel/ firrtl/ tests/ build.sbt