FireMarshal: Software Workload Management Nathan Pemberton UC Berkeley nathanp@berkeley.edu
Tutorial Roadmap Custom SoC Configuration FireMarshal RTL Generators Bare-metal & RISC-V Multi-level Custom Accelerators Peripherals Linux Cores Caches Verilog Custom Workload RTL Build Process FIRRTL FIRRTL IR Verilog QEMU & Spike Transforms Software RTL Simulation FireSim FPGA-Accelerated Simulation Automated VLSI Flow Tech- Tool- VCS Verilator Simulation Debugging Networking Hammer plugins plugins
FireMarshal Goals • Consistent Environments Software Core Application Logic • Problem: Everyone working off slightly different versions of platform/OS/etc. User-space distros (e.g. fedora) Kernel • Re-Usable Workloads OS Kernel Bypass • Problem: Tribal knowledge and non- Drivers reproducible results Bootloader • No standard way to represent workloads API • No version control for integration • Decoupled Development Hardware I/O’s & • Easy integration from SW models (like API’s Platform spike or qemu) to real RTL (FireSim or μ-Architecture actual chips) Physical Design 3 3
FireMarshal Overview FireSim Workload Management Local Development Client QEMU launch bin fedora.json Outputs rootfs build test launch Spike Server rdma.json Reference bin FireSim install rootfs EC2 F1 • Run generated workloads locally on SW simulators • Generate workload from machine-readable • Install to FireSim to run FPGA-accelerated simulation description • Automatically test and post-process results • A collection of boot binaries and disk images that run together 4
Sha3 Example Workloads 5
FireMarshal Tutorial Outline Workloads: Provided For You: • Bare Metal Unit Tests • Sha3 functional model (Spike) • sha3-bare-sw • RoCC-Enabled Linux Kernel • sha3-bare-rocc • Linux-Based Unit Test • sha3-linux Everything defined in its • sha3-linux-test own repository: • Linux-Based Benchmark sha3-workload.git • sha3-linux-jtr • sha3-linux-jtr-crack 6
Example Workload: Sha3 Workload Directory $ cd ~/chipyard-afternoon/software/firemarshal $ ls workloads/ $ ls workloads/sha3/ 7
Example Workload: Sha3 Bare-Metal Unit Test Specifies any parent workload to inherit settings from (‘bare’ is a minimal workload that runs hard- sha3-bare-rocc.json coded RISCV binaries) { "name" : "sha3-bare-rocc", Script to run when building this "workdir" : "sha3", workload (build.sh cross-compiles "base" : "bare", the unit test) "host-init" : "build.sh", "bin" : ”benchmarks/bare/sha3-rocc.riscv", Hard-coded binary to use "spike" : "spike-local/bin/spike", (produced by build.sh) "spike-args" : "--extension=sha3" } Golden-model sw simulator to use when launching this workload 8
Example Workload: Sha3 Bare-Metal Unit Test DEMO $ cd ~/chipyard-afternoon/software/firemarshal $ ./marshal build workloads/sha3-bare-rocc.json $ ./marshal launch -s workloads/sha3-bare-rocc.json $ ./marshal test -s workloads/sha3-bare-rocc.json 9
Example Workload: SHA3 on Linux Basic Buildroot-based Linux sha3-linux.json distribution (provided by Marshal) { ”name” : “sha3-linux”, “base” : “br-base.json”, Run by Marshal at build time “workdir” : “sha3”, (cross-compiles the Linux “host-init” : ”build.sh”, benchmarks) “files” : [ [“bmarks/sha3-sw.rv”, “/root/sha3-sw”], Files to copy into the guest root [”bmarks/sha3-rocc.rv”,”/root/sha3-rocc”], filesystem (the pre-compiled ] , benchmarks in this case) “linux-src” : “riscv-linux”, “spike” : “spike-local/bin/spike”, “spike-args” : “--extension=sha3” Optional custom Linux source to } compile (needed in this case to enable rocc) 10
Example Workload: Linux-based Unit Test Inherit everything we did for the basic sha3 workload, no need to sha3-linux-test.json repeat ourselves. { ”name” : “sha3-linux-test”, “base” : “sha3-linux.json”, Run by the guest every time it “workdir” : “sha3”, boots. Target will shutdown after “command” : ”/root/sha3-rocc” running the command. “testing” : { “refDir” : “goldenOutput/” } Known-good output. Marshal will } compare the run output against this when you test the workload 12
Example Workload: Linux-based Unit Test DEMO $ cd ~/chipyard-afternoon/software/firemarshal $ ./marshal -dv test -s workloads/sha3-linux-test.json 13
Linux Build Internals: What’s in a binary? Contains platform drivers and a minimal busybox Initramfs environment. Linked directly into the kernel Upstream* Linux Kernel: Linux Kernel Compiled per-workload based on configuration *Has some temporary patches for rocket chip BBL Berkeley Boot Loader: Compiled into binary for now. Derived from the PK package 14
Linux Build Internals: Diskless Designs • Problem: Not every platform has a working disk device Initramfs (e.g. spike) • Solution: Compile the whole Linux Kernel rootfs into the binary image! • ‘./marshal –nodisk …’ BBL 15
Linux Build Internals: Diskless Designs • Problem: Not every platform has a working disk device (e.g. spike) Initramfs + rootfs • Solution: Compile the whole rootfs into the binary image! Linux Kernel • ‘./marshal –nodisk …’ BBL 16
Example Workload: Linux-based Benchmark – John the Ripper Inherit from sha3-linux again. sha3-linux-jtr.json Only need to specify that stuff once. { “name” : “sha3-linux-jtr”, “base” : “sha3-linux.json”, Run on host exactly once (cross- “workdir” : “sha3”, compiles benchmark). “host-init” : ”jtr/build.sh”, “overlay” : “jtr/overlay”, } John The Ripper must be installed to work correctly. The overlay allows us to specify a complex directory structure. 17
Example Workload: Linux-based Benchmark – John the Ripper $ cd ~/chipyard-afternoon/software/firemarshal $ ./marshal -d build workloads/sha3-linux-jtr.json $ ./marshal -d launch -s workloads/sha3-linux-jtr.json In the target: user: root password: firesim $ cd sha3 $ john --format=Raw-SHA3-256-rocc short.txt $ poweroff -f 18
Linux Build Internals: Inheriting Workloads FireMarshal-Provided • Marshal avoids repeating work by inheriting from parents buildroot • Inheritance Process (recursively) br-base.json • Build parent completely Your Workload • Copy parent rootfs sha3-linux.json • Apply child rules (e.g. overlays, guest-init, etc) • GNU Make style dependency sha3-linux-jtr.json checking • FireMarshal only rebuilds if sha3-linux-jtr-test.json parents are out of date 19
More Complex Use-Cases 20
Multi-Node Workloads (“jobs”) • Each job runs on a single job-example.json node in multi-node simulations. { ”name” : “job-example”, • Described the same as any “base” : “br-base.json”, workload “jobs” : [ • implicitly ‘base’d on the { “name” : “node0”, enclosing workload “command” : “ping –c 1 172.16.0.3”, }, • Can run one at a time in { “name” : “node1”, SW simulation. “command” : “ping –c 1 172.16.0.2”, • Must use FireSim to use the } network ] } 21
Native Initialization (“guest-init”) guest-init-example.json { ”name” : “guest-init-example”, • “guest-init” script is run once “base” : “fedora-base.json”, on the guest during build “guest-init” : “init.sh” • Run in Qemu } • Can access internet init.sh • Useful for installing packages #!/bin/bash and/or natively compiling yum install –y blas python3 … benchmarks cd cafe2_src/ make 22
Automatic Results Processing (“post-run-hook”) “outputs” specifies files to copy results-example.json { from guest image after a run ”name” : “results-example”, “base” : “mytest.json”, “outputs” : [“/root/res.csv”], “post-run-hook” executed on the “post-run-hook” : “results.py” host after every run } • Good for post-processing of more complex experiments results.py #!/usr/bin/env python from pathlib import Path Path to the results directory passed import csv to the script resultPath = Path( sys.argv[1] ) / ‘results-example’ / ‘res.csv’ Do anything you want with the processResult(resultPath ) results. For example, copy to a known location, or sanity check 23
Running Workloads on FireSim 24
FireMarshal Overview FireSim Install Local Development Client QEMU launch bin Outputs fedora.json rootfs build test launch Spike rdma.json Serve Reference bin r FireSim install rootfs EC2 F1 • After running install, you can use FireSim • Generates FireSim-native workload to launch the workload on the real RTL configuration from FireMarshal • Note: unlike functional simulation, FireSim makes a copy of the rootfs before running. 25
Installing Workloads to FireSim $ cd ~/chipyard-afternoon/software/firemarshal $ ./marshal install workloads/sha3*.json $ cd ~/chipyard-afternoon/sims/firesim/deploy/ $ cat workloads/sha3-linux.json 26
Recommend
More recommend
