Introduction to Metal FS and FPGA Programming Hands-On Robert Schmid - PowerPoint PPT Presentation

Introduction to Metal FS and FPGA Programming Hands-On Robert Schmid , Max Plauth, Sven Köhler, Lukas Wenzel and Andreas Polze Operating Systems and Middleware Group 19.06.2019

Interest in FPGAs is growing (again) Logic Blocks Programmable F ield- P rogrammable G ate A rray: programmable hardware circuit ■ Interconnect Algorithms are represented as a hardware configuration ■ Reasons for using FPGAs ■ Energy efficiency □ Parallel and pipelined data processing □ ‘Computing wires’ □ RAM/ALU/... Technology Advancements ■ Blocks IO Blocks ParProg 2019 New Generation of Interconnects (OpenCAPI, CCIX, ...) □ Metal FS High-Level Synthesis (HLS) languages 19.06.2019 □ Robert Schmid Chart 2 ‘Accelerators become first-class citizens in the system’ ■

First-class citizens? How should end-users interact with FPGAs? ■ Just like with any other executable program! ■ Analogy: Builtin UNIX tools (cat, grep, sed, awk, …) ■ Do one thing, and do it well! □ ‘Operator’ ParProg 2019 $ echo "Hello World" | fpga-encrypt –k key.bin > encrypted_file.bin Metal FS 19.06.2019 Redirect UNIX Robert Schmid Standard Pipe Output Chart 3 to File

Metal FS Goal: Improve the accessibility of FPGA accelerators using a file system ■ abstraction ParProg 2019 Metal FS 19.06.2019 Robert Schmid IBM POWER CAPI + SNAP Xilinx Vivado Foundations Chart 4

Operators are specified in Vivado HLS Configuration Operator Input Stream Output Stream void my_metal_operator(mtl_stream & in, mtl_stream & out, snapu64_t offset) { mtl_stream_element element; do { element = in.read(); element.data += offset; out.write(element); } while (!element.last); } ParProg 2019 Metal FS 19.06.2019 Robert Schmid Chart 5

Chaining Operators What happens here? ■ $ cat encrypted_file.bin | fpga-decrypt | fpga-uppercase HELLO WORLD In between FPGA processing steps, data should not be copied to the ■ CPU’s main memory (slow) In conclusion: ■ Multiple operators must be deployed on the FPGA at once □ ParProg 2019 Active subset and order of Operators should be configurable at □ Metal FS runtime 19.06.2019 Robert Schmid Chart 6

Metal FS Operator Pipelines Streaming Data from different types of memory ■ Composition of Pipelines by using C++ API ■ ■ AXI Stream Switch OperatorRegistry registry; Blowfish Blowfish auto encrypt = registry.operators().at("encrypt"); Change encrypt->setOption("key", keyBuffer); Case Encrypt Decrypt auto dataSource = create_data_source(inputBuffer); Stream Switch auto dataSink = create_data_sink(outputBuffer); Non- Host PipelineDefinition pipeline volatile No-op Memory ({ dataSource, encrypt, dataSink }); Memory pipeline.run(); ParProg 2019 Metal FS 19.06.2019 Robert Schmid Chart 7

Composition of Hardware Components: Block Design axi_datamover_mm2s interrupt_zero interrupt_concat S_AXIS_MM2S_CMD M_AXI_MM2S dout[0:0] In0[0:0] m_axi_mm2s_aclk M_AXIS_MM2S_STS In1[0:0] m_axi_mm2s_aresetn Constant M_AXIS_MM2S In2[0:0] m_axis_mm2s_cmdsts_aclk mm2s_err In3[0:0] m_axis_mm2s_cmdsts_aresetn dout[7:0] In4[0:0] data_selector AXI DataMover In5[0:0] In6[0:0] hls_streamgen S00_AXIS In7[0:0] S01_AXIS s_axi_ctrl S02_AXIS Concat out_r M00_AXIS ap_clk S_AXI_CTRL interrupt M01_AXIS ap_rst_n aclk M02_AXIS axi_streamsink aresetn Hls_streamgen (Pre-Production) s_axi_ctrl_aclk data s_axi_ctrl_aresetn ap_clk metal_switch ap_rst_n AXI4-Stream Switch axi_perf_mon_0 Hls_streamsink (Pre-Production) S00_AXIS S01_AXIS S_AXI S_AXI_CTRL M00_AXIS SLOT_0_AXIS axi_metal_cpc axi_metal_ctrl_crossbar aclk M01_AXIS SLOT_1_AXIS aresetn axi_datamover_s2mm s_axi_aclk S_AXI M00_AXI s_axi_ctrl_aclk s_axi_aresetn dm_smartconnect ap_clk aclk M_AXI S00_AXI M01_AXI s_axi_ctrl_aresetn S_AXIS_S2MM slot_0_axis_aclk ap_rst_n aresetn aclk M02_AXI S_AXIS_S2MM_CMD slot_0_axis_aresetn interrupt M_AXI_S2MM S00_AXI AXI4-Stream Switch aresetn M03_AXI m_axi_s2mm_aclk slot_1_axis_aclk M_AXIS_S2MM_STS S01_AXI AXI Protocol Converter M04_AXI m_axi_s2mm_aresetn M00_AXI axi_host_mem_crossbar slot_1_axis_aresetn s2mm_err aclk m_axis_s2mm_cmdsts_awclk capture_event aresetn AXI Crossbar op_colorfilter m_axis_s2mm_cmdsts_aresetn S00_AXI reset_event AXI SmartConnect S01_AXI s_axi_control core_aclk AXI DataMover M00_AXI m_axi_host_mem aclk axis_input axis_output core_aresetn aresetn ap_clk interrupt snap_action AXI Performance Monitor ap_rst_n AXI Crossbar s_axi_ctrl_reg Hls_operator_colorfilter (Pre-Production) mm2s_sts s_axi_ctrl_reg m_axi_host_mem s2mm_sts one mm2s_cmd_V_V interrupt_reg_V_V s2mm_cmd_V_V dout[0:0] interrupt_reg_V_V_TVALID m_axi_metal_ctrl_V interrupt_reg_V_V_TDATA[7:0] interrupt interrupt Constant ap_clk ap_rst_n Hls_action (Pre-Production) ParProg 2019 Metal FS 19.06.2019 Robert Schmid Chart 8

Metal FS: Architecture Overview CPU (‘Host’) FPGA AXI Stream C++ API Operator Pipelines Operator Pipelines Operator Pipelines Switch ParProg 2019 Metal FS 19.06.2019 Robert Schmid IBM POWER CAPI + SNAP Xilinx Vivado Foundations Chart 9

Metal FS Hybrid Filesystem Leverage the NVMe storage on the Nallatech N250S FPGA card ■ One use case for Operator Pipelines ■ File System Metadata is maintained in an LMDB ■ Key-Value Store on the host inodes, directory entries, free extents □ Block Mapper on the FPGA translates file offsets to physical addresses ■ using extent lists ParProg 2019 Metal FS All file accesses are implemented as Operator Pipelines (Read -> Write) ■ 19.06.2019 Robert Schmid Data transformations can be transparently added (e.g. encryption) □ Chart 10

Metal FS: Architecture Overview CPU (‘Host’) FPGA Filesystem Data Sources Metadata Block Mapper Hybrid File System and Sinks Store AXI Stream C++ API Operator Pipelines Operator Pipelines Operator Pipelines Switch ParProg 2019 Metal FS 19.06.2019 Robert Schmid IBM POWER CAPI + SNAP Xilinx Vivado Foundations Chart 11

Metal FS FUSE Filesystem Users can mount Metal FS as a Linux file system ■ Implemented in user space ■ Example: ■ $ cp ~/orders.tbl /metal_fs/files/ ParProg 2019 Metal FS 19.06.2019 Robert Schmid Chart 12

Metal FS Symbolic Executables $ echo "Hello World" \ | /metal_fs/operators/change_case \ | /metal_fs/operators/encrypt \ | /metal_fs/operators/decrypt File System Driver & Pipeline Orchestrator Process ParProg 2019 change_case encrypt decrypt Metal FS 19.06.2019 Robert Schmid Message flow via UNIX Socket Data flow via Memory-Mapped Files Chart 13

Metal FS: Architecture Overview CPU (‘Host’) FPGA Linux AXI Symbolic User Interface & Filesystem Performance Instrumentation Executables Driver Monitor Filesystem Data Sources Metadata Block Mapper Hybrid File System and Sinks Store AXI Stream C++ API Operator Pipelines Operator Pipelines Operator Pipelines Switch ParProg 2019 Metal FS 19.06.2019 Robert Schmid IBM POWER CAPI + SNAP Xilinx Vivado Foundations Chart 14

Demo

Demo Screencast ParProg 2019 Metal FS 19.06.2019 Robert Schmid Chart 16

Hands-On: Prepare the Metal FS Simulation Steps: ■ 1. git clone https://github.com/rs22/metalfs-workshop Start the development container by using the start script: 2. start_linux , start_osx , start_win.bat – Build a simulation image 3. – make model ParProg 2019 Metal FS 19.06.2019 Robert Schmid Chart 17

Anatomy of an Operator in Vivado HLS HLS translates the mtl_stream references into AXI Stream interfaces ■ We require the keep and last signals which are optional channels in the ■ AXI Stream protocol struct mtl_stream_element { ap_uint<64> data; ap_uint<8> keep; ap_uint<1> last; }; void my_operator(mtl_stream &in, mtl_stream &out) { mtl_stream_element element; ParProg 2019 do { Metal FS element = in.read(); 19.06.2019 out.write(element); Robert Schmid } while (!element.last); Chart 18 }

Programming with HLS: Arbitrary-Precision integers HLS offers the ap_uint types for integers with arbitrary bit precision ■ snapu{8, 16, 32, 64}_t are typedefs for ap_uint<> ■ Access bit ranges of an ap_uint like this (similar to VHDL): ■ snapu16_t my_integer; snapu8_t high_byte = my_integer(15, 8); Concatenate integers: ■ ParProg 2019 Metal FS snapu8_t high_byte = 0xFF; 19.06.2019 snapu8_t low_byte = 0x0A; Robert Schmid snapu16_t both_bytes = (high_byte, low_byte); Chart 19

Hands-On: Run the Metal FS Simulation Steps: ■ Build a new model 1. – make model Start the simulation 2. – make sim In the simulation window: 3. – snap_maint – metal_fs /mnt Start a second shell in the container using the start script 4. ParProg 2019 – cat src/hls_operator_colorfilter/apples_simulation.bmp \ Metal FS | /mnt/operators/colorfilter \ 19.06.2019 > out.bmp Robert Schmid Chart 20