M 3 : Taking Microkernels to the Next Level Nils Asmussen FOSDEM, - - PowerPoint PPT Presentation

▶
m 3 taking microkernels to the next level
SMART_READER_LITE
LIVE PREVIEW

M 3 : Taking Microkernels to the Next Level Nils Asmussen FOSDEM, - - PowerPoint PPT Presentation

Aug 15, 2023 123 likes •720 views

barkhauseninstitut.org M 3 : Taking Microkernels to the Next Level Nils Asmussen FOSDEM, 02/02/2020, Brussels About Me Nils Asmussen PhD last year at the OS chair of the TU Dresden Low-level system programming and microkernels Worked on

slide-1
SLIDE 1

barkhauseninstitut.org

M3: Taking Microkernels to the Next Level

Nils Asmussen FOSDEM, 02/02/2020, Brussels

slide-2
SLIDE 2

About Me

Nils Asmussen PhD last year at the OS chair of the TU Dresden Low-level system programming and microkernels Worked on several microkernel-based OSes in the past

Escape, own hobby OS (presented here in 2013): https://github.com/Nils-TUD/Escape NRE, userland for NOVA: https://github.com/TUD-OS/NRE M3, presented today: https://github.com/TUD-OS/M3

Since 2019 at the Barkhausen Institut

2 / 23

slide-3
SLIDE 3

Barkhausen Institut

Research institute in Dresden, founded end of 2017 Currently about 30 people Low-latency and secure IoT systems Focus on research and demonstrators

2 / 23

slide-4
SLIDE 4

Barkhausen Institut

Research institute in Dresden, founded end of 2017 Currently about 30 people Low-latency and secure IoT systems Focus on research and demonstrators Wireless RF Design Privacy Lab MPSoC OS

1 / 23

slide-5
SLIDE 5

Motivation

Microkernel-based systems have proven valuable for several objectives

Security Robustness Real time Flexibility

Recently, new challenges are coming from the hardware side

Heterogeneous systems Third-party components Security issues of complex general-purpose cores

2 / 23

slide-6
SLIDE 6

Heterogeneous Systems

Demanded by performance and energy requirements Big challenge for OSes: single shared kernel on all cores does no longer work OSes need to be prepared for processing elements with different feature sets

3 / 23

slide-7
SLIDE 7

Third-party Components

Market pressure forces us to integrate third-party components We should not trust these components Currently, ofen no isolation between them Bug in such a component can compromise whole system (see Broadcom incident)

4 / 23

slide-8
SLIDE 8

Security Issues of Complex General-purpose Cores

20 known atacks (and counting ...) Allow to leak private data, sometimes bypassing all security measures of the core Mitigations exist, but these are complex and costly These security holes have been lurking in CPUs for many years Should we still trust these complex cores to properly enforce the isolation between different sofware components?

5 / 23

slide-9
SLIDE 9

Microkernel-based System as Foundation

Microkernel Core Core Core Service Service Application Application

6 / 23

slide-10
SLIDE 10

Microkernel-based System as Foundation

Microkernel Core Core Core Service Service Application Application Management

6 / 23

slide-11
SLIDE 11

Microkernel-based System as Foundation

Microkernel Core Core Core Service Service Application Application Management Enforcement

6 / 23

slide-12
SLIDE 12

Microkernel-based System as Foundation

Microkernel Core Core Core Service Service Application Application Management Enforcement FPGA TPU GPU

6 / 23

slide-13
SLIDE 13

Outline 1 The New System Architecture 2 M3: The Operating System 3 What are the Benefits?

7 / 23

slide-14
SLIDE 14

Outline 1 The New System Architecture 2 M3: The Operating System 3 What are the Benefits?

8 / 23

slide-15
SLIDE 15

Hardware/Operating System Co-Design

9 / 23

slide-16
SLIDE 16

Hardware/Operating System Co-Design

Core Core Core GPU FPGA TPU

9 / 23

slide-17
SLIDE 17

Hardware/Operating System Co-Design

Core Core Core GPU FPGA TPU

9 / 23

slide-18
SLIDE 18

Hardware/Operating System Co-Design

Core Core Core GPU FPGA TPU TCU TCU TCU TCU TCU TCU Key ideas: TCU as new hardware component

9 / 23

slide-19
SLIDE 19

Hardware/Operating System Co-Design

Core Core Core GPU FPGA TPU TCU TCU TCU TCU TCU TCU Key ideas: TCU as new hardware component

9 / 23

slide-20
SLIDE 20

Hardware/Operating System Co-Design

Core Core Core GPU FPGA TPU TCU TCU TCU TCU TCU TCU Serv Kernel App App Serv App Key ideas: TCU as new hardware component Kernel on dedicated PE

9 / 23

slide-21
SLIDE 21

Hardware/Operating System Co-Design

Core Core Core GPU FPGA TPU TCU TCU TCU TCU TCU TCU Serv Kernel App App Serv App Manage Enforce Enforce Enforce Enforce Enforce Enforce Key ideas: TCU as new hardware component Kernel on dedicated PE Kernel manages, TCU enforces

9 / 23

slide-22
SLIDE 22

Hardware/Operating System Co-Design

Core Core Core GPU FPGA TPU TCU TCU TCU TCU TCU TCU Serv Kernel App App Serv App Takes µ-kernels to the next level: TCU as secure foundation

9 / 23

slide-23
SLIDE 23

Hardware/Operating System Co-Design

Core Core Core GPU FPGA TPU TCU TCU TCU TCU TCU TCU Serv Kernel App App Serv App Takes µ-kernels to the next level: TCU as secure foundation Heterogeneity: Uniform interface

9 / 23

slide-24
SLIDE 24

Hardware/Operating System Co-Design

Core Core Core GPU FPGA TPU TCU TCU TCU TCU TCU TCU Serv Kernel App App Serv App Takes µ-kernels to the next level: TCU as secure foundation Heterogeneity: Uniform interface Untrusted HW comp.: Protected by TCU

9 / 23

slide-25
SLIDE 25

Hardware/Operating System Co-Design

Core Core Core GPU FPGA TPU TCU TCU TCU TCU TCU TCU Serv Kernel App App Serv App Takes µ-kernels to the next level: TCU as secure foundation Heterogeneity: Uniform interface Untrusted HW comp.: Protected by TCU Side channels: Physical isolation

9 / 23

slide-26
SLIDE 26

Communication

Core Core Core GPU DRAM TPU Serv Kernel App App App TCU TCU TCU TCU TCU

M

TCU provides endpoints to: Access memory (contiguous range, byte granular)

10 / 23

slide-27
SLIDE 27

Communication

Core Core Core GPU DRAM TPU Serv Kernel App App App TCU TCU TCU TCU TCU

M S R S R

TCU provides endpoints to: Access memory (contiguous range, byte granular) Receive messages into a receive buffer Send messages to a receiving endpoint

10 / 23

slide-28
SLIDE 28

Communication

Core Core Core GPU DRAM TPU Serv Kernel App App App TCU TCU TCU TCU TCU

M S R S R

TCU provides endpoints to: Access memory (contiguous range, byte granular) Receive messages into a receive buffer Send messages to a receiving endpoint Replies for RPC

10 / 23

slide-29
SLIDE 29

Isolation

User tile TCU Kernel tile TCU User tile TCU User tile TCU User tile TCU Core Core Core GPU DRAM TPU Serv Kernel App App App TCU-based isolation: Additional protection layer

11 / 23

slide-30
SLIDE 30

Isolation

User tile TCU Kernel tile TCU User tile TCU User tile TCU User tile TCU Core Core Core GPU DRAM TPU Serv Kernel App App App TCU-based isolation: Additional protection layer

11 / 23

slide-31
SLIDE 31

Isolation

User tile TCU Kernel tile TCU User tile TCU User tile TCU User tile TCU Core Core Core GPU DRAM TPU Serv Kernel App App App

M S R S R

TCU-based isolation: Additional protection layer Only kernel tile can establish communication channels

11 / 23

slide-32
SLIDE 32

Isolation

User tile TCU Kernel tile TCU User tile TCU User tile TCU User tile TCU Core Core Core GPU DRAM TPU Serv Kernel App App App

M S R S R

TCU-based isolation: Additional protection layer Only kernel tile can establish communication channels User tiles can only use established channels

11 / 23

slide-33
SLIDE 33

Outline 1 The New System Architecture 2 M3: The Operating System 3 What are the Benefits?

12 / 23

slide-34
SLIDE 34

OS Design

M3: Microkernel-based system for het. manycores (or L4 ± 1) Implemented from scratch in Rust and C++ Drivers, filesystems, etc. implemented on user tiles Kernel manages permissions, using capabilities TCU enforces permissions (communication, memory access) Kernel is independent of other tiles Kernel M3FS pipes App App App

13 / 23

slide-35
SLIDE 35

M3 System Call

User tile PE TCU Kernel tile PE TCU Kernel App

R

14 / 23

slide-36
SLIDE 36

M3 System Call

User tile PE TCU Kernel tile PE TCU Kernel App

R S

14 / 23

slide-37
SLIDE 37

M3 System Call

User tile PE TCU Kernel tile PE TCU Kernel App

R S

14 / 23

slide-38
SLIDE 38

M3 System Call

User tile PE TCU Kernel tile PE TCU Kernel App

R S

14 / 23

slide-39
SLIDE 39

M3 System Call

User tile PE TCU Kernel tile PE TCU Kernel App

R S

14 / 23

slide-40
SLIDE 40

OS Service Access

Client Server DRAM TCU TCU

15 / 23

slide-41
SLIDE 41

OS Service Access

Client Server DRAM TCU TCU File Protocol: Used for: files, pipes, ...

15 / 23

slide-42
SLIDE 42

OS Service Access

Client Server DRAM TCU TCU File Protocol: Used for: files, pipes, ... Data in memory

15 / 23

slide-43
SLIDE 43

OS Service Access

Client Server DRAM TCU TCU S R req(in/out) resp(pos,len) File Protocol: Used for: files, pipes, ... Data in memory Msg channel between client and server

req(in) for next input piece req(out) for next output piece

15 / 23

slide-44
SLIDE 44

OS Service Access

Client Server DRAM TCU TCU S R req(in/out) resp(pos,len) M File Protocol: Used for: files, pipes, ... Data in memory Msg channel between client and server

req(in) for next input piece req(out) for next output piece

Server configures client’s memory EP

15 / 23

slide-45
SLIDE 45

OS Service Access

Client Server DRAM TCU TCU S R req(in/out) resp(pos,len) M File Protocol: Used for: files, pipes, ... Data in memory Msg channel between client and server

req(in) for next input piece req(out) for next output piece

Server configures client’s memory EP Client accesses data via TCU

15 / 23

slide-46
SLIDE 46

Outline 1 The New System Architecture 2 M3: The Operating System 3 What are the Benefits?

16 / 23

slide-47
SLIDE 47

Example System

TCU TCU TCU TCU TCU TCU Core Core Core Core Core 5G Kernel

17 / 23

slide-48
SLIDE 48

Example System

TCU TCU TCU TCU TCU TCU Core Core Core Core Core 5G Kernel Control Monitor FS Net

17 / 23

slide-49
SLIDE 49

Example System

TCU TCU TCU TCU TCU TCU Core Core Core Core Core 5G Kernel Control Monitor FS Net

17 / 23

slide-50
SLIDE 50

Example System – TCB

TCU TCU TCU TCU TCU TCU Core Core Core Core Core 5G Kernel Control Monitor FS Net TCU TCU TCU TCU TCU TCU Core Core Core Core Core 5G Kernel Control Monitor FS Net

18 / 23

slide-51
SLIDE 51

Example System – TCB

TCU TCU TCU TCU TCU TCU Core Core Core Core Core 5G Kernel Control Monitor FS Net TCU TCU TCU TCU TCU TCU Core Core Core Core Core 5G Kernel Control Monitor FS Net TCU TCU TCU TCU TCU TCU Core Core Core Core Core 5G Kernel Control Monitor FS Net Control

18 / 23

slide-52
SLIDE 52

Example System – TCB

TCU TCU TCU TCU TCU TCU Core Core Core Core Core 5G Kernel Control Monitor FS Net TCU TCU TCU TCU TCU TCU Core Core Core Core Core 5G Kernel Control Monitor FS Net TCU TCU TCU TCU TCU TCU Core Core Core Core Core 5G Kernel Control Monitor FS Net Monitor

18 / 23

slide-53
SLIDE 53

Example System – Untrusted Core

TCU TCU TCU TCU TCU TCU Core Core OoO Core Core 5G Kernel Data Control Monitor FS Data Net

19 / 23

slide-54
SLIDE 54

Example System – Untrusted Core

TCU TCU TCU TCU TCU TCU Core Core OoO Core Core 5G Kernel Data Control Monitor FS Data Net

19 / 23

slide-55
SLIDE 55

Example System – Sharing (WIP)

TCU TCU TCU TCU TCU Core Core Core Core 5G Kernel Data Control Mux FS Monitor Net

20 / 23

slide-56
SLIDE 56

Prototype Platforms

gem5 simulator

21 / 23

slide-57
SLIDE 57

Prototype Platforms

gem5 simulator FPGA

21 / 23

slide-58
SLIDE 58

Demo

22 / 23

slide-59
SLIDE 59

barkhauseninstitut.org

Summary

Microkernels are great! Their ideas can also be applied to hardware:

Trusted communication unit per tile Isolated sofware and hardware components on top

Has several additional benefits:

Allows to securely integrate untrusted third-party components Prevents (known) side-channel atacks by physical isolation Simplifies heterogeneous systems by uniform interface

M3 is available at https://github.com/TUD-OS/M3, gem5 extensions at https://github.com/TUD-OS/gem5-dtu