M i c r o k e r n e l s i n t h e E r a o f D - - PowerPoint PPT Presentation

M i c r

• k

e r n e l s i n t h e E r a

• f

D a t a

• C

e n t r i c C

• m

p u t i n g

Martjn Děcký martjn.decky@huawei.com

February 2018

Martjn Děcký, FOSDEM 2018, February 3rd 2018 Microkernels in the Era of Data-Centric Computjng 2

Who Am I

Passionate programmer and operatjng systems enthusiast

With a specifjc inclinatjon towards multjserver microkernels

HelenOS developer since 2004 Research Scientjst since 2006

Charles University (Prague), Distributed Systems Research Group

Senior Research Engineer since 2017

Huawei Technologies (Munich), Central Sofuware Instjtute

Martjn Děcký, FOSDEM 2018, February 3rd 2018 Microkernels in the Era of Data-Centric Computjng 3

M

• t

i v a t i

• n

3

Martjn Děcký, FOSDEM 2018, February 3rd 2018 Microkernels in the Era of Data-Centric Computjng 4

Memory Barrier

Martjn Děcký, FOSDEM 2018, February 3rd 2018 Microkernels in the Era of Data-Centric Computjng 5

LaMarca, Ladner (1996)

Quick Sort

O(n×log n) operatjons

Radix Sort

O(n) operatjons

4 8 16 32 64 128 256 512 1024 2048 4096 200 400 600 800 1000 1200 Quick Sort Radix Sort Thousands of items Instructjons / item

Martjn Děcký, FOSDEM 2018, February 3rd 2018 Microkernels in the Era of Data-Centric Computjng 6

LaMarca, Ladner (1996)

Quick Sort

O(n×log n) operatjons

Radix Sort

O(n) operatjons

4 8 16 32 64 128 256 512 1024 2048 4096 200 400 600 800 1000 1200 1400 1600 1800 2000 Quick Sort Radix Sort Thousands of items Clock cycles / item

Martjn Děcký, FOSDEM 2018, February 3rd 2018 Microkernels in the Era of Data-Centric Computjng 7

LaMarca, Ladner (1996)

Quick Sort

O(n×log n) operatjons

Radix Sort

O(n) operatjons

4 8 16 32 64 128 256 512 1024 2048 4096 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 Quick Sort Radix Sort Thousands of items Cache misses / item

Martjn Děcký, FOSDEM 2018, February 3rd 2018 Microkernels in the Era of Data-Centric Computjng 8

The Myth of RAM

Accessing a random memory locatjon requires O(1)

• peratjons

Accessing a random memory locatjon takes O(1) tjme units

Martjn Děcký, FOSDEM 2018, February 3rd 2018 Microkernels in the Era of Data-Centric Computjng 9

The Myth of RAM

Accessing a random memory locatjon requires O(1)

• peratjons

Accessing a random memory locatjon takes O(1) tjme units

Martjn Děcký, FOSDEM 2018, February 3rd 2018 Microkernels in the Era of Data-Centric Computjng 10

The Myth of RAM

Accessing a random memory locatjon requires O(1)

• peratjons

Accessing a random memory locatjon takes O(√n) tjme units

Martjn Děcký, FOSDEM 2018, February 3rd 2018 Microkernels in the Era of Data-Centric Computjng 11

B r e a k i n g t h e B a r r i e r

Martjn Děcký, FOSDEM 2018, February 3rd 2018 Microkernels in the Era of Data-Centric Computjng 12

Von Neumann Forever?

Data Control Status ALU Input peripheral Output peripheral RAM Controller

Martjn Děcký, FOSDEM 2018, February 3rd 2018 Microkernels in the Era of Data-Centric Computjng 13

Von Neumann Forever?

New emerging memory technologies

Bridging the gap between volatjle and non-volatjle memory

No longer necessary to keep the distjnctjon between RAM and storage (peripherals)

Single-level memory (universal memory) See also the talk by Liam Proven (The circuit less traveled), Janson, Sat 13:00

Many technologies in development

Magnetoresistjve random-access memory (MRAM) Racetrack memory Ferroelectric random-access memory (FRAM) Phase-change memory (PCM) Nano-RAM (Nanotube RAM)

Martjn Děcký, FOSDEM 2018, February 3rd 2018 Microkernels in the Era of Data-Centric Computjng 14

Less Radical Solutjon

Near-Data Processing

Moving the computatjon closer to the place where the data is

Not a completely new idea at all

Spatjal locality in general GPUs processing graphics data locally

Breaking the monopoly of CPU on data processing even more

CPUs are fast, but also power-hungry CPUs can only process the data already fetched from the memory/storage

The more data we avoid moving from the memory/storage to the CPU, the more effjciently the CPU runs

Martjn Děcký, FOSDEM 2018, February 3rd 2018 Microkernels in the Era of Data-Centric Computjng 15

Near-Data Processing

Benefjts

Decreased latency, increased throughput

Not necessarily on an unloaded system, but improvement under load

[1] Gu B., Yoon A. S., Bae D.-H., Jo I., Lee J., Yoon J., Kang J.-U., Kwon M., Yoon C., Cho S., Jeong J., Chang D.: Biscuit: A Framework for Near-Data Processing of Big Data Workloads, in Proceedings of 43rd Annual Internatjonal Symposium

• n Computer Architecture, ACM/IEEE,

2016

Martjn Děcký, FOSDEM 2018, February 3rd 2018 Microkernels in the Era of Data-Centric Computjng 16

Near-Data Processing

Benefjts (2)

Decreased energy consumptjon

[2] Kim S., Oh H., Park C., Cho S., Lee S.- W.: Fast, Energy Effjcient Scan inside Flash Memory SSDs, in Proceedings of 37th Internatjonal Conference on Very Large Data Bases (VLDB), VLDB Endowment, 2011

Martjn Děcký, FOSDEM 2018, February 3rd 2018 Microkernels in the Era of Data-Centric Computjng 17

In-Memory Near-Data Processing

Adding computatjonal capability to DRAM cells

Simple logical comparators/operators that could be computed in parallel

• n individual words

Filtering based on bitwise patuern Bitwise operatjons

Making use of the inherent parallelism

Avoiding moving unnecessary data out of DRAM Avoiding linear processing of independent words of the data in CPU

Martjn Děcký, FOSDEM 2018, February 3rd 2018 Microkernels in the Era of Data-Centric Computjng 18

Dynamic RAM

memory matrix row decoder

⁞

control logic Sense amps

.........

address data Y-gatjng

.........

Y RAS CAS WE

Martjn Děcký, FOSDEM 2018, February 3rd 2018 Microkernels in the Era of Data-Centric Computjng 19

Dynamic RAM with NDP

memory matrix row decoder

⁞

control logic Sense amps

.........

address data Filtering / Computjng

.........

• pcode

RAS CAS WE

Martjn Děcký, FOSDEM 2018, February 3rd 2018 Microkernels in the Era of Data-Centric Computjng 20

In-Storage Near-Data Processing

Adding computatjonal capability to SSD controllers

Again, making use of inherent parallelism of fmash memory But SSD controllers also contain powerful embedded cores

Flash Translatjon Layer, garbage collectjon, wear leveling Thus computatjon is not limited to simple bitwise fjltering and operatjons

Complex trade-ofgs between computjng on the primary CPU and

• ffmoading to the SSD controller

Martjn Děcký, FOSDEM 2018, February 3rd 2018 Microkernels in the Era of Data-Centric Computjng 21

Our Prototype

Based on OpenSSD

htup://openssd.io/

Open source (GPL) NVMe SSD controller

Hanyang University (Seoul), Embedded and Network Computjng Lab

FPGA design for Xilinx Xynq-7000

ONFI NAND fmash controller with ECC engine PCI-e NVMe host interface with scatuer-gather DMA engine

Controller fjrmware

ARMv7 Flash Translatjon Layer, page caching, greedy garbage collectjon

Martjn Děcký, FOSDEM 2018, February 3rd 2018 Microkernels in the Era of Data-Centric Computjng 22

Our Prototype (2)

NVMe NDP extensions

NDP module deployment

Statjc natjve code so far, moving to safe byte-code (eBPF)

NDP datasets

Safety boundaries for the NDP modules (for multjtenancy, etc.)

NDP Read / Write

Extensions of the standard NVMe Read / Write commands NDP module executed on each block, transforms/fjlters data

NDP Transform

Arbitrary data transformatjons (in-place copying, etc.) Flow-based computatjonal model

Martjn Děcký, FOSDEM 2018, February 3rd 2018 Microkernels in the Era of Data-Centric Computjng 23

Our Prototype (3)

Fast prototyping

QEMU model of the OpenSSD hardware (for running the OpenSSD fjrmware on ARMv7) Connected to a second host QEMU/KVM (as a regular PCI-e NVMe storage device)

Planned evaluatjon

Real-world applicatjon proof-of-concept

Custom storage engine for MySQL with operator push-down Ceph operator push-down Key-value store Generic fjle system acceleratjon

Stjll a very much work-in-progress

Preliminary results quite promising

Martjn Děcký, FOSDEM 2018, February 3rd 2018 Microkernels in the Era of Data-Centric Computjng 24

H

• w

M i c r

• k

e r n e l s F i t i n t

• t

h i s ?

Martjn Děcký, FOSDEM 2018, February 3rd 2018 Microkernels in the Era of Data-Centric Computjng 25

Future Vision

Not just near-data processing, but data-centric computjng

As opposed to CPU-centric computjng

Running the computatjon dynamically where it is the most effjcient Not necessarily moving the data to the central processing unit

The CPU is the orchestrator

Massively distributed systems

Within the box of your machine (desktop, laptop, smartphone) Outside your machine (edge cloud, fog, cloud, data center)

Massively heterogeneous systems

Difgerent ISAs Fully programmable, partjally programmable, fjxed-functjon

Martjn Děcký, FOSDEM 2018, February 3rd 2018 Microkernels in the Era of Data-Centric Computjng 26

Future Vision & Microkernels

fjle system driver server CPU microkernel applicatjon applicatjon applicatjon

privileged mode unprivileged mode memory mgmt scheduler IPC

naming server locatjon server device driver server device driver server device driver server fjle system driver server fjle system driver server device multjplexer fjle system multjplexer network stack security server ...

Martjn Děcký, FOSDEM 2018, February 3rd 2018 Microkernels in the Era of Data-Centric Computjng 27

Future Vision & Multj-Microkernels

fjle system driver server

Primary CPU

microkernel

distributed applicatjon distributed applicatjon distributed applicatjon

privileged mode unprivileged mode

memory mgmt scheduler IPC

naming server locatjon server device driver server device driver server device driver server fjle system driver server fjle system driver server device multjplexer fjle system multjplexer network stack security server ...

SSD CPU NIC CPU GPU

microkernel

memory mgmt scheduler IPC

microkernel

memory mgmt scheduler IPC

microkernel

memory mgmt scheduler IPC

... ... compositor

distributed applicatjon

Martjn Děcký, FOSDEM 2018, February 3rd 2018 Microkernels in the Era of Data-Centric Computjng 28

Microkernels for Data-Centric Computjng

Multjserver microkernels

Naturally support distributjon and heterogeneity

IPC is a well-defjned mechanism to communicate across isolatjon boundaries

Communicatjng between multjple (heterogeneous) computjng cores is just a difgerent transport Message passing as the least common denominator

– But also memory coherency emulatjon is possible

Multjserver designers are used to think in terms of fjne-grained components

Composing an applicatjon from multjple components Missing piece: Framework for automatjc workload migratjon

Martjn Děcký, FOSDEM 2018, February 3rd 2018 Microkernels in the Era of Data-Centric Computjng 29

Microkernels for Data-Centric Computjng (2)

Why not multjkernels?

Yes, why not! Many multjkernels (i.e. Barrelfjsh) are essentjally microkernels to the extreme

Each core running its independent microkernel No memory coherency assumed

Why not unikernels?

OK, if you insist A microkernel running a single fjxed payload (e.g. a virtual machine) could be considered a unikernel

Martjn Děcký, FOSDEM 2018, February 3rd 2018 Microkernels in the Era of Data-Centric Computjng 30

Road Forward

As usual, incremental steps

Start with just the basic workload ofg-loading to the SSD, NIC, etc. Gradually transform the run-tjme for the workload ofg-loading into a proto-microkernel Consider the dynamic workload migratjon framework Gradually move to a full-fmedged multj-microkernel

Martjn Děcký, FOSDEM 2018, February 3rd 2018 Microkernels in the Era of Data-Centric Computjng 31

Martjn Děcký, FOSDEM 2018, February 3rd 2018 Microkernels in the Era of Data-Centric Computjng 32

Summary

There are potentjal ways of abandoning the von Neumann architecture

Revolutjonary: Single-level (universal) memory Evolutjonary: Near-Data Processing

Near-Data Processing is benefjcial in terms of performance and energy consumptjon Data-centric computjng leads to massively distributed and heterogeneous systems

Multj-microkernels are a natural match for such architectures

Martjn Děcký, FOSDEM 2018, February 3rd 2018 Microkernels in the Era of Data-Centric Computjng 33

Acknowledgements

Huawei Technologies in-storage NDP team

Antonio Barbalace Martjn Decky Anthony Iliopoulos Javier Picorel Dmitry Voytjk Hangjun Zhao

References & Inspiratjon

Mark Silberstein (Technion) Biscuit framework (Samsung Electronics)

Martjn Děcký, FOSDEM 2018, February 3rd 2018 Microkernels in the Era of Data-Centric Computjng 34

Q & A

T h a n k Y

• u

!