Unikernels as Processes Dan Williams, Ricardo Koller (IBM Research) - PowerPoint PPT Presentation

Unikernels as Processes Dan Williams, Ricardo Koller (IBM Research) Martin Lucina (robur.io/Center for the Cultivation of Technology) Nikhil Prakash (BITS Pilani)

What is a unikernel? • An application linked with library OS components • Run on virtual hardware (like) abstraction VM • Language-specific • MirageOS (OCaml) • IncludeOS (C++) • Legacy-oriented • Rumprun (NetBSD-based) • Can run nginx, redis, node.js, python,etc.. 2

Why unikernels? • Lightweight • Only what the application needs • Isolated VM • VM-isolation is the “gold standard” • Well suited for the cloud • Microservices • Serverless • NFV 3

Virtualization is a mixed bag • Good for isolation , but… • Tooling for VMs not designed for lightweight (e.g., lightVM) • How do you debug black-box VMs? • Poor VM performance due to vmexit s • Deployment issues on already-virtualized infrastructure 4

Why not run unikernels as processes? • Unikernels are a single process anyway! • Many benefits as a process Process • Better performance • Common tooling (gdb, perf, etc.) • ASLR • Memory sharing • Architecture independence • Isolation by limiting process interface to host • 98% reduction in accessible kernel functions 5

Outline • Introduction • Where does unikernel isolation come from? • Unikernels as processes • Isolation evaluation • Performance evaluation • Summary 6

Isolation: definitions and assumptions • Isolation : no cloud user can read/write state or modify its execution • Focus on software deficiencies in the host app • Code reachable through interface is a metric for attack surface Host Kernel • We trust HW isolation (page tables, etc.) • We do not consider covert channels, timing channels or resource starvation 7

Unikernel architecture • ukvm unikernel monitor process (e.g., ukvm) monitor • Userspace process • Uses Linux/KVM • Setup and loading • Exit handling KVM Linux I/O devices VT-x 8

Unikernel architecture • ukvm unikernel monitor process (e.g., ukvm) monitor Setup • Userspace process • Uses Linux/KVM • Setup and loading Set up 1 I/O fds • Exit handling KVM Linux I/O devices VT-x 9

Unikernel architecture Virtual CPU context • ukvm unikernel monitor process (e.g., ukvm) monitor Setup • Userspace process • Uses Linux/KVM Load 2 unikernel • Setup and loading Set up 1 I/O fds • Exit handling KVM Linux I/O devices VT-x 10

Unikernel architecture Virtual CPU context • ukvm unikernel monitor process (e.g., ukvm) monitor Setup Exit handling • Userspace process • Uses Linux/KVM Load 2 unikernel • Setup and loading Set up 1 I/O I/O fds • Exit handling KVM Linux I/O devices VT-x 11

Unikernel isolation comes from the interface Hypercall • 10 hypercalls walltime puts • 6 for I/O poll blkinfo • Network: packet level blkwrite • Storage: block level blkread netinfo • vs. >350 syscalls netwrite netread halt 12

Observations • Unikernels are not kernels! • No page table management after setup • No interrupt handlers: cooperative scheduling and poll • The ukvm monitor doesn’t “do” anything! • One-to-one mapping between hypercalls and system calls • Idea: maintain isolation by limiting syscalls available to process 13

Unikernel as process architecture • Tender : modified ukvm tender process unikernel monitor • Userspace process • Uses seccomp to restrict interface • Setup and loading Linux I/O devices 14

Unikernel as process architecture • Tender : modified ukvm tender process unikernel monitor • Userspace process Setup • Uses seccomp to restrict interface Set up 1 • Setup and loading I/O fds Linux I/O devices 15

Unikernel as process architecture • Tender : modified ukvm tender process unikernel monitor • Userspace process Setup • Uses seccomp to restrict interface Set up 1 • Setup and loading I/O fds Load unikernel 2 Linux I/O devices 16

Unikernel as process architecture • Tender : modified ukvm tender process unikernel monitor • Userspace process Setup • Uses seccomp to restrict interface Configure 3 seccomp Set up 1 • Setup and loading I/O fds Load unikernel 2 Linux I/O devices 17

Unikernel as process architecture • Tender : modified ukvm tender process unikernel monitor • Userspace process Setup Exit handling • Uses seccomp to restrict interface Configure 3 seccomp Set up 1 I/O • Setup and loading I/O fds Load unikernel 2 • “Exit” handling Linux I/O devices 18

Unikernel isolation comes from the interface Hypercall • 10 hypercalls walltime puts poll blkinfo • 6 for I/O blkwrite • Network: packet level blkread • Storage: block level netinfo netwrite netread • vs. >350 syscalls halt 19

Unikernel isolation comes from the interface Hypercall System Call Resource • Direct mapping between 10 walltime clock_gettime hypercalls and system stdout puts write call/resource pairs net_fd poll ppoll blkinfo • 6 for I/O blk_fd blkwrite pwrite64 • Network: packet level blk_fd blkread pread64 • Storage: block level netinfo net_fd netwrite write net_fd netread read • vs. >350 syscalls halt exit_group 20

Implementation: nabl nabla ! • Extended Solo5 unikernel ecosystem and ukvm • Prototype supports: • MirageOS • IncludeOS • Rumprun • https://github.com/solo5/solo5 21

Measuring isolation: common applications • Code reachable 1600 through interface is process functions accessed 1400 ukvm a metric for attack Unique kernel 1200 nabla surface 1000 • Used kernel ftrace 800 600 400 • Results: 200 0 • Processes: 5-6x more n n n r r e e g g o d d i i d n n i i e s s x x - - - • VMs: 2-3x more - g s e l a e e x r p t t g r e e s s 22

Measuring isolation: fuzz testing • Used kernel ftrace 700 accept • Used trinity nabla Unique kernel functions 600 system call fuzzer to block 500 try to access more of 30 400 the kernel 300 0 200 0 10 • Results: 100 • Nabla policy reduces 0 by 98% over a 0 50 100 150 200 250 300 “normal” process 23

Measuring performance: throughput • Applications include: 245 200% • Web servers ukvm Normalized throughput 180% nabla QEMU/KVM • Python benchmarks 160% • Redis 140% no I/O with I/O • etc. 120% 100% 80% py_tornado py_chameleon node_fib mirage_HTTP py_2to3 node_express nginx_large redis_get redis_set includeos_TCP nginx includeos_UDP • Results: • 101%-245% higher throughput than ukvm 24

Measuring performance: CPU utilization 120 • vmexit s have an effect on 100 CPU % 80 instructions per cycle (a) 60 40 20 100 0 VMexits/ms 80 • Experiment with MirageOS 60 (b) 40 web server 20 0 IPC (ins/cycle) 1.5 • Results: 1 (c) • 12% reduction in cpu 0.5 nabla utilization over ukvm ukvm 0 0 5000 10000 15000 20000 Requests/sec 25

Measuring performance: startup time • Startup time is important for serverless, NFV QEMU/KVM ukvm nabla process 30 30 30 Hello world 750 Hello world 20 20 20 500 500 • Results: 10 10 10 250 0 • Ukvm has 30-370% higher 0 0 0 0 200 200 200 latency than nabla 1500 1500 HTTP POST 150 150 150 HTTP Post 1000 1000 100 100 100 • Mostly due avoiding KVM 500 500 50 50 50 0 overheads 0 0 0 0 2 4 6 8 10 12 14 16 2 4 6 8 10 12 14 16 2 4 6 8 10 12 14 16 2 4 6 8 10 12 14 16 0 2 4 6 8 Number of cores 26

Summary and Next Steps • Unikernels should run as processes! • Maintain isolation via thin interface • Improve performance, etc. Process • Next steps: can unikernels as processes be used to improve container isolation? • Nabla containers • https://nabla-containers.github.io/ 27

28