10 Years of Xen and beyond Lars Kurth Xen Project Community - - PowerPoint PPT Presentation

▶
10 years of xen and beyond
SMART_READER_LITE
LIVE PREVIEW

10 Years of Xen and beyond Lars Kurth Xen Project Community - - PowerPoint PPT Presentation

Nov 07, 2023 521 likes •1.11k views

10 Years of Xen and beyond Lars Kurth Xen Project Community Manager lars.kurth@xen.org @lars_kurth FREENODE: lars_kurth Xen.org becomes XenProject.org Teams aka sub-projects Hypervisor XAPI ARM Hypervisor (for Servers as

slide-1
SLIDE 1

Lars Kurth Xen Project Community Manager lars.kurth@xen.org

10 Years of Xen and beyond …

@lars_kurth FREENODE: lars_kurth

slide-2
SLIDE 2
  • Teams aka sub-projects

– Hypervisor – XAPI – ARM Hypervisor (for Servers as well as Mobile Devices) – Mirage OS

  • Governance : mixture between Linux Kernel and Apache

– Consensus decision making – Sub-project life-cycle (aka incubator) – PMC style structure for team leadership

Xen.org becomes XenProject.org

slide-3
SLIDE 3

Xen contributor community is diversifying

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 2010 2011 2012 Citrix UPC SUSE Amazon University AMD GridCentric Individual NSA Intel Fujitsu iWeb Misc Oracle Spectralogic University of British Columbia

  • The number of “significant”

active vendors is increasing

  • New feature development driving

new participation

slide-4
SLIDE 4

Hypervisor Architecture

slide-5
SLIDE 5

Hypervisor Architectures

Type 1: Bare metal Hypervisor

A pure Hypervisor that runs directly on the hardware and hosts Guest OS’s.

Provides partition isolation + reliability, higher security

Host HW

Memory CPUs I/O

Hypervisor

Scheduler MMU Device Drivers/Models

VMn VM1 VM0

Guest OS and Apps

slide-6
SLIDE 6

Hypervisor Architectures

Type 1: Bare metal Hypervisor

A pure Hypervisor that runs directly on the hardware and hosts Guest OS’s.

Type 2: OS ‘Hosted’

A Hypervisor that runs within a Host OS and hosts Guest OS’s inside of it, using the host OS services to provide the virtual environment.

Provides partition isolation + reliability, higher security Low cost, no additional drivers Ease of use & installation

Host HW

Memory CPUs I/O

Host HW

Memory CPUs I/O

Hypervisor

Scheduler MMU Device Drivers/Models

VMn VM1 VM0

Guest OS and Apps

Host OS

Device Drivers Ring-0 VM Monitor “Kernel “

VMn VM1 VM0

Guest OS and Apps User Apps User-level VMM Device Models

slide-7
SLIDE 7

Xen: Type 1 with a Twist

Type 1: Bare metal Hypervisor

Host HW

Memory CPUs I/O

Hypervisor

Scheduler MMU Device Drivers/Models

VMn VM1 VM0

Guest OS and Apps

slide-8
SLIDE 8

Xen: Type 1 with a Twist

Type 1: Bare metal Hypervisor

Host HW

Memory CPUs I/O

Hypervisor

Scheduler MMU Device Drivers/Models

VMn VM1 VM0

Guest OS and Apps

Host HW

Memory CPUs I/O

Hypervisor VMn VM1 VM0

Guest OS and Apps

Xen Architecture

Scheduler MMU

slide-9
SLIDE 9

Xen: Type 1 with a Twist

Type 1: Bare metal Hypervisor

Host HW

Memory CPUs I/O

Hypervisor

Scheduler MMU Device Drivers/Models

VMn VM1 VM0

Guest OS and Apps

Host HW

Memory CPUs I/O

Hypervisor VMn VM1 VM0

Guest OS and Apps

Xen Architecture

Scheduler MMU

Control domain (dom0)

Drivers Device Models Linux & BSD

slide-10
SLIDE 10

Xen Project and Linux

  • Xen Hypervisor is not in the Linux kernel
  • BUT: everything Xen and Xen Guests need to run is!
  • Xen packages are in all Linux distros (except RHEL6)

– Install Dom0 Linux distro – Install Xen package(s) or meta package – Reboot – Config stuff: set up disks, peripherals, etc.

More info: wiki.xen.org/wiki/Category:Host_Install

slide-11
SLIDE 11

Basic Xen Concepts

11

Control domain (dom0) Host HW VMn VM1 VM0

Guest OS and Apps

Memory CPUs I/O

Console

  • Interface to the outside world

Control Domain aka Dom0

  • Dom0 kernel with drivers
  • Xen Management Toolstack

Guest Domains

  • Your apps

Driver/Stub/Service Domain(s)

  • A “driver, device model or control

service in a box”

  • De-privileged and isolated
  • Lifetime: start, stop, kill

Dom0 Kernel

Hypervisor

Scheduler MMU XSM

Trusted Computing Base

slide-12
SLIDE 12

Basic Xen Concepts

12

Control domain (dom0) Host HW VMn VM1 VM0

Guest OS and Apps

Console

Memory CPUs I/O

Dom0 Kernel Toolstack

Hypervisor

Scheduler MMU XSM

Console

  • Interface to the outside world

Control Domain aka Dom0

  • Dom0 kernel with drivers
  • Xen Management Toolstack

Guest Domains

  • Your apps

Driver/Stub/Service Domain(s)

  • A “driver, device model or control

service in a box”

  • De-privileged and isolated
  • Lifetime: start, stop, kill

Trusted Computing Base

slide-13
SLIDE 13

Basic Xen Concepts

13

Control domain (dom0) Host HW VMn VM1 VM0

Guest OS and Apps

Console

Memory CPUs I/O

One or more driver, stub or service domains

Dom0 Kernel Toolstack

Hypervisor

Scheduler MMU XSM

Console

  • Interface to the outside world

Control Domain aka Dom0

  • Dom0 kernel with drivers
  • Xen Management Toolstack

Guest Domains

  • Your apps

Driver/Stub/Service Domain(s)

  • A “driver, device model or control

service in a box”

  • De-privileged and isolated
  • Lifetime: start, stop, kill

Trusted Computing Base

slide-14
SLIDE 14

14

Xen Variants for Server & Cloud

Xen Hypervisor Hypervisor

slide-15
SLIDE 15

Single Host Basic Functions Multiple Hosts Additional Functionality

15

Xen Variants for Server & Cloud

Increased level of functionality and integration with other components

Default / XL (XM) Toolstack / Console Libvirt / VIRSH XAPI / XE Hypervisor

Single Host Additional Functionality

Xen Hypervisor

slide-16
SLIDE 16

Single Host Basic Functions Multiple Hosts Additional Functionality

Xen Variants for Server & Cloud

Increased level of functionality and integration with other components

Default / XL (XM) Toolstack / Console Libvirt / VIRSH XAPI / XE Hypervisor

Single Host Additional Functionality

Xen Hypervisor

slide-17
SLIDE 17

17

Xen Variants for Server & Cloud

Increased level of functionality and integration with other components

Default / XL (XM) Toolstack / Console Libvirt / VIRSH Products Oracle VM Huawei UVP Citrix XenServer Project XAPI / XE Xen Hypervisor

slide-18
SLIDE 18

18

Xen Variants for Server & Cloud

Increased level of functionality and integration with other components

Default / XL (XM) Toolstack / Console Libvirt / VIRSH Used by … Project XAPI / XE Products Oracle VM Huawei UVP Citrix XenServer Xen Hypervisor

slide-19
SLIDE 19

Xen : Types of Virtualization

slide-20
SLIDE 20

20

Xen Hypervisor Control domain (dom0) Host HW Guest VMn

Apps

Memory CPUs I/O

Technology:

  • Paravirtualization

Linux PV guests have limitations:

  • limited to a subset of set of virtual HW

Advantages

  • Fast
  • Works on any system

(even without virt extensions)

HW Drivers PV Back Ends PV Front Ends Guest OS Dom0 Kernel

PV Domains

slide-21
SLIDE 21

21

Xen Hypervisor Control domain (dom0) Host HW Guest VMn

Apps

Memory CPUs I/O

Technology:

  • Paravirtualization

Linux PV guests have limitations:

  • limited to a subset of virtual HW

Advantages

  • Fast
  • Works on any system

(even without virt extensions)

Driver Domains

  • Security
  • Isolation
  • Reliability and Robustness

HW Drivers PV Back Ends PV Front Ends

Driver Domain e.g.

  • Disk
  • Network

HW Driver PV Back End Dom0 Kernel*

*) Can be MiniOS

Guest OS Dom0 Kernel

PV Domains & Driver Domains

slide-22
SLIDE 22

22

Xen Hypervisor Dom0 Host HW Guest VMn

Technology:

  • Shows emulation using QEMU/Device

Model (SW Virtualization)

  • In other situation HW can be used

Disadvantages

  • Emulation slower than PV

(mainly I/O devices)

Advantages

  • No kernel support needed

Device Model

IO Emulation IO Event VMEXIT

Dom0 Kernel

HVM & Stub Domains

Memory CPUs I/O

slide-23
SLIDE 23

23

Xen Hypervisor Dom0 Host HW Guest VMn

Technology:

  • Shows emulation using QEMU/Device

Model (SW Virtualization)

  • In other situation HW can be used

Disadvantages

  • Emulation slower than PV

(mainly I/O devices)

Advantages

  • No kernel support needed

Stub Domains

  • Security
  • Isolation
  • Reliability and Robustness

Device Model

IO Emulation IO Event VMEXIT

Stubdomn

Device Model Mini OS

Guest VMn

IO Emulation IO Event VMEXIT

Dom0 Kernel

HVM & Stub Domains

Memory CPUs I/O

slide-24
SLIDE 24

The Virtualization Spectrum

Fully Virtualized (FV) VS VS VS VH FV with PV for disk & network P VS VS VH PVHVM P P VS VH PVH P P P VH Fully Paravirtualized (PV) P P P P VH Virtualized (HW) P Paravirtualized VS Virtualized (SW) HVM mode/domain PV mode/domain Xen 4.4

slide-25
SLIDE 25

The Virtualization Spectrum

Fully Virtualized (FV) VS VS VS VH FV with PV for disk & network P VS VS VH PVHVM P P VS VH PVH P P P VH Fully Paravirtualized (PV) P P P P Scope for improvement Poor performance Optimal performance HVM mode/domain Xen 4.4 PV mode/domain

slide-26
SLIDE 26

The Virtualization Spectrum

Fully Virtualized (FV) VS VS VS VH FV with PV for disk & network P VS VS VH PVHVM P P VS VH PVH P P P VH Fully Paravirtualized (PV) P P P P Scope for improvement Poor performance Optimal performance HVM mode/domain Xen 4.4 PV mode/domain

Important: Xen automatically picks the best

  • ption based on HW & OS capabilities and

available drivers. As a Xen user I chose a HVM or PV domain.

slide-27
SLIDE 27

XAPI, XCP and XCP-XAPI

slide-28
SLIDE 28

Single Host Basic Functions Multiple Hosts Additional Functionality

XAPI, XCP and XCP-XAPI : What is it?

Increased level of functionality and integration with other components

Default / XL (XM) Toolstack / Console Libvirt / VIRSH XAPI / XE Hypervisor

Single Host Additional Functionality

Xen Hypervisor

slide-29
SLIDE 29

XAPI : What do I get?

Multiple Hosts Additional Functionality

XAPI / XE Xen Hypervisor

  • VM lifecycle: live snapshots, checkpoint, migration
  • Storage XenMotion: Migrate VMs between hosts or pools

without shared storage (while the VM is running)

  • Resource pools: flexible storage and networking
  • Event tracking: progress, notification
  • Upgrade and patching capabilities
  • Real-time performance monitoring and alerting
  • Templates for Windows and Linux guests
  • Open vSwitch support built-in (default)

More info: wiki.xen.org/wiki/XCP_Release_Features

slide-30
SLIDE 30

XAPI : two variants!

Multiple Hosts Additional Functionality

XAPI / XE Xen Hypervisor XCP ISO (at v1.6) Xen 4.1.3 + XAPI CentOS 5.3 Kernel (v2.6.32.43) OVS 1.4.2 XCP-XAPI packages Debian Wheezy Ubuntu 12.04 LTS Others in progress …

slide-31
SLIDE 31

XAPI : Orchestration and UIs

Multiple Hosts Additional Functionality

XAPI / XE Xen Hypervisor

slide-32
SLIDE 32

Challenges for FOSS hypervisors

slide-33
SLIDE 33

“Security and QoS/Reliability are amongst the top 3 blockers for cloud adoption”

www.colt.net/cio-research

slide-34
SLIDE 34

System characteristics cloud users care about: “Robustness, Performance, Scalability & Security”

Results XCP User Survey 2012 – 90% of users quoted these as most important attributes

slide-35
SLIDE 35

Split Control Domain into Driver, Stub and Service Domains

– See: ”Breaking up is hard to do” @ Xen Papers – See: “Domain 0 Disaggregation for XCP and XenServer”

Used today by Qubes OS and Citrix XenClient XT Prototypes for XAPI

Disaggregation

See qubes-os.org Different windows run in different VMs

slide-36
SLIDE 36

More Security Increased serviceability and flexibility Better Robustness Better Performance Better Scalability

Benefits of Disaggregation

Ability to safely restart parts of the system (e.g. just 275ms outage from failed Ethernet driver)

slide-37
SLIDE 37

Next: XAPI Architecture Diagram Before and After Disaggregation

slide-38
SLIDE 38

CPU CPU RAM RAM NIC

(or SR- IOV VF)

NIC

(or SR- IOV VF)

NIC

(or SR- IOV VF)

NIC

(or SR- IOV VF)

RAID

Xen Dom0

Network drivers NFS/ iSCSI

drivers

Qemu xapi Local storage

drivers

NFS/ iSCSI

drivers

Network drivers Qemu

eth eth eth eth scsi

User VM User VM

NB

gntdev

NB NF BF NF BF

qemu qemu

xapi vswitch networkd tapdisk blktap3 storaged syslogd vswitch networkd tapdisk blktap3 storaged tapdisk blktap3 storaged

gntdev gntdev

Dom0

xenopsd libxl healthd Domain manager

Dom0 . . . . Xen

xapi

slide-39
SLIDE 39

CPU CPU RAM RAM NIC

(or SR- IOV VF)

NIC

(or SR- IOV VF)

NIC

(or SR- IOV VF)

NIC

(or SR- IOV VF)

RAID

Dom0

Network driver domain NFS/ iSCSI

driver domain

Qemu domain xapi domain

Logging domain

Local storage

driver domain

NFS/ iSCSI

driver domain

Network driver domain

User VM User VM

NB

gntdev

NB NF BF NF BF

dbus over v4v

qemu

xapi xenopsd libxl healthd Domain manager vswitch networkd tapdisk blktap3 storaged syslogd vswitch networkd tapdisk blktap3 storaged tapdisk blktap3 storaged

gntdev gntdev

eth eth eth eth scsi

Xen Xen D

  • m

dbus over v4v

. . .

slide-40
SLIDE 40

40

Xen Security Advantages

  • Even without Advanced Security Features

– Well-defined trusted computing base (much smaller than on type-2 HV) – Minimal services in hypervisor layer

  • Xen Security Modules (or XSM) and FLASK

– XSM is Xen equivalent of LSM – FLASK is Xen equivalent of SELinux – Developed, maintained and contributed to Xen by NSA – Compatible with SELinux (tools, architecture) – XSM object classes maps onto Xen features More info: http://www.slideshare.net/xen_com_mgr/ a-brief-tutorial-on-xens-advanced-security-features

slide-41
SLIDE 41

CPU CPU RAM RAM NIC

(or SR- IOV VF)

NIC

(or SR- IOV VF)

NIC

(or SR- IOV VF)

NIC

(or SR- IOV VF)

RAID

Xen Dom0

Network driver domain NFS/ iSCSI

driver domain

Qemu domain xapi domain

Logging domain

Local storage

driver domain

NFS/ iSCSI

driver domain

Network driver domain

eth eth eth eth scsi

User VM User VM

NB

gntdev

NB NF BF NF BF

qemu

xapi xenopsd libxl healthd Domain manager vswitch networkd tapdisk blktap3 storaged syslogd vswitch networkd tapdisk blktap3 storaged tapdisk blktap3 storaged

gntdev gntdev

FLASK policy restricting access

D

  • m

. . .

dbus over v4v dbus over v4v

Xen

slide-42
SLIDE 42

ARM Hypervisor

slide-43
SLIDE 43

Fully functional for ARM v7 & v8 ARM v7: Versatile Express, Arndale & Samsung Chromebook ARM v8: Fast Model

Xen 4.3 for ARM Servers

slide-44
SLIDE 44

ARM SOC

Xen + ARM = a perfect Match

ARM Architecture Features for Virtualization

Hypervisor mode : EL2 Kernel mode : EL1 User mode : EL0

GIC v2 GT 2 stage MMU I/O Device Tree describes …

Hypercall Interface HVC

slide-45
SLIDE 45

ARM SOC ARM Architecture Features for Virtualization

EL2 EL1 EL0

GIC v2 GT 2 stage MMU I/O Device Tree describes …

HVC

Xen + ARM = a perfect Match

Xen Hypervisor

slide-46
SLIDE 46

ARM SOC ARM Architecture Features for Virtualization

EL2 EL1 EL0

GIC v2 GT 2 stage MMU I/O Device Tree describes …

HVC

Xen + ARM = a perfect Match

Xen Hypervisor Any Xen Guest VM (including Dom0)

Kernel User Space

HVC

slide-47
SLIDE 47

ARM SOC ARM Architecture Features for Virtualization

EL2 EL1 EL0

GIC v2 GT 2 stage MMU I/O Device Tree describes …

HVC

Xen + ARM = a perfect Match

Xen Hypervisor Dom0

  • nly

Any Xen Guest VM (including Dom0)

Kernel User Space

I/O

PV back PV front

I/O

HVC

slide-48
SLIDE 48

One mode to rule them all

x86: PVHVM P P VS VH x86: PVH P P P VH ARM v7 & v8 P VH VH VH Scope for improvement Optimal performance HVM mode/domain PV mode/domain

slide-49
SLIDE 49

Code Size of x86 and ARM Hypervisors

X86 Hypervisor 100K -120K LOC Any x86 CPU ARM Hypervisor for mobile Devices 60K LOC ARM v5 – v7 (no virt extensions) (extra code for RT) ARM Hypervisor for Servers 17K LOC ARM v7+ (virt extensions)

slide-50
SLIDE 50

Mirage OS

slide-51
SLIDE 51

Application stacks only running on Xen APIs Works on any Xen based cloud or hosting service Examples

– ErlangOnXen.org : Erlang – HalVM : Haskell – Mirage OS : Ocaml

Benefits:

– Small footprint – Low startup latency – Extremely fast migration of VMs

Library Operating Systems

Xen Control domain (dom0) Host HW Guest VMn

Apps HW Drivers PV Back Ends Library OS embedded in Language run-time Dom0 Kernel

slide-52
SLIDE 52
  • Recently added to Xen Project incubator
  • In beta stage : first release on its way
  • Clean-slate protocols implementations, e.g.

– TCP/IP, DNS, SSH, Openflow (switch/controller), HTTP, XMPP, ... – New applications using next generation XAPI (disaggregated XAPI architecture)

Mirage OS

More info: http://www.slideshare.net/xen_com_mgr/ mirage-extreme-specialisation-of-virtual-appliances

slide-53
SLIDE 53

What’s Next?

slide-54
SLIDE 54
  • Code Freeze now
  • Xen ARM for Servers
  • Extend scope of Xen Security Modules
  • qxl Spice support for 3d acceleration
  • Updated and improved libvirt drivers for Xen
  • Lots of other stuff:

– scalability, performance, better NUMA support, …

Coming in Xen 4.3 (Q2 2013)

More info: blog.xen.org/index.php/2013/02/11/xen-4-3-mid-release-roadmap-update

slide-55
SLIDE 55
  • Establish a shared test infrastructure

– Most major contributors are duplicating effort – Mirage OS provides interesting opportiunities

  • Usability and better distro-integration

– Example: Xen + XAPI in CentOS 6.4

  • More focus on downstreams

– Examples: OpenStack and Xen Orchestra

  • Disaggregation in Xen toolstacks (e.g XAPI)
  • Better Libvirt and virt-manager integration

– Embed Xen more into the Linux eco-system and provide benefits for the wider Linux community

What’s next (and already happening)

slide-56
SLIDE 56
  • Document Days
  • Test Days
  • Mailing Lists and IRC
  • Find me and I can get you hooked up!

Getting Started with Xen Projects

Xen Hackathon, May 16-17, Dublin, Ireland @Google

slide-57
SLIDE 57

Thank You!

Slides available under CC-BY-SA 3.0 From www.slideshare.net/xen_com_mgr

@lars_kurth FREENODE: lars_kurth

  • News: blog.xenproject.org
  • Web: xenproject.org

– Help for IRC, Lists, … – Stackoverflow like Q&A

  • Wiki: wiki.xenproject.org
  • Presentations: slideshare.net/xen_com_mgr
  • Videos: vimeo.com/channels/xen