Virtual Machine Security CSE443 - Spring 2012 Introduction to - - PowerPoint PPT Presentation

▶
virtual machine security
SMART_READER_LITE
LIVE PREVIEW

Virtual Machine Security CSE443 - Spring 2012 Introduction to - - PowerPoint PPT Presentation

Dec 26, 2022 199 likes •490 views

Virtual Machine Security CSE443 - Spring 2012 Introduction to Computer and Network Security Professor Jaeger www.cse.psu.edu/~tjaeger/cse443-s12/ CSE443 Introduction to Computer (and Network) Security - Spring 2012 - Professor Jaeger 1

slide-1
SLIDE 1

CSE443 Introduction to Computer (and Network) Security - Spring 2012 - Professor Jaeger

Virtual Machine Security

CSE443 - Spring 2012 Introduction to Computer and Network Security Professor Jaeger

www.cse.psu.edu/~tjaeger/cse443-s12/

1

slide-2
SLIDE 2

CSE443 Introduction to Computer (and Network) Security - Spring 2012 - Professor Jaeger

Operating System Quandary

  • Q: What is the primary goal of system security?
  • OS enables multiple users/programs to share resources on

a physical device

  • Q: What happens when we try to enforce Mandatory

Access Control policies on UNIX systems

  • Think SELinux policies
  • What can we to do to simplify?

2

slide-3
SLIDE 3

CSE443 Introduction to Computer (and Network) Security - Spring 2012 - Professor Jaeger

Virtual Machines

  • Instead of using system

software to enable sharing, use system software to enable isolation

  • Virtualization
  • “a technique for hiding the physical

characteristics of computing resources from the way in which

  • thers systems, applications, and end

users interact with those resources”

  • Virtual Machines
  • Single physical resource can

appear as multiple logical resources

3

slide-4
SLIDE 4

CSE443 Introduction to Computer (and Network) Security - Spring 2012 - Professor Jaeger

Virtual Machine Architectures

  • Full system simulation
  • CPU can be simulated
  • Paravirtualization (Xen)
  • VM has a special API
  • Requires OS changes
  • Native virtualization (VMWare)
  • Simulate enough HW to run OS
  • OS is for same CPU
  • Application virtualization (JVM)
  • Application API

4

slide-5
SLIDE 5

CSE443 Introduction to Computer (and Network) Security - Spring 2012 - Professor Jaeger

Virtual Machine Types

  • Type I
  • Lowest layer of software is VMM
  • E.g., Xen, VAX VMM, etc.
  • Type II
  • Runs on a host operating system
  • E.g., VMWare, JVM, etc.
  • Q: What are the trust model issues with Type II

compared to Type I?

5

slide-6
SLIDE 6

CSE443 Introduction to Computer (and Network) Security - Spring 2012 - Professor Jaeger

Virtual Machine Types

6

Hardware! Host OS! VMM!

Guest OS 1! Guest OS 2! App! App!

Hardware!

Host OS! VMM!

Guest OS 1! Guest OS 2! App! App!

Hardware! VMM!

Guest OS 1! Guest OS 2! App! App!

Type 2 VMM! Type 1 VMM! Hybrid VMM!

JVM! CLR! VMware Workstation! MS Virtual Server! KVM! VMware ESX! Xen! MS Hyper-V!

slide-7
SLIDE 7

CSE443 Introduction to Computer (and Network) Security - Spring 2012 - Professor Jaeger

VM Security

  • Isolation of VM computing
  • Like a separate machine

7

VM VM Virtual Machine Monitor

Physical Device Controls

Guest OS Guest OS

Partitioned Resources Device Requests

slide-8
SLIDE 8

CSE443 Introduction to Computer (and Network) Security - Spring 2012 - Professor Jaeger

VAX VMM System

8

  • First system design to examine virtualization in the

context of information flow security!

  • Virtualization mechanisms necessary to implement a reference

validation mechanism that satisfies the reference monitor concept!

  • Assure system design and implementation to the highest level

– A1 level per the Orange Book!

  • Control all system information flows according to MLS and

Biba integrity policies (modulo exceptions in “privileges”)!

  • Also, covert channel countermeasures were produced,

approximating noninterference!

  • System was piloted, but not released commercially!
slide-9
SLIDE 9

CSE443 Introduction to Computer (and Network) Security - Spring 2012 - Professor Jaeger

VAX VMM System

9

  • Key design tasks of secure VMM!
  • Virtualize processor!
  • All security-sensitive instructions must be mediated by VMM!
  • VMM protection ring!
  • VMM must be deployed in a more privileged protection ring than

the VMs!

  • I/O emulation!
  • Privileged I/O tasks must be executed in VMM or trusted VM!
  • Self-virtualizable!
  • OS must not detect when running on a VMM (or VMMs)!
slide-10
SLIDE 10

CSE443 Introduction to Computer (and Network) Security - Spring 2012 - Professor Jaeger

Virtualizing Instructions

10

  • Security-Sensitive Instructions!
  • Instructions that read or modify privileged system state!
  • Privileged Instructions!
  • Instructions that cause a trap when executed in a non-

privileged ring!

  • All security-sensitive instructions must be privileged

to enable the VMM to manage privileged system state (rather than individual VMs)!

  • This requirement was not met by VAX hardware nor

x86 originally!

slide-11
SLIDE 11

CSE443 Introduction to Computer (and Network) Security - Spring 2012 - Professor Jaeger

I/O Emulation

11

  • Access to devices is expected by each operating

system, but this access is security-sensitive!

  • Thus, devices are virtualized!
  • Access to devices must be directed to the party with

physical device access!

  • Memory-mapped I/O uses unprivileged instructions!
  • VAX VMM adds a layer of indirection!
  • I/O interface that causes a trap!
  • OS must be modified to use that interface

(paravirtualize)!

slide-12
SLIDE 12

CSE443 Introduction to Computer (and Network) Security - Spring 2012 - Professor Jaeger

Other Issues

12

  • Driver management!
  • In VAX VMM, all drivers were in the VMM kernel!
  • This was for assurance, but added code to VMM!
  • Drivers are outside the VMM in most systems!
  • DMA!
  • Devices can use this mechanism to write to physical

memory, but under guidance of untrusted VMs!

  • VAX VMM trusted drivers, but not practical today!
  • Performance – E.g., page table lookups!
slide-13
SLIDE 13

CSE443 Introduction to Computer (and Network) Security - Spring 2012 - Professor Jaeger

VAX VMM System

13

Ultrix OS VMS OS

VMM Security Kernel

VMS OS Applications (Top Secret) Applications (Secret) Applications (Unclassified) Memory Device Disk Device Print Device Display Device

...

slide-14
SLIDE 14

CSE443 Introduction to Computer (and Network) Security - Spring 2012 - Professor Jaeger

NetTop

  • Isolated networks of VMs
  • Alternative to “air gap” security

14

VM: Secret VM: Public SELinux Host OS

Guest OS’ Guest OS’

VMWare MLS VM: Secret VM: Public SELinux Host OS

Guest OS’ Guest OS’

VMWare MLS

slide-15
SLIDE 15

CSE443 Introduction to Computer (and Network) Security - Spring 2012 - Professor Jaeger

Xen

  • Privileged VM

15

VM: DomU VM: DomU Xen Hypervisor

Guest OS’ Guest OS’

Partitioned Resources Device Requests

Dom 0

Host OS’

Drivers

VM Services

slide-16
SLIDE 16

CSE443 Introduction to Computer (and Network) Security - Spring 2012 - Professor Jaeger

Xen sHype

  • Controlled information flows among VMs

16

VM: DomU VM: DomU Xen Hypervisor

Guest OS’ Guest OS’

Partitioned Resources Device Requests

Dom 0

Host OS’

Drivers

VM Services

Ref Mon

slide-17
SLIDE 17

CSE443 Introduction to Computer (and Network) Security - Spring 2012 - Professor Jaeger

Xen sHype Policies

  • Type Enforcement over VM communications
  • VM labels are subjects
  • VM labels are objects
  • How do VMs communicate in Xen?
  • Grant tables: pass pages between VMs
  • Event channels: notifications (e.g., when to pass pages)
  • sHype controls these
  • Q: What about VM communication across systems?

17

slide-18
SLIDE 18

CSE443 Introduction to Computer (and Network) Security - Spring 2012 - Professor Jaeger

Xen Security Modules

  • Comprehensive Reference Monitor interface for Xen
  • Based on LSM ideas
  • Includes about 57 “hooks” (more expected)
  • Supports sHype hooks
  • Plus, hooks for VM management, resource partitioning
  • Another aim: Decompose domain 0
  • Specialize kernel for privileged operations
  • E.g., Remove drivers

18

slide-19
SLIDE 19

CSE443 Introduction to Computer (and Network) Security - Spring 2012 - Professor Jaeger

IOMMU Role in the System

19

Penn State Systems and Internet Infrastructure Security Lab Page

Application! Application!

System ! Software!

RAM!

Application!

slide-20
SLIDE 20

CSE443 Introduction to Computer (and Network) Security - Spring 2012 - Professor Jaeger

IOMMU Role in the System

20

Penn State Systems and Internet Infrastructure Security Lab Page

Application! Application!

System ! Software!

RAM!

Application!

slide-21
SLIDE 21

CSE443 Introduction to Computer (and Network) Security - Spring 2012 - Professor Jaeger

IOMMU Role in the System

21

Penn State Systems and Internet Infrastructure Security Lab Page

Application! Application!

System ! Software!

RAM!

Application!

MMU! control

slide-22
SLIDE 22

CSE443 Introduction to Computer (and Network) Security - Spring 2012 - Professor Jaeger

IOMMU Role in the System

22

Penn State Systems and Internet Infrastructure Security Lab Page

Application! Application!

System ! Software!

RAM!

Peripheral! Peripheral! Peripheral!

Application!

MMU! control

slide-23
SLIDE 23

CSE443 Introduction to Computer (and Network) Security - Spring 2012 - Professor Jaeger

IOMMU Role in the System

23

Penn State Systems and Internet Infrastructure Security Lab Page

Application! Application!

System ! Software!

RAM!

Peripheral! Peripheral! Peripheral!

Application!

MMU! IOMMU! control

slide-24
SLIDE 24

CSE443 Introduction to Computer (and Network) Security - Spring 2012 - Professor Jaeger

I/O Device Assignment

24

Penn State Systems and Internet Infrastructure Security Lab Page

VM Guest 3! VM Guest 2!

RAM!

Peripheral! Peripheral! Peripheral!

VM Guest 1!

OS!

Process! Process! VM 1!

Hypervisor!

Parent ! VM 0!

control! IOMMU! MMU!

slide-25
SLIDE 25

CSE443 Introduction to Computer (and Network) Security - Spring 2012 - Professor Jaeger

VM Security Status

  • Aim is simplicity
  • Are we achieving this?
  • Do we care what happens in the VMs?
  • When might we care?
  • Trusted computing base
  • How does this compare to traditional OS?

25

slide-26
SLIDE 26

Page CSE443 Introduction to Computer and Network Security - Spring 2012 - Professor Jaeger

Virtual Machine Threats

  • How does the insertion of a virtual machine layer change

the threats against the system?

26

slide-27
SLIDE 27

Page CSE443 Introduction to Computer and Network Security - Spring 2012 - Professor Jaeger

Virtual Machine Rootkit

  • Rootkit

– Malicious software installed by an attacker on a system – Enable it to run on each boot

  • OS Rootkits

– Kernel module, signal handler, ... – When the kernel is booted, the module is installed and intercepts user process requests, interrupts, etc. – E.g., keylogger

  • VM Rootkit

– Research project from Michigan and Microsoft – If security service runs in VM, then a rootkit in VMM can evade security – E.g., Can continue to run even if the system appears to be off

27

slide-28
SLIDE 28

CSE443 Introduction to Computer (and Network) Security - Spring 2012 - Professor Jaeger

Take Away

  • VM systems focus on isolation
  • Enable reuse, but limited by security requirements
  • Enable limited communication
  • The policies are not trivial, but refer to coarser-grained
  • bjects

28