Machine Virtualization: Efficient Hypervisors, Stealthy Malware - - PowerPoint PPT Presentation

Machine Virtualization: Efficient Hypervisors, Stealthy Malware

Muli Ben-Yehuda

Technion & Hypervisor Technologies and Consulting Ltd

Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 1 / 21

Background: x86 machine virtualization

Running multiple different unmodified operating systems Each in an isolated virtual machine Simultaneously On the x86 architecture Many uses: live migration, record & replay, testing, . . . , security Foundation of IaaS cloud computing Used nearly everywhere

Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 2 / 21

x86 virtualization primer

How does it work? Popek and Goldberg’s virtualization model [Popek74]: Trap and emulate Privileged instructions trap to the hypervisor Hypervisor emulates their behavior Without hardware support With hardware support

Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 3 / 21

What is a rootkit?

First you take control. How? Then you hide to avoid detection and maintain control. How? Usual methods are ugly and intrusive: easy to detect! Can rootkit authors do better?

Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 4 / 21

Hypervisor-level rootkits

Hypervisors have full control over the hardware Hypervisors can trap any operating system event Code can enter hypervisor-mode at any time Bluepill: run the rootkit as the hypervisor

Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 5 / 21

Bluepill: a hypervisor level rootkit [Rutkowska06]

Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 6 / 21

Recursive Bluepill

Bluepill installs itself on the fly Bluepill is now the hypervisor Reminder: x86 only supports one hypervisor in hardware So how can you bluepill bluepill?

Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 7 / 21

The Turtles project: Nested x86 Virtualization

Efficient nested virtualization for Intel x86 based on KVM Runs multiple guest hypervisors and VMs

“The Turtles Project: Design and Implementation of Nested Virtualization”, [Ben-Yehuda10] Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 8 / 21

What is the Turtles project? (cont’)

Nested VMX virtualization for nested CPU virtualization Multi-dimensional paging for nested MMU virtualization Multi-level device assignment for nested I/O virtualization Micro-optimizations to make it go fast

+ + =

Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 9 / 21

Theory of nested CPU virtualization

Trap and emulate[PopekGoldberg74] ⇒ it’s all about the traps Single-level (x86) vs. multi-level (e.g., z/VM) Single level ⇒ one hypervisor, many guests Turtles approach: L0 multiplexes the hardware between L1 and L2, running both as guests of L0—without either being aware of it (Scheme generalized for n levels; Our focus is n=2)

Hardware Host Hypervisor Guest Hardware Host Hypervisor Multiplexed on a single level Multiple logical levels

L0 L1 L2 L1

Guest

L2

Guest

L2 L0

Guest

L2 L2

Guest Hypervisor Guest Hypervisor Guest Guest Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 10 / 21

Detecting hypervisor-based rootkits

Bluepill authors claim “undetectable” “Compatibility is Not Transparency: VMM Detection Myths and Realities” [Garfinkel07] Hardware discrepancies Resource-sharing attacks Timing attacks: PCI register access, page-faults on MMIO access, cpuid timing vs. nops Can you trust time?

Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 11 / 21

The Dual Role of a Hypervisor

Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 12 / 21

Background: interrupts

IDT IDTR Limit Address IDT Entry IDT Entry … IDT Entry Vector 1 Vector n Vector 2

Interrupt Descriptor Table IDT Register Interrupt handlers

I/O devices raise interrupts CPU temporarily stops the currently executing code CPU jumps to a pre-specified interrupt handler

Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 13 / 21

Interrupts as an Attack Vector

Follow the White Rabbit [Rutkowska11] Tell the device to generate “interesting” interrupts Attack: fool the CPU into SIPI Attack: syscall/hypercall injection In interrupt-based attacks an untrusted guest generates malicious interrupts which are handled in host mode Protect: handle interrupts in guest—not host—mode Serve: bare-metal performance!

Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 14 / 21

ELI: Exitless Interrupts

bare-metal Baseline

guest hypervisor

(time)

ELI delivery

guest hypervisor

ELI delivery & completion

guest hypervisor

Physical Interrupt Interrupt Completion

(a) (b) (c)

Interrupt Injection Interrupt Completion

(d)

ELI: direct interrupts for unmodified, untrusted guests

“ELI: Bare-Metal Performance for I/O Virtualization”, Gordon12 Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 15 / 21

ELI: delivery

Shadow IDT

Hypervisor

Shadow IDT Interrupt Handler

Assigned Interrupt

Physical Interrupt

Non-assigned Interrupt (#NP/#GP exit) ELI Delivery

Guest IDT

VM

IDT Entry IDT Entry

…

IDT Entry

P=0 P=1 P=0

Handler #NP #NP IDT Entry #GP

IDTR Limit

All interrupts are delivered directly to the guest Host and other guests’ interrupts are bounced back to the host . . . without the guest being aware of it

Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 16 / 21

ELI: signaling completion

Guests signal interrupt completions by writing to the Local Advance Programmable Interrupt Controller (LAPIC) End-of-Interrupt (EOI) register Old LAPIC: hypervisor traps load/stores to LAPIC page x2APIC: hypervisor can trap specific registers Signaling completion without trapping requires x2APIC ELI gives the guest direct access only to the EOI register

Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 17 / 21

ELI: threat model

Threats: malicious guests might try to: keep interrupts disabled signal invalid completions consume other guests or host interrupts

Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 18 / 21

ELI: protection

VMX preemption timer to force exits instead of timer interrupts Ignore spurious EOIs Protect critical interrupts by:

Delivering them to a non-ELI core if available Redirecting them as NMIs→unconditional exit Use IDTR limit to force #GP exits on critical interrupts

Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 19 / 21

Conclusions

Machine virtualization be used for good, or evil How do you protect and serve? Happy hacking!

Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 20 / 21

Questions?

muli@cs.technion.ac.il mulix@hypervisorconsulting.com

Muli Ben-Yehuda (Technion & Hypervisor) Efficient Hypervisors, Stealthy Malware Cyberday, 2013 21 / 21