SLIDE 1
XXX Tolerating Malicious Drivers in Linux Silas Boyd-Wickizer and - - PowerPoint PPT Presentation
XXX Tolerating Malicious Drivers in Linux Silas Boyd-Wickizer and - - PowerPoint PPT Presentation
XXX Tolerating Malicious Drivers in Linux Silas Boyd-Wickizer and Nickolai Zeldovich How could a device driver be malicious? Today's device drivers are highly privileged Write kernel memory, allocate memory, ... Drivers are complex;
SLIDE 2
SLIDE 3
How could a device driver be malicious?
Today's device drivers are highly privileged
Write kernel memory, allocate memory, ...
Drivers are complex; developers write buggy code Result: Attackers exploit vulnerabilities
SLIDE 4
How could a device driver be malicious?
Today's device drivers are highly privileged
Write kernel memory, allocate memory, ...
Drivers are complex; developers write buggy code Result: Attackers exploit vulnerabilities
SLIDE 5
How could a device driver be malicious?
Today's device drivers are highly privileged
Write kernel memory, allocate memory, ...
Drivers are complex; developers write buggy code Result: Attackers exploit vulnerabilities
SLIDE 6
Current approach
User-space drivers in µkernels (Minix, L4, ...) Write device driver in new language (Termite) Handle common faults (Nooks, microdrivers, ...)
SLIDE 7
Secure, efficient, & unmodified drivers on Linux
Goal
SLIDE 8
Previous user-space drivers
Kernel User Kernel core Network stack Hardware Ethernet driver User User Application
µkernel
SLIDE 9
Previous user-space drivers
Kernel User Kernel core Network stack Hardware Ethernet driver User User Application
µkernel
Confine driver in a process
SLIDE 10
Previous user-space drivers
Kernel User Kernel core Network stack Hardware Ethernet driver User User Application
µkernel
Confine driver in a process General purpose syscall API to configure device
SLIDE 11
Previous user-space drivers
Kernel User Kernel core Network stack Hardware Ethernet driver User User Application
µkernel
Confine driver in a process General purpose syscall API to configure device Confine device with IO virtualization HW.
SLIDE 12
Previous user-space drivers
Kernel User Kernel core Network stack Hardware Ethernet driver User User Application
µkernel
Confine driver in a process General purpose syscall API to configure device IPC network driver API E.g. tx_packet Confine device with IO virtualization HW.
SLIDE 13
Current Linux driver architecture
Kernel User Ethernet driver Network stack Application Hardware
netdevice
Kernel RT
SLIDE 14
Current Linux driver architecture
Kernel User Ethernet driver Network stack Application Hardware
netdevice
Kernel RT Kernel runtime (e.g. kmalloc)
SLIDE 15
Current Linux driver architecture
Kernel User Ethernet driver Network stack Application Hardware
netdevice
Kernel RT Kernel runtime (e.g. kmalloc) Network driver API (e.g. tx_packet)
SLIDE 16
Linux user-space driver problem
Kernel RT and driver APIs won't work for untrusted drivers in a different AS
Kernel User Ethernet driver Network stack Application Hardware
netdevice
User Kernel RT
SLIDE 17
SUD's approach
Kernel User Ethernet driver Network stack Application Hardware
netdevice
User Kernel RT
SLIDE 18
SUD's approach
SUD UML handles calls to kernel RT
Kernel User Ethernet driver Network stack Application Hardware
netdevice
User Kernel RT SUD UML
SLIDE 19
SUD's approach
SUD UML handles calls to kernel RT Proxy driver and SUD UML allow reuse of existing driver APIs
Kernel User Ethernet driver Network stack Application Hardware
netdevice
User Kernel RT SUD UML Ethernet proxy driver
SLIDE 20
SUD's approach
SUD UML handles calls to kernel RT Proxy driver and SUD UML allow reuse of existing driver APIs
Kernel User Ethernet driver Network stack Application Hardware
netdevice
User Kernel RT SUD UML Ethernet proxy driver Network driver API
SLIDE 21
SUD's approach
SUD UML handles calls to kernel RT Proxy driver and SUD UML allow reuse of existing driver APIs
Kernel User Ethernet driver Network stack Application Hardware
netdevice
User Kernel RT SUD UML Ethernet proxy driver Network driver API SUD RPC API
SLIDE 22
SUD's approach
SUD UML handles calls to kernel RT Proxy driver and SUD UML allow reuse of existing driver APIs
Kernel User Ethernet driver Network stack Application Hardware
netdevice
User Kernel RT SUD UML Ethernet proxy driver Network driver API SUD RPC API Network driver API
SLIDE 23
SUD's results
Tolerate malicious device drivers Proxy drivers small (~500 LOC) One proxy driver per device class Few kernel modifications (~50 LOC) Unmodified drivers (6 test drivers) High performance, low overhead No need for new OS or language
SLIDE 24
Security challenge: prevent attacks
Problem: driver must perform privileged
- perations
Memory access, driver API, DMA, interrupts, …
Attacks from driver code:
Direct system attacks: memory corruption, ... Driver API attacks: invalid return value, deadlock, ...
Attacks from device:
DMA to DRAM, peer-to-peer attacks, interrupt storms
SLIDE 25
Practical challenges
High performance, low overhead
Challenge: interact with hardware and kernel at high rate, kernel-user switch expensive E.g. Ethernet driver ~100k times a second
Reuse existing drivers and kernel
Challenge: drivers assume fully-privileged kernel env. Challenge: kernel driver API complex, non-uniform
SLIDE 26
SUD overview
Kernel User Proxy driver Kernel core Application Hardware Driver User SUD UML HW access module
SLIDE 27
SUD overview
Kernel User Proxy driver Kernel core Application Hardware Driver User SUD UML HW access module
SLIDE 28
Linux driver APIs
Linux defines a driver API for each device class
Driver and kernel functions and variables
SLIDE 29
Example: wireless driver API
Linux defines a driver API for each device class
Driver and kernel functions and variables
struct wireless_ops { int (*tx)(struct sk_buff*); int (*configure_filter)(int); ... }; struct wireless_hw { int conf; int flags .... };
SLIDE 30
Example: wireless driver API
Linux defines a driver API for each device class
Driver and kernel functions and variables Proxy drivers and SUD-UML convert API to RPCs
struct wireless_ops { int (*tx)(struct sk_buff*); int (*configure_filter)(int); ... }; struct wireless_hw { int conf; int flags .... };
SLIDE 31
Example: wireless driver API
Linux defines a driver API for each device class
Driver and kernel functions and variables Proxy drivers and SUD-UML convert API to RPCs
struct wireless_ops { int (*tx)(struct sk_buff*); int (*configure_filter)(int); ... }; struct wireless_hw { int conf; int flags .... };
SLIDE 32
Example: wireless driver API
Linux defines a driver API for each device class
Driver and kernel functions and variables Proxy drivers and SUD-UML convert API to RPCs
struct wireless_ops { int (*tx)(struct sk_buff*); int (*configure_filter)(int); ... }; struct wireless_hw { int conf; int flags .... }; Called in a non- preemptable context
SLIDE 33
Example: wireless driver API
Linux defines a driver API for each device class
Driver and kernel functions and variables Proxy drivers and SUD-UML convert API to RPCs
struct wireless_ops { int (*tx)(struct sk_buff*); int (*configure_filter)(int); ... }; struct wireless_hw { int conf; int flags .... }; Called in a non- preemptable context Driver API variable
SLIDE 34
Wireless driver in SUD
Basic driver API → SUD RPC API→ driver API Non-preemptable function: implement in proxy Driver API variable: shadow variables
SLIDE 35
Example 1: transmit a packet
Kernel User Wireless proxy driver Wireless core Web browser Hardware Wireless driver User SUD UML
SLIDE 36
Example 1: transmit a packet
Kernel User Wireless proxy driver Wireless core Web browser Hardware Wireless driver User SUD UML Socket write
SLIDE 37
Example 1: transmit a packet
Kernel User Wireless proxy driver Wireless core Web browser Hardware Wireless driver User SUD UML wireless_ops.tx
SLIDE 38
Example 1: transmit a packet
Kernel User Wireless proxy driver Wireless core Web browser Hardware Wireless driver User SUD UML TX packet RPC
SLIDE 39
Example 1: transmit a packet
Kernel User Wireless proxy driver Wireless core Web browser Hardware Wireless driver User SUD UML wireless_ops.tx
SLIDE 40
Example 1: transmit a packet
Kernel User Wireless proxy driver Wireless core Web browser Hardware Wireless driver User SUD UML DMA TX
SLIDE 41
Example 2: non-preemptable callback
Problem: unable to switch to user-space
Kernel User Wireless proxy driver Wireless core Web browser Hardware Wireless driver User SUD UML
SLIDE 42
Problem: unable to switch to user-space
Kernel User Wireless proxy driver Wireless core Web browser Hardware Wireless driver User SUD UML Acquires a spin lock
Example 2: non-preemptable callback
SLIDE 43
Problem: unable to switch to user-space
Kernel User Wireless proxy driver Wireless core Web browser Hardware Wireless driver User SUD UML wireless_ops.configure_filter
Example 2: non-preemptable callback
SLIDE 44
Problem: unable to switch to user-space
Kernel User Wireless proxy driver Wireless core Web browser Hardware Wireless driver User SUD UML Filter RPC
Example 2: non-preemptable callback
SLIDE 45
Problem: unable to switch to user-space
Kernel User Wireless proxy driver Wireless core Web browser Hardware Wireless driver User SUD UML
Example 2: non-preemptable callback
SLIDE 46
Problem: unable to switch to user-space Solution: implement directly in proxy driver
Kernel User Wireless proxy driver Wireless core Web browser Hardware Wireless driver User SUD UML
Example 2: non-preemptable callback
SLIDE 47
Problem: unable to switch to user-space Solution: implement directly in proxy driver
Kernel User Wireless proxy driver Wireless core Web browser Hardware Wireless driver User SUD UML Register RX packet types
Example 2: non-preemptable callback
SLIDE 48
Problem: unable to switch to user-space Solution: implement directly in proxy driver
Kernel User Wireless proxy driver Wireless core Web browser Hardware Wireless driver User SUD UML
Example 2: non-preemptable callback
Acquires a spin lock
SLIDE 49
Problem: unable to switch to user-space Solution: implement directly in proxy driver
Kernel User Wireless proxy driver Wireless core Web browser Hardware Wireless driver User SUD UML
Example 2: non-preemptable callback
wireless_ops.configure_filter
SLIDE 50
Problem: unable to switch to user-space Solution: implement directly in proxy driver
Kernel User Wireless proxy driver Wireless core Web browser Hardware Wireless driver User SUD UML
Example 2: non-preemptable callback
Return RX packet types
SLIDE 51
Example 3: driver API variables
Problem: user-space can't access API variables
Kernel User Wireless proxy driver Wireless core Web browser Hardware Wireless driver User SUD UML
wireless_hw
SLIDE 52
Problem: user-space can't access API variables
Kernel User Wireless proxy driver Wireless core Web browser Hardware Wireless driver User SUD UML
wireless_hw
Driver API variable
Example 3: driver API variables
SLIDE 53
Problem: user-space can't access API variables
Kernel User Wireless proxy driver Wireless core Web browser Hardware Wireless driver User SUD UML
wireless_hw
Writes to wireless_hw
Example 3: driver API variables
SLIDE 54
Problem: user-space can't access API variables
Kernel User Wireless proxy driver Wireless core Web browser Hardware Wireless driver User SUD UML
wireless_hw
Example 3: driver API variables
SLIDE 55
Problem: user-space can't access API variables Solution: allocate a shadow copy and synchronize before and after RPCs
Kernel User Wireless proxy driver Wireless core Web browser Hardware Wireless driver User SUD UML
wireless_hw
Example 3: driver API variables
SLIDE 56
Problem: user-space can't access API variables Solution: allocate a shadow copy and synchronize before and after RPCs
Kernel User Wireless proxy driver Wireless core Web browser Hardware Wireless driver User SUD UML
wireless_hw wireless_hw
Shadow variable
Example 3: driver API variables
SLIDE 57
Problem: user-space can't access API variables Solution: allocate a shadow copy and synchronize before and after RPCs
Kernel User Wireless proxy driver Wireless core Web browser Hardware Wireless driver User SUD UML
wireless_hw wireless_hw
Writes to wireless_hw
Example 3: driver API variables
SLIDE 58
Problem: user-space can't access API variables Solution: allocate a shadow copy and synchronize before and after RPCs
Kernel User Wireless proxy driver Wireless core Web browser Hardware Wireless driver User SUD UML
wireless_hw wireless_hw
Synchronize before sending RPC
Example 3: driver API variables
SLIDE 59
Problem: user-space can't access API variables Solution: allocate a shadow copy and synchronize before and after RPCs
Kernel User Wireless proxy driver Wireless core Web browser Hardware Wireless driver User SUD UML
wireless_hw wireless_hw
Send RPC
Example 3: driver API variables
SLIDE 60
Problem: user-space can't access API variables Solution: allocate a shadow copy and synchronize before and after RPCs
Kernel User Wireless proxy driver Wireless core Web browser Hardware Wireless driver User SUD UML
wireless_hw wireless_hw
Reads updates from shadow variable
Example 3: driver API variables
SLIDE 61
SUD overview
Kernel User Proxy driver Kernel core Application Hardware Driver User SUD UML HW access module
SLIDE 62
SUD overview
Kernel User Proxy driver Kernel core Application Hardware Driver User SUD UML HW access module
SLIDE 63
Attacks from hardware
CPU PCI bus DRAM Memory interconnect
SLIDE 64
Attacks from hardware
CPU PCI bus DRAM Memory interconnect
Driver configures the device to execute attacks
SLIDE 65
Attacks from hardware
CPU PCI bus DRAM Memory interconnect
Driver configures the device to execute attacks
DMA to DRAM
SLIDE 66
Attacks from hardware
CPU PCI bus DRAM Memory interconnect
Driver configures the device to execute attacks
DMA to DRAM Peer-to-peer messages
SLIDE 67
Attacks from hardware
CPU PCI bus DRAM Memory interconnect
Driver configures the device to execute attacks
DMA to DRAM Peer-to-peer messages Interrupt storms
SLIDE 68
Attacks from hardware
Driver configures the device to execute attacks
DMA to DRAM Peer-to-peer messages Interrupt storms
HW access module prevents attacks
Interposes on driver-device communication Uses IO virtualization to provide direct device access
SLIDE 69
IO virtualization hardware
CPU MSI IOMMU PCI express switch DRAM Memory interconnect APIC interconnect
SLIDE 70
IO virtualization hardware
CPU MSI IOMMU PCI express switch DRAM Memory interconnect APIC interconnect
Use IOMMU to map DMA buffer pools
Prevents DMA to DRAM attacks
SLIDE 71
IO virtualization hardware
CPU MSI IOMMU PCI express switch DRAM Memory interconnect APIC interconnect
Use PCI ACS to prevent peer-to-peer messaging
Prevents peer-to-peer attacks
SLIDE 72
IO virtualization hardware
CPU MSI IOMMU PCI express switch DRAM Memory interconnect APIC interconnect
Use MSI to mask interrupts
Prevents interrupt storms
SLIDE 73
Interrupt handlers in Linux
Kernel Driver IRQ core User MSI
SLIDE 74
Interrupt handlers in Linux
Kernel Driver IRQ core User MSI
SLIDE 75
Interrupt handlers in Linux
Driver called with IRQs disabled (non-preemptable)
Kernel Driver IRQ core User MSI
SLIDE 76
Interrupt handlers in Linux
Kernel calls driver interrupt handler Driver clears interrupt flag
Kernel Driver IRQ core User MSI
SLIDE 77
Interrupt handlers with SUD
Kernel HW access module IRQ core User MSI Driver SUD UML
SLIDE 78
Interrupt handlers with SUD
Kernel calls HW access module interrupt handler HW access module masks interrupt with MSI
Kernel HW access module IRQ core User MSI Driver SUD UML
SLIDE 79
Interrupt handlers with SUD
Kernel calls HW access module interrupt handler HW access module masks interrupt with MSI
Kernel HW access module IRQ core User MSI Driver SUD UML
SLIDE 80
Interrupt handlers with SUD
Kernel calls HW access module interrupt handler HW access module masks interrupt with MSI Asynchronous RPC to driver
Kernel HW access module IRQ core User MSI Driver SUD UML
SLIDE 81
Interrupt handlers with SUD
Kernel calls HW access module interrupt handler HW access module masks interrupt with MSI Asynchronous RPC to driver Driver clears interrupt
Kernel HW access module IRQ core User MSI Driver SUD UML
SLIDE 82
Interrupt handlers with SUD
HW access module masks interrupt with MSI Asynchronous RPC to driver Driver clears interrupt HW access module unmasks MSI
Kernel HW access module IRQ core User MSI Driver SUD UML
SLIDE 83
SUD overview
Kernel User Proxy driver Kernel core Application Hardware Driver User SUD UML HW access module
SLIDE 84
Prototype of SUD
Supports all Ethernet, wireless, USB, audio drivers Tested: e1000e, ne2k-pci, iwlagn, snd_hda_intel, ehci_hcd, uhci_hcd, ...
Trusted code Lines of code PCI access module 2800 Ethernet proxy driver 300 Wireless proxy driver 600 Audio proxy driver 550 Untrusted code Lines of code User-mode runtime 5000 Drivers 5000 – 50,000 (each)
SLIDE 85
Trusted code Lines of code PCI access module 2800 Ethernet proxy driver 300 Wireless proxy driver 600 Audio proxy driver 550 Untrusted code Lines of code User-mode runtime 5000 Drivers 5000 – 50,000 (each)
Prototype of SUD
Supports all Ethernet, wireless, USB, audio drivers Tested: e1000e, ne2k-pci, iwlagn, snd_hda_intel, ehci_hcd, uhci_hcd, ...
SLIDE 86
Trusted code Lines of code PCI access module 2800 Ethernet proxy driver 300 Wireless proxy driver 600 Audio proxy driver 550 Untrusted code Lines of code User-mode runtime 5000 Drivers 5000 – 50,000 (each)
Prototype of SUD
Supports all Ethernet, wireless, USB, audio drivers Tested: e1000e, ne2k-pci, iwlagn, snd_hda_intel, ehci_hcd, uhci_hcd, ...
SLIDE 87
Performance
For most devices, does not matter
Printers, cameras, …
Stress-test: e1000e gigabit network card
Requires high throughput Requires low latency Many device driver interactions
Test machine: 1.4GHz dual core Thinkpad
SLIDE 88
Performance questions?
What performance does SUD get?
Network throughput, latency
How much does it cost?
CPU cycles
SLIDE 89
SUD achieves same device performance
TCP UDP TX UDP RX UDP latency
0.2 0.4 0.6 0.8 1
Linux Sud
Normalized throughput relative to Linux TCP: streaming (950 Mbps in both cases) UDP: one-byte-data packets
Throughput relative to Linux
SLIDE 90
CPU cost is low
TCP UDP TX UDP RX UDP latency
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Linux Sud
SUD overhead: user-kernel switch, TLB misses Overheads not significant for many workloads (packets larger than min. packet size)
CPU utilization
SLIDE 91
Future directions
Explore hierarchical untrusted device drivers
PCI bus → SATA controller → SATA disk → …
Explore giving apps direct hardware access
Safe HW access for network analyzer, X server, …
Performance analysis and optimizations
SUD specific device drivers, super pages, ...
SLIDE 92
Related work
Mircokernels (Minix, L4, ...)
Simple drivers, driver API designed for user-space
Nooks, microdrivers
Handles common bugs, many changes to kernel
Languages (e.g. Termite), source code analysis
Complimentary to user-space drivers
No need for new OS or language
SLIDE 93
Summary
Driver bugs lead to system crashes or exploits SUD protects Linux from malicious drivers using proxy drivers and IO virtualization HW
Runs unmodified Linux device drivers High performance, low overheads Few modifications to Linux kernel
SLIDE 94
SLIDE 95