A Design for Comprehensive Kernel Instrumentation Peter Feiner Angela Demke Brown Ashvin Goel peter@cs.toronto.edu demke@cs.toronto.edu ashvin@eecg.toronto.edu University of Toronto 01 / 16 1
Motivation Transparent fault isolation for device drivers ‣ Want to isolate existing driver binaries Inspired by Byte Granularity Isolation ‣ Requires source code Use Dynamic Binary Instrumentation (DBI) ‣ Does not require source code ‣ Inspect & modify instructions before they execute x86 Driver Kernel Code 01 / 16 2
Motivation Transparent fault isolation for device drivers ‣ Want to isolate existing driver binaries Inspired by Byte Granularity Isolation ‣ Requires source code Use Dynamic Binary Instrumentation (DBI) ‣ Does not require source code ‣ Inspect & modify instructions before they execute Instrumented x86 Driver Kernel Driver Code 01 / 16 2
Motivation Transparent fault isolation for device drivers ‣ Want to isolate existing driver binaries Inspired by Byte Granularity Isolation ‣ Requires source code Use Dynamic Binary Instrumentation (DBI) ‣ Does not require source code x86 Driver ‣ Inspect & modify instructions before they execute Code Instrumented x86 Driver Kernel DBI Driver Code 01 / 16 2
Motivation DBI applied for debugging and security at the user level ‣ Memcheck - checks memory errors ‣ Program Shepherding - control flow integrity Various user-level DBI frameworks are available ‣ APIs for inspecting and modifying instructions ‣ e.g., Valgrind, DynamoRIO, Pin These frameworks don’t work in the kernel ‣ What would it take? 01 / 16 3
The Key Difference User frameworks sit between applications and the OS s p ‣ p Interpose on system calls A ‣ Take advantage of OS services, e.g. I/O DBI OS Kernel frameworks need to sit between the OS & CPU CPU ‣ Isn’t that what hypervisors do? 01 / 16 4
Our Approach We need to combine a DBI framework with a hypervisor Apps ‣ Choice 1: Port DBI to an existing hypervisor • Pros: both exist • Cons: both very complex S O ‣ Choice 2: Create a minimal hypervisor, similar to SecVisor’s approach Kernel DBI • Pros: easier to do • Pros: possibly higher performance CPU We designed a minimal hypervisor around a DBI framework ‣ Let’s see how DBI works & what it needs 01 / 16 5
DBI Technique Copy basic blocks of x86 Start code into code cache before execution Dispatch x86 Code ‣ Code executed from cache ‣ Instrumentation added to copy ‣ Manipulate copies to return control to the dispatcher No Copy Block Cached? Yes Execute from Code Cache 01 / 16 6
DBI Requirements Never execute machine’s original code ‣ Necessary for security applications Hide framework from instrumented code ‣ Instrumented code should observe un-instrumented machine state Dispatcher should use instrumented code with care ‣ Implementation cannot use non-reentrant instrumented code Detect changes to the original code ‣ Invalidate stale code in the cache Preserve multicore concurrency ‣ Essential for performance and accuracy 01 / 16 7
Meeting DBI Requirements User Kernel Never Execute New Threads, Kernel Entry Points Original Code Signals Interrupts, Transparency Signals Exceptions Implement Everything Reentrance Use OS Code From Scratch System Calls Detect Code Changes Shadow Page Tables mmap, mprotect, etc. Locking, Concurrency CPU Private Thread Private We’ll look at the first three in more detail 01 / 16 8
Never Execute Original Code User Code User Mode Supervisor Mode OS Binaries (kernel, drivers) Interrupts Exceptions 01 / 16 9
Never Execute Original Code User Code User Mode Supervisor Mode OS Binaries Code Cache Dispatcher (kernel, drivers) ‣ Load kernel module that redirects entry points to the dispatcher Interrupts Exceptions 01 / 16 9
Redirecting Entry Points Entry 1 Table OS Binaries Register Entry 2 Descriptor Table 01 / 16 10
Redirecting Entry Points Code Cache Entry 1 Table OS Binaries Register Entry 2 Descriptor Table Entry 1 Dispatcher Entry 2 Shadow Table 01 / 16 10
Redirecting Entry Points Code Cache Entry 1 Table OS Binaries Register Entry 2 Descriptor Table Entry 1 Dispatcher Entry 2 Shadow Table ‣ Can’t write to table register, otherwise loose control 01 / 16 10
Redirecting Entry Points Shadow Code Cache Register Entry 1 Table OS Binaries Register Entry 2 Descriptor Table Entry 1 Dispatcher Entry 2 Shadow Table ‣ Can’t write to table register, otherwise loose control ‣ Can’t drop the write, otherwise you loose transparency 01 / 16 10
Transparency Need to hide DBI framework from instrumented code ‣ Sometimes essential for correctness Many transparency issues, including ‣ Code cache return addresses ‣ Shadowed registers ‣ Exception stack frame ‣ Interrupt stack frame 01 / 16 11
Exception Transparency Dispatching kernel’s exception handlers is tricky because they inspect machine state ‣ Registers stolen by instrumentation ‣ Address of instruction that triggers the exception • Handlers need to see original instruction addresses • Linux panics on page faults from non white-listed instructions • Problem is that code cache isn’t on the white list • Solution is to translate from code cache to original address Solution for interrupt handlers is similar 01 / 16 12
Interrupt Transparency H = Interrupt Handler I = Instrumentation = Interrupt 01 / 16 13
Interrupt Transparency H = Interrupt Handler Original Code I = Instrumentation = Interrupt A 01 / 16 13
Interrupt Transparency H = Interrupt Handler Original Code I = Instrumentation = Interrupt A 01 / 16 13
Interrupt Transparency H = Interrupt Handler Original Code I = Instrumentation = Interrupt A IH 01 / 16 13
Interrupt Transparency H = Interrupt Handler Original Code I = Instrumentation = Interrupt A IH A 01 / 16 13
Interrupt Transparency H = Interrupt Handler Original Code I = Instrumentation = Interrupt A IH A B 01 / 16 13
Interrupt Transparency H = Interrupt Handler Original Code I = Instrumentation = Interrupt A IH A B Dispatcher Copy A 01 / 16 13
Interrupt Transparency H = Interrupt Handler Original Code I = Instrumentation = Interrupt A IH A B Dispatcher Code Cache Copy A A Original A Addresses 01 / 16 13
Interrupt Transparency H = Interrupt Handler Original Code I = Instrumentation = Interrupt A IH A B Dispatcher Code Cache Copy A A I Original A Addresses 01 / 16 13
Interrupt Transparency H = Interrupt Handler Original Code I = Instrumentation = Interrupt A IH A B Dispatcher Code Cache Copy A A I Original A Addresses 01 / 16 13
Interrupt Transparency H = Interrupt Handler Original Code I = Instrumentation = Interrupt A IH A B Dispatcher Code Cache Copy A A I Original A Addresses Delay interrupts until next code-cache exit 01 / 16 13
Interrupt Transparency H = Interrupt Handler Original Code I = Instrumentation = Interrupt A IH A B Dispatcher Code Cache Copy A A I Original A Addresses Delay interrupts until next code-cache exit 01 / 16 13
Interrupt Transparency H = Interrupt Handler Original Code I = Instrumentation = Interrupt A IH A B Dispatcher Code Cache Copy A A I A Original A A Addresses Delay interrupts until next code-cache exit 01 / 16 13
Interrupt Transparency H = Interrupt Handler Original Code I = Instrumentation = Interrupt A IH A B Dispatcher Code Cache Copy A A I A Original A A Addresses Delay interrupts until next code-cache exit 01 / 16 13
Interrupt Transparency H = Interrupt Handler Original Code I = Instrumentation = Interrupt A IH A B Dispatcher Code Cache Copy A A I A Copy IH Original A A Addresses Delay interrupts until next code-cache exit 01 / 16 13
Interrupt Transparency H = Interrupt Handler Original Code I = Instrumentation = Interrupt A IH A B Dispatcher Code Cache Copy A A I A Copy IH IH Original A A IH Addresses Delay interrupts until next code-cache exit 01 / 16 13
Interrupt Transparency H = Interrupt Handler Original Code I = Instrumentation = Interrupt A IH A B Dispatcher Code Cache Copy A A I A Copy IH IH Copy B Original A A IH Addresses Delay interrupts until next code-cache exit 01 / 16 13
Interrupt Transparency H = Interrupt Handler Original Code I = Instrumentation = Interrupt A IH A B Dispatcher Code Cache Copy A A I A Copy IH IH Copy B B Original A A IH B Addresses Delay interrupts until next code-cache exit 01 / 16 13
Reentrance Code is not reentrant if it is unsafe to execute before other executions of the same code finish ‣ Dispatcher cannot use any non-reentrant OS code, e.g. print, because the non-reentrant code might be currently executing Say, print consists of basic blocks P1, P2 ‣ P1 has executed from code cache ‣ Dispatcher copies P2 ‣ Dispatcher uses print for debugging and invokes P1 ‣ print fails because it is non-reentrant ✗ P1 Copy P2 P1 Code Cache Dispatcher 01 / 16 14
Recommend
More recommend
