Shilpi Goel shilpi@centtech.com Formal Verification Engineer Centaur Technology, Inc.
Formal Verification and Computer Architecture
A Validated Formal Model of the x86 ISA for Analyzing Computing Systems
Formal Verification and Computer Architecture A Validated Formal - - PowerPoint PPT Presentation
Formal Verification and Computer Architecture A Validated Formal Model of the x86 ISA for Analyzing Computing Systems Shilpi Goel shilpi@centtech.com Formal Verification Engineer Centaur Technology, Inc. Software and Reliability Can we rely
Shilpi Goel shilpi@centtech.com Formal Verification Engineer Centaur Technology, Inc.
Formal Verification and Computer Architecture
A Validated Formal Model of the x86 ISA for Analyzing Computing Systems
Software and Reliability
2
Can we rely on our software systems? Recent example of a serious bug: CVE-2016-5195 or “Dirty COW”
Formal Verification
3
Formal Verification: Proving or disproving that the implementation of a program meets its specification using mathematical techniques
Formal Verification
3
Formal Verification: Proving or disproving that the implementation of a program meets its specification using mathematical techniques Suppose you needed to count the number of 1s in the binary representation of a natural number v (i.e., v’s population count). E.g.,: Population count of 15 (0b1111) = 4 Population count of 8 (0b1000) = 1 Specification:
popcountSpec(v): [v: natural number] if v <= 0 then return 0 else lsb = v & 1 v = v >> 1 return (lsb + popcountSpec(v)) endif
Pop-Count Computation
4
Source: Sean Anderson’s Bit-Twiddling Hacks
popcountSpec(v): [v: natural number] if v <= 0 then return 0 else lsb = v & 1 v = v >> 1 return (lsb + popcountSpec(v)) endif
Specification:
Pop-Count Computation
4
Implementation:
int popcount_32 (unsigned int v) { v = v - ((v >> 1) & 0x55555555); v = (v & 0x33333333) + ((v >> 2) & 0x33333333); v = ((v + (v >> 4) & 0xF0F0F0F) * 0x1010101) >> 24; return(v); }
Source: Sean Anderson’s Bit-Twiddling Hacks
popcountSpec(v): [v: natural number] if v <= 0 then return 0 else lsb = v & 1 v = v >> 1 return (lsb + popcountSpec(v)) endif
Specification:
Pop-Count Computation
4
Implementation:
int popcount_32 (unsigned int v) { v = v - ((v >> 1) & 0x55555555); v = (v & 0x33333333) + ((v >> 2) & 0x33333333); v = ((v + (v >> 4) & 0xF0F0F0F) * 0x1010101) >> 24; return(v); }
Source: Sean Anderson’s Bit-Twiddling Hacks
popcountSpec(v): [v: natural number] if v <= 0 then return 0 else lsb = v & 1 v = v >> 1 return (lsb + popcountSpec(v)) endif
Specification: Do the specification and the implementation behave the same way for all relevant inputs?
5
Specification and Implementation
Two very crucial points:
5
Specification and Implementation
Two very crucial points:
5
Specification and Implementation
Two very crucial points:
Inspection of a Program’s Behavior
xExhaustive testing is infeasible
18,446,744,073,709,551,616 (264) tests!
6
Inspection of a Program’s Behavior
xExhaustive testing is infeasible
18,446,744,073,709,551,616 (264) tests!
✓ Wide variety of techniques
6
7
The Pop-Count Program: x86 Version
int popcount_32 (unsigned int v) { v = v - ((v >> 1) & 0x55555555); v = (v & 0x33333333) + ((v >> 2) & 0x33333333); v = ((v + (v >> 4) & 0xF0F0F0F) * 0x1010101) >> 24; return(v); }
7
The Pop-Count Program: x86 Version
popcount_32:
89 fa mov %edi,%edx 89 d1 mov %edx,%ecx d1 e9 shr %ecx 81 e1 55 55 55 55 and $0x55555555,%ecx 29 ca sub %ecx,%edx 89 d0 mov %edx,%eax c1 ea 02 shr $0x2,%edx 25 33 33 33 33 and $0x33333333,%eax 81 e2 33 33 33 33 and $0x33333333,%edx 01 c2 add %eax,%edx 89 d0 mov %edx,%eax c1 e8 04 shr $0x4,%eax 01 c2 add %eax,%edx 48 89 f8 mov %rdi,%rax 48 c1 e8 20 shr $0x20,%rax 81 e2 0f 0f 0f 0f and $0xf0f0f0f,%edx 89 c1 mov %eax,%ecx d1 e9 shr %ecx 81 e1 55 55 55 55 and $0x55555555,%ecx 29 c8 sub %ecx,%eax 89 c1 mov %eax,%ecx c1 e8 02 shr $0x2,%eax 81 e1 33 33 33 33 and $0x33333333,%ecx 25 33 33 33 33 and $0x33333333,%eax 01 c8 add %ecx,%eax 89 c1 mov %eax,%ecx c1 e9 04 shr $0x4,%ecx 01 c8 add %ecx,%eax 25 0f 0f 0f 0f and $0xf0f0f0f,%eax 69 d2 01 01 01 01 imul $0x1010101,%edx,%edx 69 c0 01 01 01 01 imul $0x1010101,%eax,%eax c1 ea 18 shr $0x18,%edx c1 e8 18 shr $0x18,%eax 01 d0 add %edx,%eax c3 retq
7
The Pop-Count Program: x86 Version
Functional Correctness: Final EAX = popcountSpec(Initial EDI) specification function popcountSpec(v): [v: unsigned int] if v <= 0 then return 0 else lsb = v & 1 v = v >> 1 return (lsb + popcountSpec(v)) endif
popcount_32:
89 fa mov %edi,%edx 89 d1 mov %edx,%ecx d1 e9 shr %ecx 81 e1 55 55 55 55 and $0x55555555,%ecx 29 ca sub %ecx,%edx 89 d0 mov %edx,%eax c1 ea 02 shr $0x2,%edx 25 33 33 33 33 and $0x33333333,%eax 81 e2 33 33 33 33 and $0x33333333,%edx 01 c2 add %eax,%edx 89 d0 mov %edx,%eax c1 e8 04 shr $0x4,%eax 01 c2 add %eax,%edx 48 89 f8 mov %rdi,%rax 48 c1 e8 20 shr $0x20,%rax 81 e2 0f 0f 0f 0f and $0xf0f0f0f,%edx 89 c1 mov %eax,%ecx d1 e9 shr %ecx 81 e1 55 55 55 55 and $0x55555555,%ecx 29 c8 sub %ecx,%eax 89 c1 mov %eax,%ecx c1 e8 02 shr $0x2,%eax 81 e1 33 33 33 33 and $0x33333333,%ecx 25 33 33 33 33 and $0x33333333,%eax 01 c8 add %ecx,%eax 89 c1 mov %eax,%ecx c1 e9 04 shr $0x4,%ecx 01 c8 add %ecx,%eax 25 0f 0f 0f 0f and $0xf0f0f0f,%eax 69 d2 01 01 01 01 imul $0x1010101,%edx,%edx 69 c0 01 01 01 01 imul $0x1010101,%eax,%eax c1 ea 18 shr $0x18,%edx c1 e8 18 shr $0x18,%eax 01 d0 add %edx,%eax c3 retq
8
x86 Pop-Count: A Formal Statement of Correctness
Let:
Then: Let x86f denote the final x86 state obtained after the pop-count program runs to completion. EAX(x86f) == popcountSpec(v)
8
x86 Pop-Count: A Formal Statement of Correctness
Let:
Then: Let x86f denote the final x86 state obtained after the pop-count program runs to completion. EAX(x86f) == popcountSpec(v)
Pre-conditions
8
x86 Pop-Count: A Formal Statement of Correctness
Let:
Then: Let x86f denote the final x86 state obtained after the pop-count program runs to completion. EAX(x86f) == popcountSpec(v)
Pre-conditions Post-condition
9
What Else Can You Specify and Verify?
9
What Else Can You Specify and Verify?
a function of the inputs? [ performance analysis ]
9
What Else Can You Specify and Verify?
a function of the inputs? [ performance analysis ]
Does the program “clean-up” after itself? [ security analysis ]
Why x86 Machine-Code Verification?
xSometimes, high-level code is unavailable (e.g., malware) xHigh-level verification frameworks do not address compiler bugs
✓ Verified/verifying compilers can help
xBut these compilers typically generate inefficient code
xNeed to build verification frameworks for many high-level languages
✓ x86 is in widespread use
10
Heavyweight Formal Verification
11
Heavyweight Formal Verification
11
ISA model
Heavyweight Formal Verification
11
ISA model Instruction Set Architecture: interface between hardware and software
instruction encodings, etc.
Behavior of an Instruction
12
effects made to the processor state.
add %edi, %eax je 0x400304
add %edi, %eax je 0x400304
Reasoning about Programs
13
effects made to the processor state.
constituent instructions on the machine state.
x86 ISA Model
14
Why an x86 ISA Model?
15
perform various kinds of analysis. For example:
bridges, etc.
different kinds of proteins.
Why an x86 ISA Model?
15
perform various kinds of analysis. For example:
bridges, etc.
different kinds of proteins.
programs.
Tool Used: ACL2 Theorem-Proving System
16
x86 ISA Model
Operational Semantics: x86 ISA model is a machine-code interpreter written in ACL2’s formal logic
17
x860 x861 x86k … Step 1 A Run of the x86 Interpreter that executes k instructions Step 2 Step k
Run Function
Recursively defined interpreter that specifies the x86 model run(n, x86): if n == 0 then return x86 else if model-related error encountered then return x86 else run(n - 1, step(x86)) end if end if
18
Step Function
State-transition function that corresponds to the execution of a single x86 instruction step(x86): pc = rip(x86) [prefixes, opcode, ... , imm] = Fetch-and-Decode(pc, x86) case opcode: #x00 -> add-semantic-fn(prefixes, ... , imm, x86) ... ... #xFF -> inc-semantic-fn(prefixes, ... , imm, x86)
19
Instruction Semantic Functions
20
add-semantic-fn(prefixes, ... , imm, x86):
resultSum = fix(operand1 + operand2, ...) resultFlags = computeFlags(operand1, operand2, result, x86) x86 = updateState(resultSum, dst, resultFlags) return x86
Obtaining the x86 ISA Specification
21
~3000 pages ~3400 pages
__asm__ volatile ("stc\n\t" // Set CF. "mov $0, %%eax\n\t" // Set EAX = 0. "mov $0, %%ebx\n\t" // Set EBX = 0. "mov $0, %%ecx\n\t" // Set ECX = 0. "mov %4, %%ecx\n\t" // Set CL = rotate_by. "mov %3, %%edx\n\t" // Set EDX = old_cf = 1. "mov %2, %%eax\n\t" // Set EAX = num. "rcl %%cl, %%al\n\t" // Rotate AL by CL. "cmovb %%edx, %%ebx\n\t" // Set EBX = old_cf if CF = 1. // Otherwise, EBX = 0. "mov %%eax, %0\n\t" // Set res = EAX. "mov %%ebx, %1\n\t" // Set cf = EBX. : "=g"(res), "=g"(cf) : "g"(num), "g"(old_cf), "g"(rotate_by) : "rax", "rbx", "rcx", "rdx");
Running tests on x86 machines
x86 State
22
Figure 3-2. 64-Bit Mode Execution Environment 2^64 -1 Sixteen 64-bit 64-bits 64-bits General-Purpose Registers Segment Registers RFLAGS Register RIP (Instruction Pointer Register) Address Space Six 16-bit Registers Registers Eight 80-bit Registers Floating-Point Data Registers Eight 64-bit Registers MMX Registers XMM Registers Sixteen 128-bit Registers 16 bits Control Register 16 bits Status Register 64 bits FPU Instruction Pointer Register 64 bits FPU Data (Operand) Pointer Register FPU Registers MMX Registers XMM Registers 32-bits MXCSR Register Opcode Register (11-bits) Basic Program Execution Registers 16 bits Tag RegisterFocus: Intel’s 64-bit mode
Source: Intel Manuals
x86 State
22
Figure 3-2. 64-Bit Mode Execution Environment 2^64 -1 Sixteen 64-bit 64-bits 64-bits General-Purpose Registers Segment Registers RFLAGS Register RIP (Instruction Pointer Register) Address Space Six 16-bit Registers Registers Eight 80-bit Registers Floating-Point Data Registers Eight 64-bit Registers MMX Registers XMM Registers Sixteen 128-bit Registers 16 bits Control Register 16 bits Status Register 64 bits FPU Instruction Pointer Register 64 bits FPU Data (Operand) Pointer Register FPU Registers MMX Registers XMM Registers 32-bits MXCSR Register Opcode Register (11-bits) Basic Program Execution Registers 16 bits Tag RegisterFocus: Intel’s 64-bit mode
Source: Intel Manuals
Model Validation
How can we know that our model faithfully represents the x86 ISA? Validate the model to increase trust in the applicability of formal analysis
23
Reasoning about Programs: Symbolic Execution
24
What is Symbolic Execution?
25
What is Symbolic Execution?
25
symbolic updates made to the initial x86 state
What is Symbolic Execution?
25
symbolic updates made to the initial x86 state
It’s nothing but algebra really:
mathematical functions.
relationships among various unknowns.
Symbolic Execution: Example
26
addr: 0xf8 // CLC: clear the carry flag addr + 1: 0xf9 // STC: set the carry flag
Symbolic Execution: Example
26
addr: 0xf8 // CLC: clear the carry flag addr + 1: 0xf9 // STC: set the carry flag
located at a good memory location addr; PC points to addr.
Symbolic Execution: Example
26
addr: 0xf8 // CLC: clear the carry flag addr + 1: 0xf9 // STC: set the carry flag
located at a good memory location addr; PC points to addr.
let x861a := write(CF, 0, x86i) x861b := write(PC, addr + 1, x861a) then x861 == x861b
Symbolic Execution: Example
26
addr: 0xf8 // CLC: clear the carry flag addr + 1: 0xf9 // STC: set the carry flag
located at a good memory location addr; PC points to addr.
let x861a := write(CF, 0, x86i) x861b := write(PC, addr + 1, x861a) then x861 == x861b
let x862a := write(CF, 1, x861) x862b := write(PC, addr + 2, x862a) then x86f == x862b
Symbolic Execution: Example
27
let x861a := write(CF, 0, x86i) x861b := write(PC, addr + 1, x861a) then x861 == x861b let x862a := write(CF, 1, x861) x862b := write(PC, addr + 2, x862a) then x86f == x862b
Symbolic Execution: Example
27
let x861a := write(CF, 0, x86i) x861b := write(PC, addr + 1, x861a) then x861 == x861b let x862a := write(CF, 1, x861) x862b := write(PC, addr + 2, x862a) then x86f == x862b x86f in terms of x86i: x86f == write[PC, addr + 2, write(CF, 1, write{PC, addr + 1, write(CF, 0, x86i)})]
Symbolic Execution: Example
27
let x861a := write(CF, 0, x86i) x861b := write(PC, addr + 1, x861a) then x861 == x861b let x862a := write(CF, 1, x861) x862b := write(PC, addr + 2, x862a) then x86f == x862b x86f in terms of x86i: x86f == write[PC, addr + 2, write(CF, 1, write{PC, addr + 1, write(CF, 0, x86i)})] After simplifying: x86f == write[PC, addr + 2, write(CF, 1, x86i)]
Component Projections
28
Can project out “interesting” components from the final state: read(CF,x86f) == read(CF, write[PC, addr + 2, write(CF, 1, x86i)]) == read(CF, write(CF, 1, x86i)) == 1 x86f == write[PC, addr + 2, write(CF, 1, x86i)]
Efficient Symbolic Execution
29
values to the x86 state enable efficient symbolic execution.
30
y
memory non-interference Program Order
i j
Read-over-Write Theorem #1
30
y
Wi(x) memory non-interference Program Order
x
i j
Read-over-Write Theorem #1
30
y
Wi(x) Rj: y memory non-interference Program Order
x
i j
Read-over-Write Theorem #1
31
memory
i
Read-over-Write Theorem #2
Program Order
31
Wi(x) memory
x
i
Read-over-Write Theorem #2
Program Order
31
Wi(x) Ri: x memory
x
i
Read-over-Write Theorem #2
Program Order
32
memory independent writes commute safely
i j
Program Order
Write-over-Write Theorem #1
32
memory independent writes commute safely Wi(x)
i j
x
Program Order
Write-over-Write Theorem #1
32
memory independent writes commute safely Wi(x)
i j
x y
Wj(y) Program Order
Write-over-Write Theorem #1
32
=
memory independent writes commute safely memory Wi(x)
i j
x y
Wj(y)
i j
Program Order
Write-over-Write Theorem #1
Program Order
32
=
memory independent writes commute safely memory Wi(x)
i j
x y
Wj(y)
i j
Wj(y)
y
Program Order
Write-over-Write Theorem #1
Program Order
32
=
memory independent writes commute safely memory Wi(x)
i j
x y
Wj(y)
i j
Wj(y) Wi(x)
x y
Program Order
Write-over-Write Theorem #1
Program Order
33
memory visibility of writes
i
Write-over-Write Theorem #2
Program Order
33
memory visibility of writes Wi(x)
i
x
Write-over-Write Theorem #2
Program Order
33
memory visibility of writes Wi(x)
i
Wi(y)
y
Write-over-Write Theorem #2
Program Order
33
=
memory visibility of writes memory Wi(x)
i
Wi(y)
i
y
Write-over-Write Theorem #2
Program Order Program Order
33
=
memory visibility of writes memory Wi(x)
i
Wi(y)
i
Wi(y)
y y
Write-over-Write Theorem #2
Program Order Program Order
What I Haven’t Talked About Today
34
Conclusions
35
Resources
36
There are exciting research and engineering projects in this area!
Opportunities for Future Research
37
Operating System Verification detect reliance on non-portable or undefined behaviors User-friendly Program Analysis automate the discovery of preconditions Multi-process/threaded Program Verification reason about concurrency-related issues Reasoning about the Memory System determine if caches are (mostly) transparent, as intended Firmware Verification formally specify software/hardware interfaces Micro-architecture Verification x86 ISA model serves as a build-to specification
What We Do at
38
x86 Hardware Verification
Centaur Technology
39
Formal Verification at Centaur Technology
40
correctly implement (parts of) the x86 instruction-set architecture.
Hardware Software
Instruction-Set Architecture Micro-architecture Machine Code … …
[Source Code]
Github
[Documentation]
x86isa in the ACL2+Community Books Manual
Publications
Shilpi Goel, Warren A. Hunt, Jr., and Matt Kaufmann. Abstract Stobjs and Their Application to ISA Modeling. In Proceedings of the ACL2 Workshop 2013, EPTCS 114, pp. 54-69, 2013 Shilpi Goel and Warren A. Hunt, Jr. Automated Code Proofs on a Formal Model of the x86. In Verified Software: Theories, Tools, Experiments (VSTTE’13), volume 8164 of Lecture Notes in Computer Science, pages 222– 241. Springer Berlin Heidelberg, 2014 Shilpi Goel, Warren A. Hunt, Jr., Matt Kaufmann, and Soumava Ghosh. Simulation and Formal Verification of x86 Machine-Code Programs That Make System Calls. In Proceedings of the 14th Conference on Formal Methods in Computer-Aided Design (FMCAD’14), pages 18:91–98, 2014 Shilpi Goel, Warren A. Hunt, Jr., and Matt Kaufmann. Engineering a Formal, Executable x86 ISA Simulator for Software Verification. In Provably Correct Systems (ProCoS), 2015 Shilpi Goel. Formal Verification of Application and System Programs Based on a Validated x86 ISA Model. Ph.D. Dissertation, The University of Texas at Austin, 2016 Shilpi Goel. The x86isa Books: Features, Usage, and Future Plans. In the Fourteenth International Workshop on the ACL2 Theorem Prover and Its Applications (ACL2 Workshop), 2017
42