Scaling symbolic evaluation for automated verification of systems - - PowerPoint PPT Presentation

▶
scaling symbolic evaluation for automated verification of
SMART_READER_LITE
LIVE PREVIEW

Scaling symbolic evaluation for automated verification of systems - - PowerPoint PPT Presentation

Dec 21, 2023 315 likes •714 views

Scaling symbolic evaluation for automated verification of systems code with Serval Luke Nelson , James Bornholt , Ronghui Gu , Andrew Baumann , Emina Torlak , Xi Wang University of Washington, Columbia University,

slide-1
SLIDE 1

Scaling symbolic evaluation for automated verification of systems code with Serval

Luke Nelson¹, James Bornholt¹, Ronghui Gu², Andrew Baumann³, Emina Torlak¹, Xi Wang¹ ¹University of Washington, ²Columbia University, ³Microsoft Research

1

slide-2
SLIDE 2

Goal: eliminating bugs with formal verification

2

System implementation Functional specification Safety specification (e.g. noninterference)

OS Kernel / security monitor Process Process Process

Refinement

slide-3
SLIDE 3

Using interactive / auto-active verification

3

  • Require manual proof annotations/tactics
  • Expensive: CertiKOS 200k LOC proof
  • Multiple person-years

Komodo (SOSP’17) CertiKOS (PLDI’16) Ironclad Apps (OSDI’14) seL4 (SOSP’09) FSCQ (SOSP’15)

slide-4
SLIDE 4

This talk: automated (push-button) verification

4

Specification Implementation Automated verifier SMT formula

  • Trade-off: automation vs expressivity
  • No proofs on implementation
  • Requires bounded implementation
  • Restricts spec to first-order logic
  • Examples: Hyperkernel (SOSP’17), Nickel

(OSDI’18)

SMT solver

✔ ✘

slide-5
SLIDE 5

This talk: automated (push-button) verification

5

How to write and maintain automated verifiers? How to systematically fix verification bottlenecks? How to retrofit to existing systems?

Specification Implementation Automated verifier SMT formula SMT solver

✔ ✘

slide-6
SLIDE 6

Contributions

  • Serval: a framework for writing automated verifiers
  • Lift interpreters into verifiers: RISC-V, BPF, x86-32, LLVM
  • Symbolic optimization for repairing verification bottlenecks
  • Experience with Serval
  • Retrofitted CertiKOS and Komodo for automated verification
  • Found 18 new bugs in Linux BPF JIT and Keystone
  • Assumption: no guarantees on concurrency or side channels

6

slide-7
SLIDE 7

Verification stack

7

Z3

SMT solver: Constraint solving, counterexample generation Rosette: Symbolic evaluation, symbolic profiling, symbolic reflection Serval: Specification library, symbolic optimizations, machine code support RISC-V verifier x86-32 verifier LLVM verifier BPF verifier System specification x86-32 instructions LLVM instructions RISC-V instructions BPF instructions

slide-8
SLIDE 8

Verification stack

8

Z3

SMT solver: Constraint solving, counterexample generation Rosette: Symbolic evaluation, symbolic profiling, symbolic reflection Serval: Specification library, symbolic optimizations, machine code support RISC-V verifier x86-32 verifier LLVM verifier BPF verifier System specification x86-32 instructions LLVM instructions RISC-V instructions BPF instructions

slide-9
SLIDE 9

Verification stack

9

Z3

SMT solver: Constraint solving, counterexample generation Rosette: Symbolic evaluation, symbolic profiling, symbolic reflection Serval: Specification library, symbolic optimizations, machine code support System specification RISC-V verifier x86-32 verifier LLVM verifier BPF verifier x86-32 instructions LLVM instructions RISC-V instructions BPF instructions

slide-10
SLIDE 10

Verification stack

10

Z3

SMT solver: Constraint solving, counterexample generation Rosette: Symbolic evaluation, symbolic profiling, symbolic reflection Serval: Specification library, symbolic optimizations, machine code support System specification RISC-V verifier x86-32 verifier LLVM verifier BPF verifier x86-32 instructions LLVM instructions RISC-V instructions BPF instructions

slide-11
SLIDE 11

Verification stack

11

Z3

SMT solver: Constraint solving, counterexample generation Rosette: Symbolic evaluation, symbolic profiling, symbolic reflection Serval: Specification library, symbolic optimizations, machine code support System specification RISC-V verifier x86-32 verifier LLVM verifier BPF verifier x86-32 instructions LLVM instructions RISC-V instructions BPF instructions

slide-12
SLIDE 12

Verification stack

12

Z3

SMT solver: Constraint solving, counterexample generation Rosette: Symbolic evaluation, symbolic profiling, symbolic reflection Serval: Specification library, symbolic optimizations, machine code support System specification RISC-V verifier x86-32 verifier LLVM verifier BPF verifier x86-32 instructions LLVM instructions RISC-V instructions BPF instructions

slide-13
SLIDE 13

Verifier = interpreter + symbolic optimization

13

Write a verifier as interpreter Symbolic profiling to find bottleneck Develop / apply symbolic optimizations

✔

slide-14
SLIDE 14

Example: proving refinement for sign

14

Serval

RISC-V verifier Specification library

0: sltz a0 a1 1: bnez a1 4 2: sgtz a0 a0 3: ret 4: li a0 -1 5: ret

(define (sign x) (cond [(negative? x) -1] [(positive? x) 1] [(zero? x) 0]))

slide-15
SLIDE 15

Verifier [1/3]: writing an interpreter

15

RISC-V verifier x86-32 verifier System specification x86-32 instructions RISC-V instructions

(struct cpu (pc regs ...) #:mutable) (define (interpret c program) (define pc (cpu-pc c)) (define insn (fetch c program)) (match insn [('li rd imm) (set-cpu-pc! c (+ 1 pc)) (set-cpu-reg! c rd imm)] [('bnez rs imm) (if (! (= (cpu-reg c rs) 0)) (set-cpu-pc! c imm) (set-cpu-pc! c (+ 1 pc)))] ...))

slide-16
SLIDE 16

Verifier [1/3]: writing an interpreter

16

RISC-V verifier x86-32 verifier System specification x86-32 instructions RISC-V instructions

(struct cpu (pc regs ...) #:mutable) (define (interpret c program) (define pc (cpu-pc c)) (define insn (fetch c program)) (match insn [('li rd imm) (set-cpu-pc! c (+ 1 pc)) (set-cpu-reg! c rd imm)] [('bnez rs imm) (if (! (= (cpu-reg c rs) 0)) (set-cpu-pc! c imm) (set-cpu-pc! c (+ 1 pc)))] ...))

slide-17
SLIDE 17

(struct cpu (pc regs ...) #:mutable) (define (interpret c program) (define pc (cpu-pc c)) (define insn (fetch c program)) (match insn [('li rd imm) (set-cpu-pc! c (+ 1 pc)) (set-cpu-reg! c rd imm)] [('bnez rs imm) (if (! (= (cpu-reg c rs) 0)) (set-cpu-pc! c imm) (set-cpu-pc! c (+ 1 pc)))] ...))

Verifier [1/3]: writing an interpreter

17

slide-18
SLIDE 18

(struct cpu (pc regs ...) #:mutable) (define (interpret c program) (define pc (cpu-pc c)) (define insn (fetch c program)) (match insn [('li rd imm) (set-cpu-pc! c (+ 1 pc)) (set-cpu-reg! c rd imm)] [('bnez rs imm) (if (! (= (cpu-reg c rs) 0)) (set-cpu-pc! c imm) (set-cpu-pc! c (+ 1 pc)))] ...))

Verifier [1/3]: writing an interpreter

18

slide-19
SLIDE 19

(struct cpu (pc regs ...) #:mutable) (define (interpret c program) (define pc (cpu-pc c)) (define insn (fetch c program)) (match insn [('li rd imm) (set-cpu-pc! c (+ 1 pc)) (set-cpu-reg! c rd imm)] [('bnez rs imm) (if (! (= (cpu-reg c rs) 0)) (set-cpu-pc! c imm) (set-cpu-pc! c (+ 1 pc)))] ...))

Verifier [1/3]: writing an interpreter

19

  • Natural to write
  • Easy to audit
  • Can reuse CPU test suite
slide-20
SLIDE 20

Verifier [2/3]: identifying bottlenecks in symbolic evaluation

20

Serval

RISC-V verifier Specification library

0: sltz a0 a1 1: bnez a1 4 2: sgtz a0 a0 3: ret 4: li a0 -1 5: ret

(define (sign x) (cond [(negative? x) -1] [(positive? x) 1] [(zero? x) 0]))

slide-21
SLIDE 21

Verifier [2/3]: identifying bottlenecks in symbolic evaluation

21

Serval

RISC-V verifier Specification library

0: sltz a0 a1 1: bnez a1 4 2: sgtz a0 a0 3: ret 4: li a0 -1 5: ret

(define (sign x) (cond [(negative? x) -1] [(positive? x) 1] [(zero? x) 0])) Slow / Timeout

slide-22
SLIDE 22

Verifier [2/3]: identifying bottlenecks in symbolic evaluation

22

slide-23
SLIDE 23

Bottleneck: state explosion due to symbolic PC

23

0: sltz a0 a1 1: bnez a1 4 2: sgtz a0 a0 3: ret 4: li a0 -1 5: ret

(struct cpu (pc regs) #:mutable) (define (interpret c program) (define pc (cpu-pc c)) (define insn (fetch c program)) (match insn [('li rd imm) (set-cpu-pc! c (+ 1 pc)) (set-cpu-reg! c rd imm)] [('bnez rs imm) (if (! (= (cpu-reg c rs) 0)) (set-cpu-pc! c imm) (set-cpu-pc! c (+ 1 pc)))] ...))

slide-24
SLIDE 24

(struct cpu (pc regs) #:mutable) (define (interpret c program) (define pc (cpu-pc c)) (define insn (fetch c program)) (match insn [('li rd imm) (set-cpu-pc! c (+ 1 pc)) (set-cpu-reg! c rd imm)] [('bnez rs imm) (if (! (= (cpu-reg c rs) 0)) (set-cpu-pc! c imm) (set-cpu-pc! c (+ 1 pc)))] ...))

Bottleneck: state explosion due to symbolic PC

24

0: sltz a0 a1 1: bnez a1 4 2: sgtz a0 a0 3: ret 4: li a0 -1 5: ret

v0 : X < 0 v1 : ¬v0 v2 : ite(v0, 1, 0) v3 : ite(v0, 4, 2) v4 : ite(v0, li, sgtz) v5 : ite(v0, # f, 0) v6 : ite(v0, 1, # f) v7 : ite(v0, 5, 3) v8 : X > 0 v9 : ¬v8 v10 : ite(v8, 1, 0) v11 : ite(v0, 1, v10) c 7! cpu(0, X, Y ) i 7! (sltz 1 0 # f) B c 7! cpu(1, X, 1) I c 7! cpu(1, X, 0) J c 7! cpu(1, X, v2) i 7! (bnez # f 1 4) B v0 v1 c 7! cpu(4, X, v2) E c 7! cpu(2, X, v2) F c 7! cpu(v3, X, v2) A v0 v1 c 7! cpu(v3, X, v2) i 7! (v4 0 v5 v6) B v4 = bnez v4 = sltz vector-ref v1 v0 v3 = 0 v3 = 1 v3 = 3 v3 = 5 C

slide-25
SLIDE 25

Bottleneck: state explosion due to symbolic PC

25

0: sltz a0 a1 1: bnez a1 4 2: sgtz a0 a0 3: ret 4: li a0 -1 5: ret

(struct cpu (pc regs) #:mutable) (define (interpret c program) (define pc (cpu-pc c)) (define insn (fetch c program)) (match insn [('li rd imm) (set-cpu-pc! c (+ 1 pc)) (set-cpu-reg! c rd imm)] [('bnez rs imm) (if (! (= (cpu-reg c rs) 0)) (set-cpu-pc! c imm) (set-cpu-pc! c (+ 1 pc)))] ...))

  • Conditional jump
slide-26
SLIDE 26

(struct cpu (pc regs) #:mutable) (define (interpret c program) (define pc (cpu-pc c)) (define insn (fetch c program)) (match insn [('li rd imm) (set-cpu-pc! c (+ 1 pc)) (set-cpu-reg! c rd imm)] [('bnez rs imm) (if (! (= (cpu-reg c rs) 0)) (set-cpu-pc! c imm) (set-cpu-pc! c (+ 1 pc)))] ...))

Bottleneck: state explosion due to symbolic PC

26

0: sltz a0 a1 1: bnez a1 4 2: sgtz a0 a0 3: ret 4: li a0 -1 5: ret

c 7! cpu(v3, X, v2) A c 7! cpu(v3, X, v2) i 7! (v4 0 v5 v6) B c 7! cpu(v7, 1, v2) L c 7! cpu(v3, X, v2) c 7! cpu(v7, 1, v2) G c 7! cpu(v7, 0, v2) H c 7! cpu(v7, v10, v2) c 7! cpu(v7, v11, v2) A c 7! cpu(v7, v11, v2) i 7! (ret # f # f # f) B (v4 = li) ⌘ v0 (v4 = sgtz) ⌘ v1 v4 = ret v4 = bnez v4 = sltz v8 v9 vector-ref v1 v0 v3 = 0 v3 = 1 v3 = 3 v3 = 5 C vector-ref v9 v8 v7 = 0 v7 = 1 v7 = 2 v7 = 4 C

slide-27
SLIDE 27

Verifier [3/3]: Repairing with symbolic optimizations

27

  • Symbolic optimization:
  • “Peephole” optimization on symbolic state
  • Fine-tune symbolic evaluation
  • Use domain knowledge
  • Serval provides set of symbolic optimizations for verifiers
slide-28
SLIDE 28

Verifier [3/3]: Repairing with symbolic optimizations

28

(struct cpu (pc regs) #:mutable) (define (interpret c program)

  • (define pc (cpu-pc c))

(define insn (fetch c program)) (match insn ...)) (struct cpu (pc regs) #:mutable) (define (interpret c program) + (serval:split-pc [cpu pc] c (define insn (fetch c program)) (match insn ...)))

Developer uses symbolic

  • ptimization
slide-29
SLIDE 29

Verifier [3/3]: Repairing with symbolic optimizations

29

3× fewer merges

v0 : X < 0 v1 : ¬v0 v2 : ite(v0, 1, 0) v3 : ite(v0, 4, 2) v4 : ite(v0, li, sgtz) v5 : ite(v0, # f, 0) v6 : ite(v0, 1, # f) v7 : ite(v0, 5, 3) v8 : X > 0 v9 : ¬v8 v10 : ite(v8, 1, 0) v11 : ite(v0, 1, v10) c 7! cpu(0, X, Y ) i 7! (sltz 1 0 # f) B c 7! cpu(1, X, 1) I c 7! cpu(1, X, 0) J c 7! cpu(1, X, v2) i 7! (bnez # f 1 4) B v0 v1 c 7! cpu(4, X, v2) E c 7! cpu(2, X, v2) F c 7! cpu(v3, X, v2) A v0 v1 c 7! cpu(v3, X, v2) i 7! (v4 0 v5 v6) B c 7! cpu(v7, 1, v2) L c 7! cpu(v3, X, v2) c 7! cpu(v7, 1, v2) G c 7! cpu(v7, 0, v2) H c 7! cpu(v7, v10, v2) c 7! cpu(v7, v11, v2) A c 7! cpu(v7, v11, v2) i 7! (ret # f # f # f) B c 7! cpu(0, v11, v2) D (v4 = li) ⌘ v0 (v4 = sgtz) ⌘ v1 v4 = ret v4 = bnez v4 = sltz v8 v9 vector-ref v1 v0 v3 = 0 v3 = 1 v3 = 3 v3 = 5 C vector-ref v9 v8 v7 = 0 v7 = 1 v7 = 2 v7 = 4 C

v0 : X < 0 v1 : ¬v0 v2 : ite(v0, 1, 0) v3 : ite(v0, 4, 2) v4 : X > 0 v5 : ¬X > 0 v6 : ite(v4, 1, 0) v7 : ite(v0, 1, v6) c 7! cpu(0, X, Y ) i 7! (sltz 1 0 # f) B c 7! cpu(1, X, 1) I c 7! cpu(1, X, 0) J c 7! cpu(1, X, v2) i 7! (bnez # f 1 4) B v0 v1 c 7! cpu(4, X, v2) E c 7! cpu(2, X, v2) F c 7! cpu(v3, X, v2) O v0 v1 c 7! cpu(4, X, v2) i 7! (li 0 # f 1) B v0 c 7! cpu(5, 1, v2) i 7! (ret # f # f # f) B c 7! cpu(0, 1, v2) D c 7! cpu(2, X, v2) i 7! (sgtz 0 0 # f) B v1 c 7! cpu(3, 1, v2) G c 7! cpu(3, 0, v2) H c 7! cpu(3, v6, v2) i 7! (ret # f # f # f) B v4 v5 c 7! cpu(0, v6, v2) D c 7! cpu(0, v7, v2)

slide-30
SLIDE 30

Verifier summary

  • Verifier = interpreter + symbolic optimizations
  • Easy to test verifiers
  • Systematic way to scale symbolic evaluation
  • Caveats:
  • Symbolic profiling cannot identify expensive SMT operations
  • Repair requires expertise

30

slide-31
SLIDE 31

Retrofitting previously verified security monitors

  • Port CertiKOS (PLDI’16) and Komodo (SOSP’17) to RISC-V
  • Prove functional correctness and noninterference

31

slide-32
SLIDE 32

Retrofitting overview

32

Serval RISC-V verifier System specification System implementation

Is the specification expressible in Serval? Is the implementation free of unbounded loops?

slide-33
SLIDE 33

CertiKOS (PLDI’16)

  • OS kernel providing strict isolation
  • Physical memory quota, partitioned PIDs

33

CertiKOS Process Process Process

slide-34
SLIDE 34

Retrofitting: implementation

34

  • CertiKOS interface already expressible without unbounded loops
  • Tweak spawn system call to close potential information leaks
  • Did not account for memory consumed by ELF loading
  • Leaked number of children
slide-35
SLIDE 35

Retrofitting: specification

  • CertiKOS specifies noninterference using traces of unbounded length
  • Broken down into 3 properties of individual “actions”
  • Local action, yield to another process, yield back
  • We reuse the properties as our noninterference specification
  • We also prove noninterference spec as in Nickel (OSDI’18)

35

slide-36
SLIDE 36

Retrofitting summary

  • Security monitors good fit for automated verification
  • No unbounded loops
  • No inductive data structures
  • Verify binary images directly
  • Develop in standard languages (C / Asm)
  • No need to trust compiler / linker / etc.

36

slide-37
SLIDE 37

15 new bugs found in Linux BPF JIT

37

Serval: Specification library, symbolic optimizations, machine code support RISC-V verifier x86-32 verifier LLVM verifier BPF verifier System specification x86-32 instructions LLVM instructions RISC-V instructions BPF instructions

slide-38
SLIDE 38

Conclusion

  • Writing automated verifiers using lifting
  • A systematic method for scaling symbolic evaluation
  • Retrofit Serval to verify existing systems
  • Come to SOSP to learn more
  • For paper and more info:
  • https://serval.unsat.systems

38