Scaling symbolic evaluation for automated verification of systems - - PowerPoint PPT Presentation

▶
scaling symbolic evaluation for automated verification of
SMART_READER_LITE
LIVE PREVIEW

Scaling symbolic evaluation for automated verification of systems - - PowerPoint PPT Presentation

Jan 20, 2023 232 likes •662 views

Scaling symbolic evaluation for automated verification of systems code with Serval Luke Nelson , James Bornholt , Ronghui Gu , Andrew Baumann , Emina Torlak , Xi Wang University of Washington, Columbia University,

slide-1
SLIDE 1

Scaling symbolic evaluation for automated verification of systems code with Serval

Luke Nelson¹, James Bornholt¹, Ronghui Gu², Andrew Baumann³, Emina Torlak¹, Xi Wang¹ ¹University of Washington, ²Columbia University, ³Microsoft Research

1

slide-2
SLIDE 2

Eliminating bugs with formal verification

2

OS Kernel / security monitor Process Process Process

seL4 (SOSP’09) Ironclad Apps (OSDI’14) FSCQ (SOSP’15) CertiKOS (PLDI’16) Komodo (SOSP’17)

slide-3
SLIDE 3

Eliminating bugs with formal verification

3

OS Kernel / security monitor Process Process Process

seL4 (SOSP’09) Ironclad Apps (OSDI’14) FSCQ (SOSP’15) CertiKOS (PLDI’16) Komodo (SOSP’17)

  • Strong correctness guarantees
  • Require manual proofs
  • CertiKOS 200k lines of proof
  • Multiple person-years
slide-4
SLIDE 4

Prior work: automated (push-button) verification

4

Specification Implementation Automated verifier SMT formula

  • No proofs on implementation
  • Requires bounded implementation
  • Restricts specification

Example: Hyperkernel (SOSP’17)

SMT solver

✔ ✘

slide-5
SLIDE 5

Challenges

5

Specification Implementation Automated verifier SMT formula SMT solver

✔ ✘

How to lower effort of writing automated verifiers? How to find and fix performance bottlenecks? How to retrofit to existing systems?

slide-6
SLIDE 6

Contributions

  • Serval: a framework for writing automated verifiers
  • RISC-V, x86-32, LLVM, BPF
  • Scaling via symbolic optimizations
  • Experience
  • Retrofitted CertiKOS and Komodo for Serval
  • Found 15 new bugs in Linux BPF JIT

6

slide-7
SLIDE 7

Contributions

  • Serval: a framework for writing automated verifiers
  • RISC-V, x86-32, LLVM, BPF
  • Scaling via symbolic optimizations
  • Experience
  • Retrofitted CertiKOS and Komodo for Serval
  • Found 15 new bugs in Linux BPF JIT

7

no guarantees on concurrency or side channels

slide-8
SLIDE 8

Verifying a system with Serval

8

Z3

SMT solver Rosette Serval

RISC-V verifier System specification RISC-V instructions

slide-9
SLIDE 9

Verifying a system with Serval

9

Z3

SMT solver Rosette Serval

RISC-V verifier RISC-V instructions System specification

slide-10
SLIDE 10

Verifying a system with Serval

10

SMT solver Rosette Serval

RISC-V verifier RISC-V instructions

Z3

System specification

slide-11
SLIDE 11

Verifying a system with Serval

11

Z3

SMT solver Rosette Serval

RISC-V verifier RISC-V instructions System specification

slide-12
SLIDE 12

Verifying a system with Serval

12

Z3

SMT solver Rosette Serval

RISC-V verifier RISC-V instructions System specification

slide-13
SLIDE 13

Verifying a system with Serval

13

Z3

SMT solver Rosette Serval

RISC-V verifier RISC-V instructions System specification

slide-14
SLIDE 14

Example: proving refinement for sign

14

Serval

RISC-V verifier

(define (sign x) (cond [(negative? x) -1] [(positive? x) 1] [(zero? x) 0]))

0: sltz a1 a0 1: bnez a1 4 2: sgtz a0 a0 3: ret 4: li a0 -1 5: ret

slide-15
SLIDE 15

Verifier = interpreter + symbolic optimization

15

  • 1. Write a verifier

as interpreter

  • 2. Symbolic profiling

to find bottleneck

  • 3. Apply symbolic
  • ptimizations

✔

slide-16
SLIDE 16

Verifier [1/3]: writing an interpreter

16

RISC-V verifier x86-32 verifier System specification x86-32 instructions RISC-V instructions

(struct cpu (pc regs ...) #:mutable) (define (interpret c program) (define pc (cpu-pc c)) (define insn (fetch pc program)) (match insn [('li rd imm) (set-cpu-pc! c (+ 1 pc)) (set-cpu-reg! c rd imm)] [('bnez rs imm) (if (! (= (cpu-reg c rs) 0)) (set-cpu-pc! c imm) (set-cpu-pc! c (+ 1 pc)))] ...))

slide-17
SLIDE 17

Verifier [1/3]: writing an interpreter

17

RISC-V verifier x86-32 verifier System specification x86-32 instructions RISC-V instructions

(struct cpu (pc regs ...) #:mutable) (define (interpret c program) (define pc (cpu-pc c)) (define insn (fetch pc program)) (match insn [('li rd imm) (set-cpu-pc! c (+ 1 pc)) (set-cpu-reg! c rd imm)] [('bnez rs imm) (if (! (= (cpu-reg c rs) 0)) (set-cpu-pc! c imm) (set-cpu-pc! c (+ 1 pc)))] ...))

slide-18
SLIDE 18

(struct cpu (pc regs ...) #:mutable) (define (interpret c program) (define pc (cpu-pc c)) (define insn (fetch pc program)) (match insn [('li rd imm) (set-cpu-pc! c (+ 1 pc)) (set-cpu-reg! c rd imm)] [('bnez rs imm) (if (! (= (cpu-reg c rs) 0)) (set-cpu-pc! c imm) (set-cpu-pc! c (+ 1 pc)))] ...))

Verifier [1/3]: writing an interpreter

18

slide-19
SLIDE 19

(struct cpu (pc regs ...) #:mutable) (define (interpret c program) (define pc (cpu-pc c)) (define insn (fetch pc program)) (match insn [('li rd imm) (set-cpu-pc! c (+ 1 pc)) (set-cpu-reg! c rd imm)] [('bnez rs imm) (if (! (= (cpu-reg c rs) 0)) (set-cpu-pc! c imm) (set-cpu-pc! c (+ 1 pc)))] ...))

Verifier [1/3]: writing an interpreter

19

slide-20
SLIDE 20

(struct cpu (pc regs ...) #:mutable) (define (interpret c program) (define pc (cpu-pc c)) (define insn (fetch pc program)) (match insn [('li rd imm) (set-cpu-pc! c (+ 1 pc)) (set-cpu-reg! c rd imm)] [('bnez rs imm) (if (! (= (cpu-reg c rs) 0)) (set-cpu-pc! c imm) (set-cpu-pc! c (+ 1 pc)))] ...))

Verifier [1/3]: writing an interpreter

20

  • Easy to write
  • Reuse CPU test suite
slide-21
SLIDE 21

Verifier [2/3]: identifying bottlenecks in symbolic evaluation

21

Serval

RISC-V verifier

(define (sign x) (cond [(negative? x) -1] [(positive? x) 1] [(zero? x) 0]))

😁

0: sltz a1 a0 1: bnez a1 4 2: sgtz a0 a0 3: ret 4: li a0 -1 5: ret

slide-22
SLIDE 22

0: sltz a1 a0 1: bnez a1 4 2: sgtz a0 a0 3: ret 4: li a0 -1 5: ret

(define (sign x) (cond [(negative? x) -1] [(positive? x) 1] [(zero? x) 0]))

Verifier [2/3]: identifying bottlenecks in symbolic evaluation

22

Serval

RISC-V verifier

Slow/Timeout

🤰

slide-23
SLIDE 23

Verifier [2/3]: identifying bottlenecks in symbolic evaluation

23

slide-24
SLIDE 24

Verifier [2/3]: identifying bottlenecks in symbolic evaluation

24

fetch

slide-25
SLIDE 25

Verifier [2/3]: identifying bottlenecks in symbolic evaluation

25

0: sltz a1 a0 1: bnez a1 4 2: sgtz a0 a0 3: ret 4: li a0 -1 5: ret

(struct cpu (pc regs) #:mutable) (define (interpret c program) (define pc (cpu-pc c)) (define insn (fetch pc program)) (match insn [('li rd imm) (set-cpu-pc! c (+ 1 pc)) (set-cpu-reg! c rd imm)] [('bnez rs imm) (if (! (= (cpu-reg c rs) 0)) (set-cpu-pc! c imm) (set-cpu-pc! c (+ 1 pc)))] ...))

slide-26
SLIDE 26

Merge states to avoid path explosion

26

0: sltz a1 a0 1: bnez a1 4 2: sgtz a0 a0 3: ret 4: li a0 -1 5: ret

PC → 0 a0 → X a1 → Y PC → 1 a0 → X a1 → 1 PC → 1 a0 → X a1 → 0 PC → 1 a0 → X a1 → if(X < 0, 1, 0)

¬(X < 0) X < 0

slide-27
SLIDE 27

0: sltz a1 a0 1: bnez a1 4 2: sgtz a0 a0 3: ret 4: li a0 -1 5: ret

Bottleneck: state explosion due to symbolic PC

27

Conditional jump PC → 1 a0 → X a1 → if(X < 0, 1, 0) PC → if(X < 0, 4, 2) a0 → X a1 → if(X < 0, 1, 0)

... ...

slide-28
SLIDE 28

0: sltz a1 a0 1: bnez a1 4 2: sgtz a0 a0 3: ret 4: li a0 -1 5: ret

Bottleneck: state explosion due to symbolic PC

28

Conditional jump PC → if(...) a0 → X a1 → if(...) PC → 1 PC → 3 PC → 4 PC → 2 PC → 5 PC → 0

slide-29
SLIDE 29

Verifier [3/3]: Repairing with symbolic optimizations

29

  • Symbolic optimization:
  • “Peephole” optimization on symbolic state
  • Fine-tune symbolic evaluation
  • Use domain knowledge
  • Serval provides set of symbolic optimizations for verifiers
slide-30
SLIDE 30

Verifier [3/3]: Repairing with symbolic optimizations

30

(define (interpret c program)

  • (define pc (cpu-pc c))

(define insn (fetch pc program)) (match insn ...)) (define (interpret c program) + (serval:split-pc [cpu pc] c (define insn (fetch pc program)) (match insn ...)))

  • Match on symbolic structure of PC
  • Evaluate separately using each concrete PC value
  • Merge states afterwards
slide-31
SLIDE 31

Verifier [3/3]: Repairing with symbolic optimizations

31

PC → if(X < 0, 4, 2) a0 → X a1 → if(...) PC → 4 PC → 2 PC → if(X < 0, 4, 2) a0 → X a1 → if(...) PC → 1 PC → 3 PC → 4 PC → 2 PC → 5 PC → 0 split-pc

slide-32
SLIDE 32

Verifier [3/3]: Repairing with symbolic optimizations

32

PC → if(X < 0, 4, 2) a0 → X a1 → if(...) PC → 4 PC → 2 PC → if(X < 0, 4, 2) a0 → X a1 → if(...) PC → 1 PC → 3 PC → 4 PC → 2 PC → 5 PC → 0

Domain knowledge:

  • Split PC to avoid state explosion
  • Merge other registers to avoid path explosion
slide-33
SLIDE 33

Verifier summary

  • Verifier = interpreter + symbolic optimizations
  • Easy to test verifiers
  • Systematic way to scale symbolic evaluation
  • Caveats:
  • Symbolic profiling cannot identify expensive SMT operations
  • Repair requires expertise

33

slide-34
SLIDE 34

Implementation

34

Z3

SMT solver Rosette Serval

RISC-V verifier x86-32 verifier LLVM verifier BPF verifier

slide-35
SLIDE 35

Experience

35

  • Can existing systems be retrofitted for Serval?
  • Are Serval’s verifiers reusable?
slide-36
SLIDE 36

Retrofitting previously verified security monitors

  • Port CertiKOS (PLDI’16) and Komodo (SOSP’17) to RISC-V
  • Retrofit the systems to automated verification
  • Apply the RISC-V verifier to binary image
  • Prove functional correctness and noninterference
  • ≈4 weeks each

36

slide-37
SLIDE 37

Retrofitting overview

37

System specification System implementation

Is the specification expressible in Serval? Is the implementation free of unbounded loops?

slide-38
SLIDE 38

Example: retrofitting CertiKOS

  • OS kernel providing strict isolation
  • Physical memory quota, partitioned PIDs
  • Security specification: noninterference

38

CertiKOS Process Process Process

slide-39
SLIDE 39

Example: retrofitting CertiKOS

  • Implementation
  • Already free of unbounded loops
  • Tweak spawn to close two potential information leaks
  • Specification
  • Noninterference using traces of unbounded length
  • Broken down into 3 properties of individual “actions”

39

slide-40
SLIDE 40

Retrofitting summary

  • Security monitors good fit for automated verification
  • No unbounded loops
  • No inductive data structures

40

slide-41
SLIDE 41

Reusing verifiers to find bugs

41

  • Combine RISC-V, x86-32, and BPF verifiers
  • Found 15 bugs in the Linux kernel’s BPF JIT compiler
  • Bug fixes and new tests upstreamed
slide-42
SLIDE 42

Conclusion

  • Writing automated verifiers using lifting
  • A systematic method for scaling symbolic evaluation
  • Retrofit Serval to verify existing systems
  • For paper and more info:
  • https://serval.unsat.systems

42