[PPT] - CSE507 Computer-Aided Reasoning for Software Solver-Aided Languages PowerPoint Presentation

SLIDE 1

CSE507

Emina Torlak

emina@cs.washington.edu

courses.cs.washington.edu/courses/cse507/14au/

Computer-Aided Reasoning for Software

Solver-Aided Languages

SLIDE 2

Today

2

SLIDE 3

Today

2

Last lecture

Program synthesis

SLIDE 4

Today

2

Last lecture

Program synthesis

Today

The next N years: Solver-Aided Languages (?)

SLIDE 5

Today

2

Last lecture

Program synthesis

Today

The next N years: Solver-Aided Languages (?)

Reminders

Please fill out the course evaluation form (Dec 02-08)
8 min final presentations on Monday, Dec 08, 10:30am, MGH 254
Final projects due on Monday, Dec 08, at 11pm

SLIDE 6

vision

a little programming for everyone

SLIDE 7

A little programming for everyone

4 We all want to build programs …

SLIDE 8

spreadsheet data manipulation [Flashfill, POPL’11]

A little programming for everyone

4 We all want to build programs … social scientist

SLIDE 9

spreadsheet data manipulation [Flashfill, POPL’11]
models of cell fates [SBL, POPL’13]

A little programming for everyone

4 We all want to build programs … biologist social scientist

SLIDE 10

spreadsheet data manipulation [Flashfill, POPL’11]
models of cell fates [SBL, POPL’13]
cache coherence protocols [Transit, PLDI’13]
memory models [MemSAT, PLDI’10]

A little programming for everyone

4 We all want to build programs … hardware designer biologist social scientist

SLIDE 11

spreadsheet data manipulation [Flashfill, POPL’11]
models of cell fates [SBL, POPL’13]
cache coherence protocols [Transit, PLDI’13]
memory models [MemSAT, PLDI’10]

A little programming for everyone

4 code time effort We all want to build programs … hardware designer biologist social scientist

SLIDE 12

A little programming for everyone

5 less code less time less effort solver-aided languages We all want to build programs …

spreadsheet data manipulation
models of cell fates
cache coherence protocols
memory models

hardware designer biologist social scientist

SLIDE 13 software crisis program logics (Floyd, Hoare, Dijkstra) mechanization of logic (Milner, Pnueli) mechanized tools (Clarke, Emerson, Sifakis) software gap SAT/SMT solvers and tools solver-aided languages

A little history

6 better programs

SLIDE 14 software crisis program logics (Floyd, Hoare, Dijkstra) mechanization of logic (Milner, Pnueli) mechanized tools (Clarke, Emerson, Sifakis) software gap SAT/SMT solvers and tools solver-aided languages

A little history

6 better programs 1960 1970 1980 1990 2000

SLIDE 15 software crisis program logics (Floyd, Hoare, Dijkstra) mechanization of logic (Milner, Pnueli) mechanized tools (Clarke, Emerson, Sifakis) software gap SAT/SMT solvers and tools solver-aided languages

A little history

6 better programs 1960 1970 1980 1990 2000 ASTRÉE [AbsInt] SLAM [MSR] 6TH SENSE [IBM]

SLIDE 16 software crisis program logics (Floyd, Hoare, Dijkstra) mechanization of logic (Milner, Pnueli) mechanized tools (Clarke, Emerson, Sifakis) software gap SAT/SMT solvers and tools solver-aided languages

A little history

6 better programs 1960 1970 1980 1990 2000

SLIDE 17 software crisis program logics (Floyd, Hoare, Dijkstra) mechanization of logic (Milner, Pnueli) mechanized tools (Clarke, Emerson, Sifakis) software gap SAT/SMT solvers and tools solver-aided languages

A little history

6 better programs 1960 1970 1980 1990 2000

SLIDE 18 software crisis program logics (Floyd, Hoare, Dijkstra) mechanization of logic (Milner, Pnueli) mechanized tools (Clarke, Emerson, Sifakis) software gap SAT/SMT solvers and tools solver-aided languages

A little history

6 better programs more easily 1960 1970 1980 1990 2000 2010

SLIDE 19

utline

solver-aided tools, languages and beyond

SLIDE 20

utline

solver-aided tools, languages and beyond solver-aided tools

SLIDE 21

utline

solver-aided tools, languages and beyond solver-aided tools, languages

SLIDE 22

utline

solver-aided tools, languages and beyond solver-aided tools, languages and beyond

SLIDE 23

story

solver-aided tools

SLIDE 24 P(x) { … … }

Programming …

9 specification

SLIDE 25 P(x) { … … }

Programming …

9 specification test case assert safe(P(2))

SLIDE 26

Programming with a solver-aided tool

10

?

SAT/SMT solver translate(…) P(x) { … … } assert safe(P(x))

SLIDE 27

Solver-aided tools: verification

11 Find an input

n which the

program fails. ∃x . ¬ safe(P(x)) SAT/SMT solver P(x) { … … } assert safe(P(x)) CBMC [Kroening et al., DAC’03] Dafny [Leino, LPAR’10] Miniatur [Vaziri et al., FSE’07] Klee [Cadar et al., OSDI’08]

SLIDE 28

Solver-aided tools: verification

11 Find an input

n which the

program fails. ∃x . ¬ safe(P(x)) model values SAT/SMT solver P(x) { … … } assert safe(P(x)) 42 CBMC [Kroening et al., DAC’03] Dafny [Leino, LPAR’10] Miniatur [Vaziri et al., FSE’07] Klee [Cadar et al., OSDI’08]

SLIDE 29 SAT/SMT solver

Solver-aided tools: debugging

12 Localize bad parts of the program. x = 42 ⋀ safe(P(x)) P(x) { v = x + 2 … } assert safe(P(x)) 42 BugAssist [Jose & Majumdar, PLDI’11] Angelina [Chandra et al., ICSE’11]

SLIDE 30 SAT/SMT solver

Solver-aided tools: debugging

12 min core Localize bad parts of the program. x = 42 ⋀ safe(P(x)) P(x) { v = x + 2 … } assert safe(P(x)) x + 2 42 expressions BugAssist [Jose & Majumdar, PLDI’11] Angelina [Chandra et al., ICSE’11]

SLIDE 31

Solver-aided tools: angelic execution

13 Find values that repair the failing execution.

∃v . safe(P(42, v))

SAT/SMT solver P(x) { v = choose() … } assert safe(P(x)) 42 Kaplan [Koksal et al, POPL’12] PBnJ [Samimi et al., ECOOP’10] Squander [Milicevic et al., ICSE’11]

SLIDE 32

Solver-aided tools: angelic execution

13 Find values that repair the failing execution.

∃v . safe(P(42, v))

model values SAT/SMT solver P(x) { v = choose() … } assert safe(P(x)) 42 40 Kaplan [Koksal et al, POPL’12] PBnJ [Samimi et al., ECOOP’10] Squander [Milicevic et al., ICSE’11]

SLIDE 33 P(x) { v = ?? … } assert safe(P(x))

Solver-aided tools: synthesis

14 Synthesize code that repairs the program.

∃e . ∀x . safe(Pe(x))

SAT/SMT solver Sketch [Solar-Lezama et al., ASPLOS’06] Comfusy [Kuncak et al., CAV’10]

SLIDE 34 P(x) { v = ?? … } assert safe(P(x))

Solver-aided tools: synthesis

14 Synthesize code that repairs the program.

∃e . ∀x . safe(Pe(x))

SAT/SMT solver x − 2 model expressions Sketch [Solar-Lezama et al., ASPLOS’06] Comfusy [Kuncak et al., CAV’10]

SLIDE 35

wanted

more solver-aided tools …

SLIDE 36

wanted

more solver-aided tools …

SLIDE 37

Building solver-aided tools: state-of-the-art

16 tools expert execute synth

SLIDE 38

Building solver-aided tools: state-of-the-art

16 learn the problem domain I need a tool to create models of biological cells … tools expert domain expert execute synth

SLIDE 39

Building solver-aided tools: state-of-the-art

16 learn the problem domain design a domain language I need a tool to create models of biological cells … Abstractions for cells, components, interactions, … tools expert domain expert execute synth

SLIDE 40 build a symbolic compiler from the domain language to constraints

Building solver-aided tools: state-of-the-art

16 learn the problem domain design a domain language months

r years
f work

I need a tool to create models of biological cells … tools expert domain expert verify debug execute synth A solver-aided tool for creating biological models

SLIDE 41

Can we do better?

17 design a domain language weeks domain expert verify debug execute synth implement an interpreter for the language, get a symbolic compiler for free

SLIDE 42

Can we do better?

17 design a domain language weeks domain expert a solver-aided host language a solver-aided domain-specific language (SDSL) verify debug execute synth implement an interpreter for the language, get a symbolic compiler for free

SLIDE 43

design

solver-aided languages

SLIDE 44 domain-specific language (DSL)

Layers of languages

19 host language A formal language that is specialized to a particular application domain and often limited in capability. A high-level language for implementing DSLs, usually with meta-programming features. interpreter library

SLIDE 45 Scala, Racket, JavaScript domain-specific language (DSL) artificial intelligence Church, BLOG databases SQL, Datalog hardware design Bluespec, Chisel, Verilog, VHDL math and statistics Eigen, Matlab, R layout and visualization LaTex, dot, dygraphs, D3

Layers of languages

19 host language interpreter library

SLIDE 46 domain-specific language (DSL)

Layers of languages

19 host language C = A * B C / Java Eigen / Matlab [associativity] C = A * B for (i = 0; i < n; i++) for (j = 0; j < m; j++) for (k = 0; k < p; k++) C[i][k] += A[i][j] * B[j][k] interpreter library

SLIDE 47 solver-aided domain-specific language (SDSL)

Layers of solver-aided languages

20 solver-aided host language symbolic virtual machine interpreter library

SLIDE 48 solver-aided domain-specific language (SDSL)

Layers of solver-aided languages

20 solver-aided host language symbolic virtual machine

ROSETTE

[Torlak & Bodik, Onward’13, PLDI’14] interpreter library

SLIDE 49 spatial programming Chlorophyll data-parallel programming SynthCL web scraping WebSynth secure stack machines IFC solver-aided domain-specific language (SDSL)

Layers of solver-aided languages

20 solver-aided host language symbolic virtual machine

ROSETTE

[Torlak & Bodik, Onward’13, PLDI’14] interpreter library

SLIDE 50 development time (weeks) 4 8 12 16 Chlorophyll SynthCL WebSynth IFC

SDSLs developed with ROSETTE

21 (first-year grad) (expert) (undergrad) (expert)

SLIDE 51 development time (weeks) 4 8 12 16 Chlorophyll SynthCL WebSynth IFC Spatial programming for a low-power chip, using synthesis to partition code and data across 144 tiny cores.

SDSLs developed with ROSETTE

21 (first-year grad) (expert) (undergrad) (expert) GreenArrays GA144

x + z

SLIDE 52 development time (weeks) 4 8 12 16 Chlorophyll SynthCL WebSynth IFC Spatial programming for a low-power chip, using synthesis to partition code and data across 144 tiny cores. Optimal partitioning synthesized in minutes, while manual partitioning takes days [Phothilimthana et al., PLDI’14].

SDSLs developed with ROSETTE

21 (first-year grad) (expert) (undergrad) (expert) GreenArrays GA144

x + z x + z x + z

SLIDE 53 development time (weeks) 4 8 12 16 Chlorophyll SynthCL WebSynth IFC Verification and synthesis for data-parallel programming with OpenCL.

SDSLs developed with ROSETTE

21 (first-year grad) (expert) (undergrad) (expert)

SLIDE 54 development time (weeks) 4 8 12 16 Chlorophyll SynthCL WebSynth IFC Verification and synthesis for data-parallel programming with OpenCL. Used by a novice to develop new vectorized kernels that are as fast as expert code.

SDSLs developed with ROSETTE

21 (first-year grad) (expert) (undergrad) (expert)

SLIDE 55 development time (weeks) 4 8 12 16 Chlorophyll SynthCL WebSynth IFC Synthesis of web scraping scripts from examples (PBE).

SDSLs developed with ROSETTE

21 (first-year grad) (expert) (undergrad) (expert)

SLIDE 56 development time (weeks) 4 8 12 16 Chlorophyll SynthCL WebSynth IFC Synthesis of web scraping scripts from examples (PBE). Works on real web pages (e.g., iTunes) in seconds.

SDSLs developed with ROSETTE

21 (first-year grad) (expert) (undergrad) (expert)

SLIDE 57 development time (weeks) 4 8 12 16 Chlorophyll SynthCL WebSynth IFC Verification for executable specifications

f secure stack machines.

SDSLs developed with ROSETTE

21 (first-year grad) (expert) (undergrad) (expert)

SLIDE 58 development time (weeks) 4 8 12 16 Chlorophyll SynthCL WebSynth IFC Verification for executable specifications

f secure stack machines.

Finds all bugs reported by a specialized tool [Hritcu et al., ICFP’13].

SDSLs developed with ROSETTE

21 (first-year grad) (expert) (undergrad) (expert)

SLIDE 59

Modern descendent of Scheme with macro-based metaprogramming.

Anatomy of a solver-aided host language

22

Racket

SLIDE 60

ROSETTE

Anatomy of a solver-aided host language

22 (define-symbolic id type) (assert expr) (verify expr) (debug [expr] expr) (solve expr) (synthesize [expr] expr)

SLIDE 61

A tiny example SDSL

23 def bvmax(r0, r1) : r2 = bvge(r0, r1) r3 = bvneg(r2) r4 = bvxor(r0, r2) r5 = bvand(r3, r4) r6 = bvxor(r1, r5) return r6 BV: A tiny assembly-like language for writing fast, low- level library functions. debug synth

SLIDE 62

A tiny example SDSL

23 def bvmax(r0, r1) : r2 = bvge(r0, r1) r3 = bvneg(r2) r4 = bvxor(r0, r2) r5 = bvand(r3, r4) r6 = bvxor(r1, r5) return r6 BV: A tiny assembly-like language for writing fast, low- level library functions. test verify debug synth

SLIDE 63

A tiny example SDSL

23 def bvmax(r0, r1) : r2 = bvge(r0, r1) r3 = bvneg(r2) r4 = bvxor(r0, r2) r5 = bvand(r3, r4) r6 = bvxor(r1, r5) return r6 BV: A tiny assembly-like language for writing fast, low- level library functions. test verify debug synth 1. interpreter [10 LOC] 2. verifier [free] 3. debugger [free] 4. synthesizer [free]

SLIDE 64 def bvmax(r0, r1) : r2 = bvge(r0, r1) r3 = bvneg(r2) r4 = bvxor(r0, r2) r5 = bvand(r3, r4) r6 = bvxor(r1, r5) return r6 > bvmax(-2, -1)

A tiny example SDSL:

24

ROSETTE

SLIDE 65 def bvmax(r0, r1) : r2 = bvge(r0, r1) r3 = bvneg(r2) r4 = bvxor(r0, r2) r5 = bvand(r3, r4) r6 = bvxor(r1, r5) return r6 > bvmax(-2, -1)

A tiny example SDSL:

24 (define bvmax `((2 bvge 0 1) (3 bvneg 2) (4 bvxor 0 2) (5 bvand 3 4) (6 bvxor 1 5))) parse

ROSETTE

SLIDE 66 def bvmax(r0, r1) : r2 = bvge(r0, r1) r3 = bvneg(r2) r4 = bvxor(r0, r2) r5 = bvand(r3, r4) r6 = bvxor(r1, r5) return r6 > bvmax(-2, -1)

A tiny example SDSL:

24 (define bvmax `((2 bvge 0 1) (3 bvneg 2) (4 bvxor 0 2) (5 bvand 3 4) (6 bvxor 1 5))) parse

ROSETTE

(out opcode in ...) (define bvmax `((2 bvge 0 1) (3 bvneg 2) (4 bvxor 0 2) (5 bvand 3 4) (6 bvxor 1 5)))

SLIDE 67 def bvmax(r0, r1) : r2 = bvge(r0, r1) r3 = bvneg(r2) r4 = bvxor(r0, r2) r5 = bvand(r3, r4) r6 = bvxor(r1, r5) return r6 > bvmax(-2, -1)

1

(define (interpret prog inputs) (make-registers prog inputs) (for ([stmt prog]) (match stmt [(list out opcode in ...) (define op (eval opcode)) (define args (map load in)) (store out (apply op args))])) (load (last)))

A tiny example SDSL:

25

ROSETTE

interpret (define bvmax `((2 bvge 0 1) (3 bvneg 2) (4 bvxor 0 2) (5 bvand 3 4) (6 bvxor 1 5))) `(-2 -1)

SLIDE 68 def bvmax(r0, r1) : r2 = bvge(r0, r1) r3 = bvneg(r2) r4 = bvxor(r0, r2) r5 = bvand(r3, r4) r6 = bvxor(r1, r5) return r6 > bvmax(-2, -1)

1

(define (interpret prog inputs) (make-registers prog inputs) (for ([stmt prog]) (match stmt [(list out opcode in ...) (define op (eval opcode)) (define args (map load in)) (store out (apply op args))])) (load (last)))

A tiny example SDSL:

25

ROSETTE

interpret (define bvmax `((2 bvge 0 1) (3 bvneg 2) (4 bvxor 0 2) (5 bvand 3 4) (6 bvxor 1 5))) 0 -2 1 -1 2 3 4 -2 5 6 -1

SLIDE 69 def bvmax(r0, r1) : r2 = bvge(r0, r1) r3 = bvneg(r2) r4 = bvxor(r0, r2) r5 = bvand(r3, r4) r6 = bvxor(r1, r5) return r6 > bvmax(-2, -1)

1

(define (interpret prog inputs) (make-registers prog inputs) (for ([stmt prog]) (match stmt [(list out opcode in ...) (define op (eval opcode)) (define args (map load in)) (store out (apply op args))])) (load (last)))

A tiny example SDSL:

25

ROSETTE

interpret (define bvmax `((2 bvge 0 1) (3 bvneg 2) (4 bvxor 0 2) (5 bvand 3 4) (6 bvxor 1 5))) 0 -2 1 -1 2 3 4 -2 5 6 -1 (2 bvge 0 1)

SLIDE 70 def bvmax(r0, r1) : r2 = bvge(r0, r1) r3 = bvneg(r2) r4 = bvxor(r0, r2) r5 = bvand(r3, r4) r6 = bvxor(r1, r5) return r6 > bvmax(-2, -1)

1

(define (interpret prog inputs) (make-registers prog inputs) (for ([stmt prog]) (match stmt [(list out opcode in ...) (define op (eval opcode)) (define args (map load in)) (store out (apply op args))])) (load (last)))

A tiny example SDSL:

25

ROSETTE

interpret (define bvmax `((2 bvge 0 1) (3 bvneg 2) (4 bvxor 0 2) (5 bvand 3 4) (6 bvxor 1 5))) 0 -2 1 -1 2 3 4 -2 5 6 -1 (2 bvge 0 1)

SLIDE 71 def bvmax(r0, r1) : r2 = bvge(r0, r1) r3 = bvneg(r2) r4 = bvxor(r0, r2) r5 = bvand(r3, r4) r6 = bvxor(r1, r5) return r6 > bvmax(-2, -1)

1

(define (interpret prog inputs) (make-registers prog inputs) (for ([stmt prog]) (match stmt [(list out opcode in ...) (define op (eval opcode)) (define args (map load in)) (store out (apply op args))])) (load (last)))

A tiny example SDSL:

25

ROSETTE

interpret (define bvmax `((2 bvge 0 1) (3 bvneg 2) (4 bvxor 0 2) (5 bvand 3 4) (6 bvxor 1 5))) 0 -2 1 -1 2 3 4 -2 5 6 -1 (2 bvge 0 1)

SLIDE 72 def bvmax(r0, r1) : r2 = bvge(r0, r1) r3 = bvneg(r2) r4 = bvxor(r0, r2) r5 = bvand(r3, r4) r6 = bvxor(r1, r5) return r6 > bvmax(-2, -1)

1

(define (interpret prog inputs) (make-registers prog inputs) (for ([stmt prog]) (match stmt [(list out opcode in ...) (define op (eval opcode)) (define args (map load in)) (store out (apply op args))])) (load (last)))

A tiny example SDSL:

25

ROSETTE

interpret (define bvmax `((2 bvge 0 1) (3 bvneg 2) (4 bvxor 0 2) (5 bvand 3 4) (6 bvxor 1 5))) 0 -2 1 -1 2 3 4 -2 5 6 -1 (2 bvge 0 1)

SLIDE 73 def bvmax(r0, r1) : r2 = bvge(r0, r1) r3 = bvneg(r2) r4 = bvxor(r0, r2) r5 = bvand(r3, r4) r6 = bvxor(r1, r5) return r6 > bvmax(-2, -1)

1

(define (interpret prog inputs) (make-registers prog inputs) (for ([stmt prog]) (match stmt [(list out opcode in ...) (define op (eval opcode)) (define args (map load in)) (store out (apply op args))])) (load (last)))

A tiny example SDSL:

25

ROSETTE

interpret (define bvmax `((2 bvge 0 1) (3 bvneg 2) (4 bvxor 0 2) (5 bvand 3 4) (6 bvxor 1 5))) 0 -2 1 -1 2 3 4 -2 5 6 -1 (2 bvge 0 1) (define bvmax `((2 bvge 0 1) (3 bvneg 2) (4 bvxor 0 2) (5 bvand 3 4) (6 bvxor 1 5)))

SLIDE 74 def bvmax(r0, r1) : r2 = bvge(r0, r1) r3 = bvneg(r2) r4 = bvxor(r0, r2) r5 = bvand(r3, r4) r6 = bvxor(r1, r5) return r6 > bvmax(-2, -1)

1

(define (interpret prog inputs) (make-registers prog inputs) (for ([stmt prog]) (match stmt [(list out opcode in ...) (define op (eval opcode)) (define args (map load in)) (store out (apply op args))])) (load (last)))

A tiny example SDSL:

25

ROSETTE

interpret (define bvmax `((2 bvge 0 1) (3 bvneg 2) (4 bvxor 0 2) (5 bvand 3 4) (6 bvxor 1 5))) 0 -2 1 -1 2 3 4 -2 5 6 -1 (2 bvge 0 1) (define bvmax `((2 bvge 0 1) (3 bvneg 2) (4 bvxor 0 2) (5 bvand 3 4) (6 bvxor 1 5)))

SLIDE 75 def bvmax(r0, r1) : r2 = bvge(r0, r1) r3 = bvneg(r2) r4 = bvxor(r0, r2) r5 = bvand(r3, r4) r6 = bvxor(r1, r5) return r6 > bvmax(-2, -1)

1

A tiny example SDSL:

26

ROSETTE

(define bvmax `((2 bvge 0 1) (3 bvneg 2) (4 bvxor 0 2) (5 bvand 3 4) (6 bvxor 1 5)))

pattern matching
dynamic evaluation
first-class &

higher-order procedures

side effects

(define (interpret prog inputs) (make-registers prog inputs) (for ([stmt prog]) (match stmt [(list out opcode in ...) (define op (eval opcode)) (define args (map load in)) (store out (apply op args))])) (load (last)))

SLIDE 76 (define-symbolic n0 n1 number?) (define inputs (list n0 n1)) (verify (assert (= (interpret bvmax inputs) (interpret max inputs)))) def bvmax(r0, r1) : r2 = bvge(r0, r1) r3 = bvneg(r2) r4 = bvxor(r0, r2) r5 = bvand(r3, r4) r6 = bvxor(r1, r5) return r6 > verify(bvmax, max) (0, -2) > bvmax(0, -2)

1

A tiny example SDSL:

27

ROSETTE

query

SLIDE 77 (define-symbolic n0 n1 number?) (define inputs (list n0 n1)) (verify (assert (= (interpret bvmax inputs) (interpret max inputs)))) (define-symbolic n0 n1 number?) (define inputs (list n0 n1)) (verify (assert (= (interpret bvmax inputs) (interpret max inputs)))) def bvmax(r0, r1) : r2 = bvge(r0, r1) r3 = bvneg(r2) r4 = bvxor(r0, r2) r5 = bvand(r3, r4) r6 = bvxor(r1, r5) return r6 > verify(bvmax, max) (0, -2) > bvmax(0, -2)

1

A tiny example SDSL:

27

ROSETTE

Creates two fresh symbolic constants of type number and binds them to variables n0 and n1. query

SLIDE 78 (define-symbolic n0 n1 number?) (define inputs (list n0 n1)) (verify (assert (= (interpret bvmax inputs) (interpret max inputs)))) (define-symbolic n0 n1 number?) (define inputs (list n0 n1)) (verify (assert (= (interpret bvmax inputs) (interpret max inputs)))) (define-symbolic n0 n1 number?) (define inputs (list n0 n1)) (verify (assert (= (interpret bvmax inputs) (interpret max inputs)))) def bvmax(r0, r1) : r2 = bvge(r0, r1) r3 = bvneg(r2) r4 = bvxor(r0, r2) r5 = bvand(r3, r4) r6 = bvxor(r1, r5) return r6 > verify(bvmax, max) (0, -2) > bvmax(0, -2)

1

A tiny example SDSL:

27

ROSETTE

Symbolic values can be used just like concrete values of the same type. query

SLIDE 79 (define-symbolic n0 n1 number?) (define inputs (list n0 n1)) (verify (assert (= (interpret bvmax inputs) (interpret max inputs)))) (define-symbolic n0 n1 number?) (define inputs (list n0 n1)) (verify (assert (= (interpret bvmax inputs) (interpret max inputs)))) (define-symbolic n0 n1 number?) (define inputs (list n0 n1)) (verify (assert (= (interpret bvmax inputs) (interpret max inputs)))) (define-symbolic n0 n1 number?) (define inputs (list n0 n1)) (verify (assert (= (interpret bvmax inputs) (interpret max inputs)))) def bvmax(r0, r1) : r2 = bvge(r0, r1) r3 = bvneg(r2) r4 = bvxor(r0, r2) r5 = bvand(r3, r4) r6 = bvxor(r1, r5) return r6 > verify(bvmax, max) (0, -2) > bvmax(0, -2)

1

A tiny example SDSL:

27

ROSETTE

(verify expr) searches for a concrete interpretation of symbolic constants that causes expr to fail. query

SLIDE 80 (define-symbolic n0 n1 number?) (define inputs (list n0 n1)) (verify (assert (= (interpret bvmax inputs) (interpret max inputs)))) (define-symbolic n0 n1 number?) (define inputs (list n0 n1)) (verify (assert (= (interpret bvmax inputs) (interpret max inputs)))) (define-symbolic n0 n1 number?) (define inputs (list n0 n1)) (verify (assert (= (interpret bvmax inputs) (interpret max inputs)))) (define-symbolic n0 n1 number?) (define inputs (list n0 n1)) (verify (assert (= (interpret bvmax inputs) (interpret max inputs)))) def bvmax(r0, r1) : r2 = bvge(r0, r1) r3 = bvneg(r2) r4 = bvxor(r0, r2) r5 = bvand(r3, r4) r6 = bvxor(r1, r5) return r6 > verify(bvmax, max) (0, -2) > bvmax(0, -2)

1

A tiny example SDSL:

27

ROSETTE

query

SLIDE 81 (define inputs (list 0 -2)) (debug [input-register?] (assert (= (interpret bvmax inputs) (interpret max inputs)))) def bvmax(r0, r1) : r2 = bvge(r0, r1) r3 = bvneg(r2) r4 = bvxor(r0, r2) r5 = bvand(r3, r4) r6 = bvxor(r1, r5) return r6 > debug(bvmax, max, (0, -2))

A tiny example SDSL:

28

ROSETTE

query

SLIDE 82 (define inputs (list 0 -2)) (debug [input-register?] (assert (= (interpret bvmax inputs) (interpret max inputs)))) def bvmax(r0, r1) : r2 = bvge(r0, r1) r3 = bvneg(r2) r4 = bvxor(r0, r2) r5 = bvand(r3, r4) r6 = bvxor(r1, r5) return r6 > debug(bvmax, max, (0, -2))

A tiny example SDSL:

28

ROSETTE

query def bvmax(r0, r1) : r2 = bvge(r0, r1) r3 = bvneg(r2) r4 = bvxor(r0, r2) r5 = bvand(r3, r4) r6 = bvxor(r1, r5) return r6

SLIDE 83 (define-symbolic n0 n1 number?) (define inputs (list n0 n1)) (synthesize [inputs] (assert (= (interpret bvmax inputs) (interpret max inputs))))) def bvmax(r0, r1) : r2 = bvge(r0, r1) r3 = bvneg(r2) r4 = bvxor(??, ??) r5 = bvand(r3, ??) r6 = bvxor(??, ??) return r6 > synthesize(bvmax, max)

A tiny example SDSL:

29

ROSETTE

query

SLIDE 84 (define-symbolic n0 n1 number?) (define inputs (list n0 n1)) (synthesize [inputs] (assert (= (interpret bvmax inputs) (interpret max inputs))))) def bvmax(r0, r1) : r2 = bvge(r0, r1) r3 = bvneg(r2) r4 = bvxor(??, ??) r5 = bvand(r3, ??) r6 = bvxor(??, ??) return r6 > synthesize(bvmax, max) def bvmax(r0, r1) : r2 = bvge(r0, r1) r3 = bvneg(r2) r4 = bvxor(r0, r1) r5 = bvand(r3, r4) r6 = bvxor(r1, r5) return r6

A tiny example SDSL:

29

ROSETTE

query

SLIDE 85

tech

symbolic virtual machine (SVM)

SLIDE 86

ROSETTE How it all works: a big picture view

31

symbolic virtual machine

SDSL program

query

solver

[Torlak & Bodik, Onward’13] [Torlak & Bodik, PLDI’14]

SLIDE 87

ROSETTE How it all works: a big picture view

31

symbolic virtual machine

SDSL program

result

solver

[Torlak & Bodik, Onward’13] [Torlak & Bodik, PLDI’14]

SLIDE 88

ROSETTE How it all works: a big picture view

31

symbolic virtual machine

SDSL program

result

solver

pattern matching
dynamic evaluation
first-class procedures
higher-order procedures
side effects
macros

theory of bitvectors [Torlak & Bodik, Onward’13] [Torlak & Bodik, PLDI’14]

SLIDE 89 (3, 1, -2) (1, 3)

Translation to constraints by example

32 solve: ps = () for v in vs: if v > 0: ps = insert(v, ps) assert len(ps) == len(vs) reverse and filter, keeping

nly positive numbers

vs ps

SLIDE 90 (3, 1, -2) (1, 3)

Translation to constraints by example

32 solve: ps = () for v in vs: if v > 0: ps = insert(v, ps) assert len(ps) == len(vs) vs ps

SLIDE 91

Translation to constraints by example

32 solve: ps = () for v in vs: if v > 0: ps = insert(v, ps) assert len(ps) == len(vs) vs ps constraints

SLIDE 92 a>0 ∧ b>0 (a, b)

Translation to constraints by example

32 solve: ps = () for v in vs: if v > 0: ps = insert(v, ps) assert len(ps) == len(vs) vs ps constraints

SLIDE 93

Design space of precise symbolic encodings

33 symbolic execution bounded model checking solve: ps = () for v in vs: if v > 0: ps = insert(v, ps) assert len(ps) == len(vs)

SLIDE 94

{ }

a > 0 b ≤ 0 false

Design space of precise symbolic encodings

33 a > 0 b ≤ 0 ps ↦ (a) ps ↦ (a) vs ↦ (a, b) ps ↦ ( ) symbolic execution bounded model checking solve: ps = () for v in vs: if v > 0: ps = insert(v, ps) assert len(ps) == len(vs)

SLIDE 95

{ }

a > 0 b ≤ 0 false

∨ ∨ ∨ Design space of precise symbolic encodings

33 a > 0 b ≤ 0 ps ↦ (a) ps ↦ (a) vs ↦ (a, b) ps ↦ ( ) b > 0 b > 0 ps ↦ (b) ps ↦ (b, a)

{ }

a ≤ 0 b > 0 false

{ }

a > 0 b > 0 true a ≤ 0 b ≤ 0 ps ↦ ( ) ps ↦ ( )

{ }

a ≤ 0 b ≤ 0 false symbolic execution bounded model checking solve: ps = () for v in vs: if v > 0: ps = insert(v, ps) assert len(ps) == len(vs)

SLIDE 96

{ }

a > 0 b ≤ 0 false

∨ ∨ ∨ Design space of precise symbolic encodings

33 vs ↦ (a, b) ps ↦ ( ) ps ↦ (a) ps ↦ ps0 a > 0 a ≤ 0 ps0 = ite(a > 0, (a), ( )) ps1 = insert(b, ps0) ps2 = ite(b > 0, ps0, ps1) assert len(ps2) = 2 a > 0 b ≤ 0 ps ↦ (a) ps ↦ (a) vs ↦ (a, b) ps ↦ ( ) b > 0 b > 0 ps ↦ (b) ps ↦ (b, a)

{ }

a ≤ 0 b > 0 false

{ }

a > 0 b > 0 true a ≤ 0 b ≤ 0 ps ↦ ( ) ps ↦ ( )

{ }

a ≤ 0 b ≤ 0 false symbolic execution bounded model checking ps ↦ ( ) solve: ps = () for v in vs: if v > 0: ps = insert(v, ps) assert len(ps) == len(vs)

SLIDE 97

{ }

a > 0 b ≤ 0 false

∨ ∨ ∨ Design space of precise symbolic encodings

33 vs ↦ (a, b) ps ↦ ( ) ps ↦ (a) ps ↦ ps0 a > 0 a ≤ 0 ps0 = ite(a > 0, (a), ( )) ps1 = insert(b, ps0) ps2 = ite(b > 0, ps0, ps1) assert len(ps2) = 2 a > 0 b ≤ 0 ps ↦ (a) ps ↦ (a) vs ↦ (a, b) ps ↦ ( ) b > 0 b > 0 ps ↦ (b) ps ↦ (b, a)

{ }

a ≤ 0 b > 0 false

{ }

a > 0 b > 0 true a ≤ 0 b ≤ 0 ps ↦ ( ) ps ↦ ( )

{ }

a ≤ 0 b ≤ 0 false symbolic execution bounded model checking ps ↦ ( ) ps ↦ ps1 b > 0 solve: ps = () for v in vs: if v > 0: ps = insert(v, ps) assert len(ps) == len(vs)

SLIDE 98

{ }

a > 0 b ≤ 0 false

∨ ∨ ∨ Design space of precise symbolic encodings

33 vs ↦ (a, b) ps ↦ ( ) ps ↦ (a) ps ↦ ps0 a > 0 a ≤ 0 ps0 = ite(a > 0, (a), ( )) ps1 = insert(b, ps0) ps2 = ite(b > 0, ps0, ps1) assert len(ps2) = 2 a > 0 b ≤ 0 ps ↦ (a) ps ↦ (a) vs ↦ (a, b) ps ↦ ( ) b > 0 b > 0 ps ↦ (b) ps ↦ (b, a)

{ }

a ≤ 0 b > 0 false

{ }

a > 0 b > 0 true a ≤ 0 b ≤ 0 ps ↦ ( ) ps ↦ ( )

{ }

a ≤ 0 b ≤ 0 false symbolic execution bounded model checking ps ↦ ( ) ps ↦ ps1 ps ↦ ps2 b > 0 b ≤ 0 ps ↦ ps0 solve: ps = () for v in vs: if v > 0: ps = insert(v, ps) assert len(ps) == len(vs)

SLIDE 99

A new design: type-driven state merging

34

{ }

a > 0 b > 0 true solve: ps = () for v in vs: if v > 0: ps = insert(v, ps) assert len(ps) == len(vs)

SLIDE 100

Merge values of

primitive types: symbolically
immutable types: structurally
all other types: via unions

A new design: type-driven state merging

34

{ }

a > 0 b > 0 true solve: ps = () for v in vs: if v > 0: ps = insert(v, ps) assert len(ps) == len(vs)

SLIDE 101

Merge values of

primitive types: symbolically
immutable types: structurally
all other types: via unions

b a

A new design: type-driven state merging

34

{ }

a > 0 b > 0 true solve: ps = () for v in vs: if v > 0: ps = insert(v, ps) assert len(ps) == len(vs) ⁊g g c

SLIDE 102

Merge values of

primitive types: symbolically
immutable types: structurally
all other types: via unions

b a (c, d) (a, b)

A new design: type-driven state merging

34

{ }

a > 0 b > 0 true solve: ps = () for v in vs: if v > 0: ps = insert(v, ps) assert len(ps) == len(vs) ⁊g g c (e, f)

SLIDE 103

Merge values of

primitive types: symbolically
immutable types: structurally
all other types: via unions

b a (c, d)

A new design: type-driven state merging

34

{ }

a > 0 b > 0 true solve: ps = () for v in vs: if v > 0: ps = insert(v, ps) assert len(ps) == len(vs) ⁊g g c (e, f) { ¬g ⊦ a, g ⊦ () } ()

SLIDE 104 solve: ps = () for v in vs: if v > 0: ps = insert(v, ps) assert len(ps) == len(vs)

A new design: type-driven state merging

35 symbolic virtual machine

SLIDE 105 solve: ps = () for v in vs: if v > 0: ps = insert(v, ps) assert len(ps) == len(vs) a > 0 a ≤ 0

A new design: type-driven state merging

35 vs ↦ (a, b) ps ↦ ( ) ps ↦ (a) ps ↦ ( ) symbolic virtual machine

SLIDE 106 solve: ps = () for v in vs: if v > 0: ps = insert(v, ps) assert len(ps) == len(vs)

Symbolic union: a set

f guarded values, with

disjoint guards.

g0 = a > 0 g1 = b > 0 g2 = g0 ∧ g1 g3 = ¬(g0 ⇔ g1) g4 = ¬g0 ∧ ¬g1 c = ite(g1, b, a) assert g2 a > 0 a ≤ 0 g0 ¬ g0

A new design: type-driven state merging

35 vs ↦ (a, b) ps ↦ ( ) ps ↦ (a) ps ↦ ( ) symbolic virtual machine ps ↦ { g0 ⊦ (a), ¬g0 ⊦ ( ) }

SLIDE 107 solve: ps = () for v in vs: if v > 0: ps = insert(v, ps) assert len(ps) == len(vs)

Execute insert concretely on all lists in the union.

g0 = a > 0 g1 = b > 0 g2 = g0 ∧ g1 g3 = ¬(g0 ⇔ g1) g4 = ¬g0 ∧ ¬g1 c = ite(g1, b, a) assert g2 a > 0 a ≤ 0 g1 g0 ¬ g0

A new design: type-driven state merging

35 vs ↦ (a, b) ps ↦ ( ) ps ↦ (a) ps ↦ ( ) symbolic virtual machine ps ↦ { g0 ⊦ (a), ¬g0 ⊦ ( ) } ps ↦ { g0 ⊦ (b, a), ¬g0 ⊦ (b) }

SLIDE 108 solve: ps = () for v in vs: if v > 0: ps = insert(v, ps) assert len(ps) == len(vs) g0 = a > 0 g1 = b > 0 g2 = g0 ∧ g1 g3 = ¬(g0 ⇔ g1) g4 = ¬g0 ∧ ¬g1 c = ite(g1, b, a) assert g2 a > 0 a ≤ 0 ¬ g1 g1 g0 ¬ g0

A new design: type-driven state merging

35 vs ↦ (a, b) ps ↦ ( ) ps ↦ (a) ps ↦ ( ) symbolic virtual machine ps ↦ { g0 ⊦ (a), ¬g0 ⊦ ( ) } ps ↦ { g0 ⊦ (b, a), ¬g0 ⊦ (b) } ps ↦ { g0 ⊦ (a), ¬g0 ⊦ ( ) }

SLIDE 109 solve: ps = () for v in vs: if v > 0: ps = insert(v, ps) assert len(ps) == len(vs) g0 = a > 0 g1 = b > 0 g2 = g0 ∧ g1 g3 = ¬(g0 ⇔ g1) g4 = ¬g0 ∧ ¬g1 c = ite(g1, b, a) assert g2 a > 0 a ≤ 0 ¬ g1 g1 g0 ¬ g0

A new design: type-driven state merging

35 vs ↦ (a, b) ps ↦ ( ) ps ↦ (a) ps ↦ ( ) symbolic virtual machine ps ↦ { g0 ⊦ (a), ¬g0 ⊦ ( ) } ps ↦ { g0 ⊦ (b, a), ¬g0 ⊦ (b) } ps ↦ { g0 ⊦ (a), ¬g0 ⊦ ( ) } ps ↦ { g2 ⊦ (b, a), g3 ⊦ (c), g4 ⊦ ( ) }

SLIDE 110 solve: ps = () for v in vs: if v > 0: ps = insert(v, ps) assert len(ps) == len(vs)

Evaluate len concretely

n all lists in the union;

assertion true only on the list guarded by g2.

g0 = a > 0 g1 = b > 0 g2 = g0 ∧ g1 g3 = ¬(g0 ⇔ g1) g4 = ¬g0 ∧ ¬g1 c = ite(g1, b, a) assert g2 a > 0 a ≤ 0 ¬ g1 g1 g0 ¬ g0

A new design: type-driven state merging

35 vs ↦ (a, b) ps ↦ ( ) ps ↦ (a) ps ↦ ( ) symbolic virtual machine ps ↦ { g0 ⊦ (a), ¬g0 ⊦ ( ) } ps ↦ { g0 ⊦ (b, a), ¬g0 ⊦ (b) } ps ↦ { g0 ⊦ (a), ¬g0 ⊦ ( ) } ps ↦ { g2 ⊦ (b, a), g3 ⊦ (c), g4 ⊦ ( ) }

SLIDE 111 solve: ps = () for v in vs: if v > 0: ps = insert(v, ps) assert len(ps) == len(vs) g0 = a > 0 g1 = b > 0 g2 = g0 ∧ g1 g3 = ¬(g0 ⇔ g1) g4 = ¬g0 ∧ ¬g1 c = ite(g1, b, a) assert g2 a > 0 a ≤ 0 ¬ g1 g1 g0 ¬ g0

A new design: type-driven state merging

35 vs ↦ (a, b) ps ↦ ( ) ps ↦ (a) ps ↦ ( ) symbolic virtual machine ps ↦ { g0 ⊦ (a), ¬g0 ⊦ ( ) } ps ↦ { g0 ⊦ (b, a), ¬g0 ⊦ (b) } ps ↦ { g0 ⊦ (a), ¬g0 ⊦ ( ) } ps ↦ { g2 ⊦ (b, a), g3 ⊦ (c), g4 ⊦ ( ) }

concrete evaluation polynomial encoding

SLIDE 112

Effectiveness of type-driven state merging

36 Merging performance for verification and synthesis queries in SynthCL, WebSynth and IFC programs 10000 20000 30000 40000 number of control flow joins 4500 9000 13500 18000 R² = 0.9884 R² = 0.95 number of unions size of all unions

SLIDE 113

Effectiveness of type-driven state merging

37 SVM and solving time for verification and synthesis queries in SynthCL, WebSynth and IFC programs running time (sec) 75 150 225 300 FWT3 B1 B2 J2 B3 MM1 J1 FWT1 B4 SF3 CR1 CR2 CR3 FWT4 CR4 MM2 SF1 FWT2 iTunes IMDb MM3 AlAn SF4 SF2 SVM Z3

SLIDE 114

future

advanced programming for everyone

SLIDE 115 solver-aided languages

Where next?

39 less code less time less effort web scraping scripts secure stack machines spatial programs data-parallel programs

SLIDE 116 advanced solver-aided languages solver-aided languages

Where next?

39 less code less time less effort new kinds of programs harder programs

SLIDE 117

Keeping the programmer in the loop

40 SAT/SMT solver SVM SDSL program

?

verify debug execute synth

SLIDE 118

Keeping the programmer in the loop

40 SAT/SMT solver SVM SDSL program

?

verify debug execute synth

SLIDE 119

Keeping the programmer in the loop

40 SAT/SMT solver SVM SDSL program

?

verify debug execute synth domain properties, invariants, insight

SLIDE 120

Keeping the programmer in the loop

40 SAT/SMT solver SVM SDSL program

?

verify debug execute synth domain properties, invariants, insight

? ?

SLIDE 121

Keeping the programmer in the loop

40 SAT/SMT solver SVM SDSL program

?

verify debug execute synth domain properties, invariants, insight

symbolic profiling

? ?

SLIDE 122 recursive fibonacci dynamic programming fibonacci ✘

✓

Keeping the system in the loop

41 SAT/SMT solver SVM

?

verify debug execute synth domain properties, invariants, insight SDSL program

SLIDE 123

symbolic design patterns Keeping the system in the loop

41 SAT/SMT solver SVM

?

verify debug execute synth domain properties, invariants, insight SDSL program

SLIDE 124

Domain-specific solvers

42

?

verify debug execute synth Sometimes you need a special-purpose solver … SDSL program

SLIDE 125

Domain-specific solvers for everyone

43

?

verify debug execute synth

synthesis of domain- specific solvers

SLIDE 126

SLIDE 127

So long, and thanks for all the fish!