SYNAPSE with Metasketches POPL 2016 James Bornholt Emina Torlak - - PowerPoint PPT Presentation

▶
synapse
SMART_READER_LITE
LIVE PREVIEW

SYNAPSE with Metasketches POPL 2016 James Bornholt Emina Torlak - - PowerPoint PPT Presentation

Mar 21, 2024 579 likes •1.4k views

f a i c t t r A C o m p * * l e t t n e e * A t * s i W s E n e L o l l C C P D * o O * e c s u P u m e E e R n * o t e v t d y s a E * a d l e u t a Optimizing Synthesis

slide-1
SLIDE 1

Optimizing Synthesis with Metasketches

James Bornholt Emina Torlak Dan Grossman Luis Ceze

C
  • n
s i s t e n t * C
  • m
p l e t e * W e l l D
  • c
u m e n t e d * E a s y t
  • R
e u s e *

*

E v a l u a t e d

* P O P L *

A r t i f a c t

* A E C

POPL 2016 University of Washington

SYNAPSE

slide-2
SLIDE 2

Optimizing Synthesis with Metasketches

James Bornholt Emina Torlak Dan Grossman Luis Ceze

C
  • n
s i s t e n t * C
  • m
p l e t e * W e l l D
  • c
u m e n t e d * E a s y t
  • R
e u s e *

*

E v a l u a t e d

* P O P L *

A r t i f a c t

* A E C

POPL 2016 University of Washington

SYNAPSE

f

  • r

A u t

  • m

a t e d A p p r

  • x

i m a t e P r

  • g

r a m m i n g

slide-3
SLIDE 3

Approximate computing in the future

slide-4
SLIDE 4

Approximate computing in the future

Precise Program

slide-5
SLIDE 5

Approximate computing in the future

Precise Program Acceptability Criteria

slide-6
SLIDE 6

Approximate computing in the future

Approximation Specification

slide-7
SLIDE 7

Approximate computing in the future

Magical Approximate Compiler

Approximation Specification

slide-8
SLIDE 8

Approximate computing in the future

Magical Approximate Compiler

Approximate Program Approximation Specification

slide-9
SLIDE 9

Program Synthesis

Synthesis finds programs automatically

Specification Program

slide-10
SLIDE 10

Program Synthesis

Synthesis finds programs automatically

Specification Program f(x) = 4x + 1

slide-11
SLIDE 11

Program Synthesis

Synthesis finds programs automatically

Specification Program f(x) = 4x + 1

4*x + 1

slide-12
SLIDE 12

Program Synthesis

Synthesis finds programs automatically

Specification Program f(x) = 4x + 1

4*x + 1 (x << 2) + 1

slide-13
SLIDE 13

Program Synthesis

What if synthesis doesn’t work?

Specification Program

slide-14
SLIDE 14

f(L, k) = Search(L, k)

Program Synthesis

What if synthesis doesn’t work?

Specification Program

slide-15
SLIDE 15

f(L, k) = Search(L, k)

Program Synthesis

What if synthesis doesn’t work?

Specification Program

slide-16
SLIDE 16

f(L, k) = Search(L, k)

Program Synthesis

What if synthesis doesn’t work?

Specification Program Syntactic Template Restricts the space of candidate programs (e.g. sketch, CFG)

slide-17
SLIDE 17

f(L, k) = Search(L, k)

Syntax-Guided Synthesis

What if synthesis doesn’t work?

Specification Program Syntactic Template Restricts the space of candidate programs (e.g. sketch, CFG)

slide-18
SLIDE 18

f(L, k) = Search(L, k)

Syntax-Guided Synthesis

What if synthesis doesn’t work?

Specification Program Syntactic Template Restricts the space of candidate programs (e.g. sketch, CFG) Linear search Binary search

slide-19
SLIDE 19

f(L, k) = Search(L, k)

Syntax-Guided Synthesis

What if synthesis doesn’t work?

Specification Program Syntactic Template Restricts the space of candidate programs (e.g. sketch, CFG) Linear search Binary search 7.1 MB program

slide-20
SLIDE 20

f(L, k) = Search(L, k)

Syntax-Guided Synthesis

What if synthesis doesn’t work?

Specification Program Syntactic Template Linear search Binary search Cost Function Selects the solution with minimal cost 7.1 MB program

slide-21
SLIDE 21

f(L, k) = Search(L, k)

Optimal Synthesis

What if synthesis doesn’t work?

Specification Program Syntactic Template Linear search Binary search Cost Function Selects the solution with minimal cost

slide-22
SLIDE 22

f(L, k) = Search(L, k)

Optimal Synthesis

What if synthesis doesn’t work?

Specification Program Syntactic Template Cost Function Selects the solution with minimal cost

slide-23
SLIDE 23

Optimal Synthesis

Metasketches offer control over synthesis

Specification Program

slide-24
SLIDE 24

Optimal Synthesis

Metasketches offer control over synthesis

Specification Program Metasketch Guides the search towards good solutions

slide-25
SLIDE 25

Optimal Synthesis

Metasketches offer control over synthesis

Specification Program Metasketch

  • 1. Metasketches
  • 2. Solving
  • 3. Results

Guides the search towards good solutions

slide-26
SLIDE 26

Optimal Synthesis

Metasketches offer control over synthesis

Specification Program Metasketch

  • 1. Metasketches
  • 2. Solving
  • 3. Results

Guides the search towards good solutions

slide-27
SLIDE 27

Metasketches

Explicit control over search strategy

slide-28
SLIDE 28

Synthesizing straight-line SSA programs

max(x, y):

  • 1 = ??op(??a, ??a, ??a)
  • 2 = ??op(??a, ??a, ??a)

return o2

slide-29
SLIDE 29

Synthesizing straight-line SSA programs

max(x, y):

  • 1 = ??op(??a, ??a, ??a)
  • 2 = ??op(??a, ??a, ??a)

return o2

Operations: +, -, <, if, …

slide-30
SLIDE 30

Synthesizing straight-line SSA programs

max(x, y):

  • 1 = ??op(??a, ??a, ??a)
  • 2 = ??op(??a, ??a, ??a)

return o2

Operations: +, -, <, if, … Inputs + earlier outputs

slide-31
SLIDE 31

Synthesizing straight-line SSA programs

max(x, y):

  • 1 = >(x, y)
  • 2 = ??op(??a, ??a, ??a)

return o2

slide-32
SLIDE 32

Synthesizing straight-line SSA programs

max(x, y):

  • 1 = >(x, y)
  • 2 = ite(o1, x, y)

return o2

slide-33
SLIDE 33

Synthesizing straight-line SSA programs

max(x, y):

  • 1 = ??op(??a, ??a, ??a)
  • 2 = ??op(??a, ??a, ??a)

return o2

slide-34
SLIDE 34

Synthesizing straight-line SSA programs

max(x, y):

  • 1 = ??op(??a, ??a, ??a)
  • 2 = ??op(??a, ??a, ??a)

return o2 S2

slide-35
SLIDE 35

Synthesizing straight-line SSA programs

max(x, y):

  • 1 = ??op(??a, ??a, ??a)
  • 2 = ??op(??a, ??a, ??a)

return o2 S2 max(x, y):

  • 1 = ??op(??a, ??a, ??a)

return o1 S1 max(x, y):

  • 1 = ??op(??a, ??a, ??a)
  • 2 = ??op(??a, ??a, ??a)
  • 3 = ??op(??a, ??a, ??a)

return o3 S3 S4 S5 S6 S7 S8 S9 …

slide-36
SLIDE 36

Unbounded space of candidate programs

Synthesizing straight-line SSA programs

max(x, y):

  • 1 = ??op(??a, ??a, ??a)
  • 2 = ??op(??a, ??a, ??a)

return o2 S2 max(x, y):

  • 1 = ??op(??a, ??a, ??a)

return o1 S1 max(x, y):

  • 1 = ??op(??a, ??a, ??a)
  • 2 = ??op(??a, ??a, ??a)
  • 3 = ??op(??a, ??a, ??a)

return o3 S3 S4 S5 S6 S7 S8 S9 …

slide-37
SLIDE 37

Unbounded space of candidate programs

Optimal synthesis could be unbounded

max(x, y):

  • 1 = ??op(??a, ??a, ??a)
  • 2 = ??op(??a, ??a, ??a)

return o2 S2 max(x, y):

  • 1 = ??op(??a, ??a, ??a)

return o1 S1 max(x, y):

  • 1 = ??op(??a, ??a, ??a)
  • 2 = ??op(??a, ??a, ??a)
  • 3 = ??op(??a, ??a, ??a)

return o3 S3 S4 S5 S6 S7 S8 S9 …

slide-38
SLIDE 38

Unbounded space of candidate programs

Optimal synthesis could be unbounded

max(x, y):

  • 1 = ??op(??a, ??a, ??a)
  • 2 = ??op(??a, ??a, ??a)

return o2 S2 max(x, y):

  • 1 = ??op(??a, ??a, ??a)

return o1 S1 max(x, y):

  • 1 = ??op(??a, ??a, ??a)
  • 2 = ??op(??a, ??a, ??a)
  • 3 = ??op(??a, ??a, ??a)

return o3 S3 S4 S5 S6 S7 S8 S9 …

UNSAT

slide-39
SLIDE 39

Unbounded space of candidate programs

Optimal synthesis could be unbounded

max(x, y):

  • 1 = ??op(??a, ??a, ??a)
  • 2 = ??op(??a, ??a, ??a)

return o2 S2 max(x, y):

  • 1 = ??op(??a, ??a, ??a)

return o1 S1 max(x, y):

  • 1 = ??op(??a, ??a, ??a)
  • 2 = ??op(??a, ??a, ??a)
  • 3 = ??op(??a, ??a, ??a)

return o3 S3 S4 S5 S6 S7 S8 S9 …

UNSAT SAT, cost 5

slide-40
SLIDE 40

Unbounded space of candidate programs

Optimal synthesis could be unbounded

max(x, y):

  • 1 = ??op(??a, ??a, ??a)
  • 2 = ??op(??a, ??a, ??a)

return o2 S2 max(x, y):

  • 1 = ??op(??a, ??a, ??a)

return o1 S1 max(x, y):

  • 1 = ??op(??a, ??a, ??a)
  • 2 = ??op(??a, ??a, ??a)
  • 3 = ??op(??a, ??a, ??a)

return o3 S3 S4 S5 S6 S7 S8 S9 …

UNSAT SAT, cost 5 SAT, cost 4

slide-41
SLIDE 41

Unbounded space of candidate programs

Optimal synthesis could be unbounded

max(x, y):

  • 1 = ??op(??a, ??a, ??a)
  • 2 = ??op(??a, ??a, ??a)

return o2 S2 max(x, y):

  • 1 = ??op(??a, ??a, ??a)

return o1 S1 max(x, y):

  • 1 = ??op(??a, ??a, ??a)
  • 2 = ??op(??a, ??a, ??a)
  • 3 = ??op(??a, ??a, ??a)

return o3 S3 S4 S5 S6 S7 S8 S9 …

UNSAT SAT, cost 5 SAT, cost 4 4 4 4 3 3 3

slide-42
SLIDE 42

Termination of optimal synthesis Unbounded space of candidate programs

Optimal synthesis could be unbounded

max(x, y):

  • 1 = ??op(??a, ??a, ??a)
  • 2 = ??op(??a, ??a, ??a)

return o2 S2 max(x, y):

  • 1 = ??op(??a, ??a, ??a)

return o1 S1 max(x, y):

  • 1 = ??op(??a, ??a, ??a)
  • 2 = ??op(??a, ??a, ??a)
  • 3 = ??op(??a, ??a, ??a)

return o3 S3 S4 S5 S6 S7 S8 S9 …

UNSAT SAT, cost 5 SAT, cost 4 4 4 4 3 3 3

slide-43
SLIDE 43

Termination of optimal synthesis Unbounded space of candidate programs

Optimal synthesis could be unbounded

max(x, y):

  • 1 = ??op(??a, ??a, ??a)
  • 2 = ??op(??a, ??a, ??a)

return o2 S2 max(x, y):

  • 1 = ??op(??a, ??a, ??a)

return o1 S1 max(x, y):

  • 1 = ??op(??a, ??a, ??a)
  • 2 = ??op(??a, ??a, ??a)
  • 3 = ??op(??a, ??a, ??a)

return o3 S3 S4 S5 S6 S7 S8 S9 …

UNSAT SAT, cost 5 SAT, cost 4 4 4 4 3 3 3 Search order

slide-44
SLIDE 44

Search order

Metasketches control search strategy

Termination of optimal synthesis Unbounded space of candidate programs

slide-45
SLIDE 45

Search order

Metasketches control search strategy

A metasketch consists of:

Termination of optimal synthesis Unbounded space of candidate programs

slide-46
SLIDE 46

Search order

Metasketches control search strategy

A metasketch consists of:

  • An ordered countable set of sketches S

Termination of optimal synthesis Unbounded space of candidate programs

slide-47
SLIDE 47

Search order

Metasketches control search strategy

A metasketch consists of:

  • An ordered countable set of sketches S

S4 S5 … S1 S2 S3

S = { }

Termination of optimal synthesis Unbounded space of candidate programs

slide-48
SLIDE 48

Search order

Metasketches control search strategy

A metasketch consists of:

  • An ordered countable set of sketches
  • A cost function

S

κ : L → R

S4 S5 … S1 S2 S3

S = { }

Termination of optimal synthesis Unbounded space of candidate programs

slide-49
SLIDE 49

Search order

Metasketches control search strategy

A metasketch consists of:

  • An ordered countable set of sketches
  • A cost function

S

κ : L → R

S4 S5 … S1 S2 S3

S = { } κ(P) = length(P)

Termination of optimal synthesis Unbounded space of candidate programs

slide-50
SLIDE 50

Search order

Metasketches control search strategy

A metasketch consists of:

  • An ordered countable set of sketches
  • A cost function
  • A gradient function

S

κ : L → R g : R → 2S

S4 S5 … S1 S2 S3

S = { } κ(P) = length(P)

Termination of optimal synthesis Unbounded space of candidate programs

slide-51
SLIDE 51

Search order

Metasketches control search strategy

A metasketch consists of:

  • An ordered countable set of sketches
  • A cost function
  • A gradient function

S

κ : L → R g : R → 2S

S4 S5 … S1 S2 S3

S = { } κ(P) = length(P) g(c) = {Si | i < c}

Termination of optimal synthesis Unbounded space of candidate programs

slide-52
SLIDE 52

Unbounded space of candidate programs Search order

Metasketches control search strategy

A metasketch consists of:

  • An ordered countable set of sketches
  • A cost function
  • A gradient function

S

κ : L → R g : R → 2S

S4 S5 … S1 S2 S3

S = { } κ(P) = length(P) g(c) = {Si | i < c}

Termination of optimal synthesis

slide-53
SLIDE 53

Termination of optimal synthesis Unbounded space of candidate programs Search order

Metasketches control search strategy

A metasketch consists of:

  • An ordered countable set of sketches
  • A cost function
  • A gradient function

S

κ : L → R g : R → 2S

S4 S5 … S1 S2 S3

S = { } κ(P) = length(P) g(c) = {Si | i < c}

slide-54
SLIDE 54

Termination of optimal synthesis Search order Unbounded space of candidate programs

Metasketches control search strategy

A metasketch consists of:

  • An ordered countable set of sketches
  • A cost function
  • A gradient function

S

κ : L → R g : R → 2S

S4 S5 … S1 S2 S3

S = { } κ(P) = length(P) g(c) = {Si | i < c}

slide-55
SLIDE 55

Solving a metasketch

Cooperating parallel searches

slide-56
SLIDE 56

Thread 2 S3 S2 S1 Thread 1

Metasketches provide implicit parallelism

S4 S5 …

S = { } κ(P) = length(P) g(c) = {Si | i < c}

slide-57
SLIDE 57

Thread 2 S3 S2 S1 Thread 1

Metasketches provide implicit parallelism

S4 S5 …

S = { } κ(P) = length(P) g(c) = {Si | i < c}

S1

slide-58
SLIDE 58

Thread 2 S3 S2 S1 Thread 1

Metasketches provide implicit parallelism

S4 S5 …

S = { } κ(P) = length(P) g(c) = {Si | i < c}

S1 S2

slide-59
SLIDE 59

Thread 2 S3 S2 S1 Thread 1

Metasketches provide implicit parallelism

S4 S5 …

S = { } κ(P) = length(P) g(c) = {Si | i < c}

UNSAT

S1 S2

slide-60
SLIDE 60

Thread 2 S3 S2 Thread 1

Metasketches provide implicit parallelism

S4 S5 …

S = { } κ(P) = length(P) g(c) = {Si | i < c}

S2 S3

slide-61
SLIDE 61

Thread 2

SAT, cost 2

S3 S2 Thread 1

Metasketches provide implicit parallelism

S4 S5 …

S = { } κ(P) = length(P) g(c) = {Si | i < c}

S2 S3

slide-62
SLIDE 62

Thread 2

SAT, cost 2

S3 S2 Thread 1

Metasketches provide implicit parallelism

S4 S5 …

S = { } κ(P) = length(P) g(c) = {Si | i < c}

S2 S3

SAT, cost 2

slide-63
SLIDE 63

Thread 2

SAT, cost 2

S3 S2 Thread 1

Metasketches provide implicit parallelism

S4 S5 …

S = { } κ(P) = length(P) g(c) = {Si | i < c}

g(2) = {S1}

S2 S3

SAT, cost 2

slide-64
SLIDE 64

Thread 2

SAT, cost 2

Thread 1

Metasketches provide implicit parallelism

S = { } κ(P) = length(P) g(c) = {Si | i < c}

g(2) = {S1}

S2 S3

SAT, cost 2

slide-65
SLIDE 65

Thread 2

SAT, cost 2

Thread 1

Metasketches provide implicit parallelism

S = { } κ(P) = length(P) g(c) = {Si | i < c}

g(2) = {S1}

S2

SAT, cost 2

slide-66
SLIDE 66

Thread 2 Thread 1

Metasketches provide implicit parallelism

S = { } κ(P) = length(P) g(c) = {Si | i < c}

g(2) = {S1}

SAT, cost 2

slide-67
SLIDE 67

Metasketches for approximation

Novel sofuware-based approximations, automatically

slide-68
SLIDE 68

Benchmark ffu-sin ffu-cos inversek2j-1 inversek2j-2 kmeans sobel-x sobel-y

Metasketches find novel, efficient approximations

slide-69
SLIDE 69

Benchmark ffu-sin ffu-cos inversek2j-1 inversek2j-2 kmeans sobel-x sobel-y Metasketch Polynomial Polynomial Polynomial Polynomial Superoptimization Superoptimization Superoptimization

Metasketches find novel, efficient approximations

slide-70
SLIDE 70

Benchmark ffu-sin ffu-cos inversek2j-1 inversek2j-2 kmeans sobel-x sobel-y Metasketch Polynomial Polynomial Polynomial Polynomial Superoptimization Superoptimization Superoptimization

Metasketches find novel, efficient approximations

Specification: < 50% error Cost function: sum of instruction costs

slide-71
SLIDE 71

Benchmark ffu-sin ffu-cos inversek2j-1 inversek2j-2 kmeans sobel-x sobel-y Metasketch Polynomial Polynomial Polynomial Polynomial Superoptimization Superoptimization Superoptimization Speedup 11.4× 12.0× 34.8× 10.0×

1.6×

10.6× 10.7×

Metasketches find novel, efficient approximations

Specification: < 50% error Cost function: sum of instruction costs

slide-72
SLIDE 72

Benchmark ffu-sin ffu-cos inversek2j-1 inversek2j-2 kmeans sobel-x sobel-y Metasketch Polynomial Polynomial Polynomial Polynomial Superoptimization Superoptimization Superoptimization Speedup 11.4× 12.0× 34.8× 10.0×

1.6×

10.6× 10.7× Error 21.3% 28.9% 16.3% 18.5% 14.9%

0.0% 0.0%

Metasketches find novel, efficient approximations

Specification: < 50% error Cost function: sum of instruction costs

slide-73
SLIDE 73

Benchmark ffu-sin ffu-cos inversek2j-1 inversek2j-2 kmeans sobel-x sobel-y Metasketch Polynomial Polynomial Polynomial Polynomial Superoptimization Superoptimization Superoptimization Speedup 11.4× 12.0× 34.8× 10.0×

1.6×

10.6× 10.7× Error 21.3% 28.9% 16.3% 18.5% 14.9%

0.0% 0.0%

Metasketches find novel, efficient approximations

Missed compiler

  • ptimization

Specification: < 50% error Cost function: sum of instruction costs

slide-74
SLIDE 74

Benchmark ffu-sin ffu-cos inversek2j-1 inversek2j-2 kmeans sobel-x sobel-y Metasketch Polynomial Polynomial Polynomial Polynomial Superoptimization Superoptimization Superoptimization Speedup 11.4× 12.0× 34.8× 10.0×

1.6×

10.6× 10.7× Error 21.3% 28.9% 16.3% 18.5% 14.9%

0.0% 0.0%

Solving Time (8 cores)

6.2 s

6.0 s

11.5 s

6.7 s 2000.4 s

1611.9 s

1740.6 s

Metasketches find novel, efficient approximations

Missed compiler

  • ptimization

Specification: < 50% error Cost function: sum of instruction costs

slide-75
SLIDE 75

Metasketches support complex cost functions

slide-76
SLIDE 76

Metasketches support complex cost functions

?? ?? ??

{

cat not cat

. . . . .

64 inputs ?? ?? ?? ?? ??

slide-77
SLIDE 77

Metasketches support complex cost functions

?? ?? ??

{

cat not cat

. . . . .

64 inputs ?? ?? ?? ?? ??

κ(P) = X

xi∈X

|P(xi) − yi|

slide-78
SLIDE 78

Metasketches support complex cost functions

?? ?? ??

{

cat not cat

. . . . .

64 inputs ?? ?? ?? ?? ??

κ(P) = X

xi∈X

|P(xi) − yi|

slide-79
SLIDE 79

Metasketches support complex cost functions

?? ?? ??

{

cat not cat

. . . . .

64 inputs ?? ?? ?? ?? ??

{

64–1–1 64–2–1 64–1–1–1 64–1–2–1 …

S =

{

κ(P) = X

xi∈X

|P(xi) − yi|

slide-80
SLIDE 80

Optimizing Synthesis with Metasketches

Optimal Synthesis

Specification Optimal Program Metasketch

slide-81
SLIDE 81

Optimizing Synthesis with Metasketches

Optimal Synthesis

Specification Optimal Program Metasketch

SYNAPSE

synapse.uwplse.org