Fast Synthesis of Fast Collections Calvin Loncaric Emina Torlak - - PowerPoint PPT Presentation

Fast Synthesis of Fast Collections

Calvin Loncaric Emina Torlak Michael D. Ernst University of Washington

Data structures are everywhere

2

Data structures are everywhere

2

Data structures are everywhere

2

Data structures are everywhere

2

Data structures are everywhere

2

Lists, maps, and sets solve many problems

Data structures are everywhere

2

Lists, maps, and sets solve many problems What if I need a custom data structure?

3

Cozy synthesizes collections

Impl. Impl. Impl. Outline Rep. Rep. Rep.

Inductive Synthesizer Verifier

Specification

3

Cozy synthesizes collections

Impl. Impl. Impl. Outline Rep. Rep. Rep.

Inductive Synthesizer Verifier

Specification

• Correct by construction

3

Cozy synthesizes collections

Impl. Impl. Impl. Outline Rep. Rep. Rep.

Inductive Synthesizer Verifier

Specification

• Correct by construction
• Specifications orders-of-magnitude shorter than

implementations, synthesized in < 90 seconds

3

Cozy synthesizes collections

Impl. Impl. Impl. Outline Rep. Rep. Rep.

Inductive Synthesizer Verifier

Specification

• Correct by construction
• Specifications orders-of-magnitude shorter than

implementations, synthesized in < 90 seconds

• Equivalent performance to human-written code

Myria Analytics Storage

4

Request 1 Request 2 time

Myria Analytics Storage

4

Request 1 Request 2 time

Myria Analytics Storage

4

Request 1 Request 2 time

Myria Analytics Storage

4

Request 1 Request 2 time

Goal: efficient retrieval of entries for a particular request ID in a particular timespan

Myria Analytics Storage

class AnalyticsLog {
 
 void log(Entry e)
 
 Iterator<Entry> getEntries(
 int queryId,
 int subqueryId,
 int fragmentId,
 long start,
 long end)
 
 }

5

Myria Analytics Storage

class AnalyticsLog {
 
 void log(Entry e)
 
 Iterator<Entry> getEntries(
 int queryId,
 int subqueryId,
 int fragmentId,
 long start,
 long end)
 
 }

5

Insert an entry into the data structure

Myria Analytics Storage

class AnalyticsLog {
 
 void log(Entry e)
 
 Iterator<Entry> getEntries(
 int queryId,
 int subqueryId,
 int fragmentId,
 long start,
 long end)
 
 }

5

Insert an entry into the data structure Retrieve entries

Myria Analytics Storage

6

Specification:

Entry has:
 queryId : Int,
 subqueryId : Int,
 fragmentId : Int,
 start, end : Long,
 … getEntries: all e where
 e.queryId = queryId and
 e.subqueryId = subqueryId and
 e.fragmentId = fragmentId and
 e.end >= start and
 e.start <= end

class AnalyticsLog {
 
 void log(Entry e)
 
 Iterator<Entry> getEntries(
 int queryId,
 int subqueryId,
 int fragmentId,
 long start,
 long end)
 
 }

7

Specification:

Entry has:
 field1 : Type1,
 field2 : Type2,
 …,
 start, end,
 … retrieveA: all e where
 condition
 e.subqueryId = subqueryId and
 retrieveB: all e where
 condition art and
 e.start <= end

Cozy

Cozy synthesizes collections

class Structure {
 
 void add(Entry e)
 void remove(Entry e)
 void update(Entry e, …)
 
 Iterator<Entry> retrieveA(…)
 Iterator<Entry> retrieveB(…)
 
 }
 
 
 
 


Trivial Solution

8

retrieve: all e where
 P(e, input)


Trivial Solution

8

List<Entry> data;
 
 Iterator<Entry> retrieve(input) {
 for e in data:
 if P(e, input):
 yield e
 }

retrieve: all e where
 P(e, input)


Trivial Solution

8

List<Entry> data;
 
 Iterator<Entry> retrieve(input) {
 for e in data:
 if P(e, input):
 yield e
 }

retrieve: all e where
 P(e, input)


There has to be a better way!

In the quest for a good solution, the search space of “all possible programs” is simply too large

9

Specification Implementation

Intractable

synthesis algorithm

Entry has:
 field1, field2, …
 retrieveA: all e where
 condition
 retrieveB: all e where
 condition void add(Entry e)
 void remove(Entry e)
 void update(Entry e, …)
 
 Iterator retrieveA(…)
 Iterator retrieveB(…)

In the quest for a good solution, the search space of “all possible programs” is simply too large

9

Specification Implementation

Intractable

synthesis algorithm

Entry has:
 field1, field2, …
 retrieveA: all e where
 condition
 retrieveB: all e where
 condition void add(Entry e)
 void remove(Entry e)
 void update(Entry e, …)
 
 Iterator retrieveA(…)
 Iterator retrieveB(…)

Tractable ? Tractable

9

Specification Implementation

Intractable

synthesis algorithm

Outline

Specification → Outline Outline → Implementation Entry has:
 field1, field2, …
 retrieveA: all e where
 condition
 retrieveB: all e where
 condition void add(Entry e)
 void remove(Entry e)
 void update(Entry e, …)
 
 Iterator retrieveA(…)
 Iterator retrieveB(…)

Tractable ? Tractable

9

Specification Implementation

Intractable

synthesis algorithm

Outline

Specification → Outline Outline → Implementation

specific enough to describe asymptotic performance general enough to encode a data structure succinctly

Entry has:
 field1, field2, …
 retrieveA: all e where
 condition
 retrieveB: all e where
 condition void add(Entry e)
 void remove(Entry e)
 void update(Entry e, …)
 
 Iterator retrieveA(…)
 Iterator retrieveB(…)

Tractable ? Tractable

10

Outlines

Plans for retrieving entries

• All ( )

10

Outlines

Plans for retrieving entries

• All ( )
• HashLookup ( outline, field = var )

10

Outlines

Plans for retrieving entries

• All ( )
• HashLookup ( outline, field = var )
• BinarySearch ( outline, field > var )

10

Outlines

Plans for retrieving entries

• All ( )
• HashLookup ( outline, field = var )
• BinarySearch ( outline, field > var )
• Concat ( outline1, outline2 )

10

Outlines

Plans for retrieving entries

• All ( )
• HashLookup ( outline, field = var )
• BinarySearch ( outline, field > var )
• Concat ( outline1, outline2 )
• Filter ( outline, predicate )

10

Outlines

Plans for retrieving entries

11

Impl. Impl. Impl. Outline Rep. Rep. Rep.

Inductive Synthesizer Verifier

Specification

Outlines → Implementations

Impl. Impl. Impl. Outline Rep. Rep. Rep.

Inductive Synthesizer Verifier

Specification

11

Impl. Impl. Impl. Outline Rep. Rep. Rep.

Inductive Synthesizer Verifier

Specification

Outlines → Implementations

Outlines → Implementations

12

HashLookup (
 All(),
 e.queryId = q )

class Structure {
 
 
 
 Iterator<Entry>
 retrieve(q) { … }
 
 }


Outlines → Implementations

12

HashLookup (
 All(),
 e.queryId = q )

class Structure {
 
 
 
 Iterator<Entry>
 retrieve(q) { … }
 
 }


Outlines → Implementations

12

HashLookup (
 All(),
 e.queryId = q )

class Structure {
 
 
 
 Iterator<Entry>
 retrieve(q) { … }
 
 }


T data;

Outlines → Implementations

13

HashLookup (
 data,
 e.queryId = q )

class Structure {
 
 
 
 Iterator<Entry>
 retrieve(q) { … }
 
 }


T data;

Outlines → Implementations

13

HashLookup (
 data,
 e.queryId = q )

class Structure {
 
 
 
 Iterator<Entry>
 retrieve(q) { … }
 
 }


T data;

Outlines → Implementations

13

HashLookup (
 data,
 e.queryId = q )

class Structure {
 
 
 
 Iterator<Entry>
 retrieve(q) { … }
 
 }


T data;

Outlines → Implementations

14

HashLookup (
 data,
 e.queryId = q )

class Structure {
 
 
 
 Iterator<Entry>
 retrieve(q) { … }
 
 }


HMap<K,V> data;

Outlines → Implementations

14

HashLookup (
 data,
 e.queryId = q )

class Structure {
 
 
 
 Iterator<Entry>
 retrieve(q) { … }
 
 }


HMap<K,V> data;

Outlines → Implementations

14

HashLookup (
 data,
 e.queryId = q )

class Structure {
 
 
 
 Iterator<Entry>
 retrieve(q) { … }
 
 }


HMap<K,V> data;

Outlines → Implementations

15

HashLookup (
 data,
 e.queryId = q )

class Structure {
 
 
 
 Iterator<Entry>
 retrieve(q) { … }
 
 }


HMap<int,V> data;

Outlines → Implementations

15

HashLookup (
 data,
 e.queryId = q )

class Structure {
 
 
 
 Iterator<Entry>
 retrieve(q) { … }
 
 }


HMap<int,V> data;

Outlines → Implementations

15

HashLookup (
 data,
 e.queryId = q )

class Structure {
 
 
 
 Iterator<Entry>
 retrieve(q) { … }
 
 }


HMap<int,V> data;

Outlines → Implementations

15

HashLookup (
 data,
 e.queryId = q )

class Structure {
 
 
 
 Iterator<Entry>
 retrieve(q) { … }
 
 }


HMap<int,V> data;

V = ArrayList<Entry>

Outlines → Implementations

15

HashLookup (
 data,
 e.queryId = q )

class Structure {
 
 
 
 Iterator<Entry>
 retrieve(q) { … }
 
 }


HMap<int,V> data;

V = ArrayList<Entry> V = LinkedList<Entry>

Outlines → Implementations

16

HashLookup (
 data,
 e.queryId = q )

class Structure {
 
 
 
 Iterator<Entry>
 retrieve(q) { … }
 
 }


HMap<int,V> data;

Outlines → Implementations

17

HashLookup (
 data,
 e.queryId = q )

class Structure {
 
 
 
 Iterator<Entry>
 retrieve(q)
 
 


HMap<int,V> data;

{ v = data.get(q);
 return v.iterator(); }

Outlines → Implementations

17

HashLookup (
 data,
 e.queryId = q )

class Structure {
 
 
 
 Iterator<Entry>
 retrieve(q)
 
 


HMap<int,V> data;

{ v = data.get(q);
 return v.iterator(); }

add, remove, update

18

Impl. Impl. Impl. Rep. Rep. Rep. Outline

Inductive Synthesizer Verifier

Specification

Specification → Outline

Impl. Impl. Impl. Outline Rep. Rep. Rep.

Inductive Synthesizer Verifier

Specification

18

Impl. Impl. Impl. Rep. Rep. Rep. Outline

Inductive Synthesizer Verifier

Specification

Specification → Outline

Specification → Outline

CEGIS

19

Inductive Synthesizer

candidate

Verifier

counterexample

• or -

certification of correctness

retrieve: all e where
 e.queryId = q and …

Specification → Outline

CEGIS

19

Inductive Synthesizer

candidate

Verifier

counterexample

• or -

certification of correctness Remembers all examples; only reasons about examples collected thus far.

retrieve: all e where
 e.queryId = q and …

Specification → Outline

CEGIS

19

Inductive Synthesizer

candidate

Verifier

counterexample

• or -

certification of correctness Remembers all examples; only reasons about examples collected thus far. Must ensure the

• utline is correct for

all possible inputs and all possible data structure states.

∀I ∀S, out = { e | e ∈ S ∧ P(I, e) }

retrieve: all e where
 e.queryId = q and …

Cost Model

20

HashLookup (
 All(),
 e.queryId = q ) Filter (
 All(),
 e.queryId = q )

Cost Model

20

HashLookup (
 All(),
 e.queryId = q ) Filter (
 All(),
 e.queryId = q )

O(1)

Cost Model

20

HashLookup (
 All(),
 e.queryId = q ) Filter (
 All(),
 e.queryId = q )

O(1) O(n)

Cost Model

20

HashLookup (
 All(),
 e.queryId = q ) Filter (
 All(),
 e.queryId = q )

O(1) O(n) O(1)

Cost Model

20

HashLookup (
 All(),
 e.queryId = q ) Filter (
 All(),
 e.queryId = q )

O(1) O(n) O(1) O(1)

Cost Model

20

HashLookup (
 All(),
 e.queryId = q ) Filter (
 All(),
 e.queryId = q )

O(1) O(n) O(1) O(1)

>

Cost Model

20

HashLookup (
 All(),
 e.queryId = q ) Filter (
 All(),
 e.queryId = q )

O(1) O(n) O(1) O(1)

>

Cozy prefers outlines with lower cost

Inductive Synthesis

Enumerative search

21

Inductive Synthesis

Enumerative search

21

All

size 1

Inductive Synthesis

Enumerative search

21

All

size 1 size 2

HashLookup(All, x=y) BinarySearch(All, x>y) Filter(All, x=y)

…

Inductive Synthesis

Enumerative search

21

All

size 1 size 2

HashLookup(All, x=y) BinarySearch(All, x>y) Filter(All, x=y)

…

Inductive Synthesis

Enumerative search

21

All

size 1 size 2

HashLookup(All, x=y) BinarySearch(All, x>y) Filter(All, x=y)

…

Concat(HashLookup(…),…) vs Concat(Filter(…),…)

Inductive Synthesis

Enumerative search

21

All

size 1 size 2

HashLookup(All, x=y) BinarySearch(All, x>y) Filter(All, x=y)

… size 3

HashLookup( HashLookup(…), a=b) Filter( BinarySearch(…), x<y) Filter( HashLookup(…), p=q)

…

Inductive Synthesis

Enumerative search

21

All

size 1 size 2

HashLookup(All, x=y) BinarySearch(All, x>y) Filter(All, x=y)

… size 3

HashLookup( HashLookup(…), a=b) Filter( BinarySearch(…), x<y) Filter( HashLookup(…), p=q)

…

Filter( HashLookup(…), p=q)

correct on all current examples

Outline Verification

22

Specification:

Entry has:
 queryId : Int,
 subqueryId : Int,
 … retrieve: all e where
 e.queryId = q and …
 
 
 


P

Outline Verification

22

Specification:

Entry has:
 queryId : Int,
 subqueryId : Int,
 … retrieve: all e where
 e.queryId = q and …
 
 
 


P { e | e ∈ S ∧ P(I, e) }

Outline Verification

22

Specification:

Entry has:
 queryId : Int,
 subqueryId : Int,
 … retrieve: all e where
 e.queryId = q and …
 
 
 


HashLookup(
 All(),
 e.queryId = q) P { e | e ∈ S ∧ P(I, e) }

Outline Verification

22

Specification:

Entry has:
 queryId : Int,
 subqueryId : Int,
 … retrieve: all e where
 e.queryId = q and …
 
 
 


HashLookup(
 All(),
 e.queryId = q) e.queryId = q representative predicate Q P { e | e ∈ S ∧ P(I, e) }

Outline Verification

22

Specification:

Entry has:
 queryId : Int,
 subqueryId : Int,
 … retrieve: all e where
 e.queryId = q and …
 
 
 


HashLookup(
 All(),
 e.queryId = q) e.queryId = q representative predicate Q { e | e ∈ S ∧ Q(I, e) } P { e | e ∈ S ∧ P(I, e) }

Outline Verification

23

{ e | e ∈ S ∧ P(I, e) } { e | e ∈ S ∧ Q(I, e) }

=

?

Outline Verification

23

{ e | e ∈ S ∧ P(I, e) } { e | e ∈ S ∧ Q(I, e) }

=

?

yes if and only if for all I, e: P(I, e) = Q(I, e)

Outline Verification

23

equivalence can be checked with an SMT solver { e | e ∈ S ∧ P(I, e) } { e | e ∈ S ∧ Q(I, e) }

=

?

yes if and only if for all I, e: P(I, e) = Q(I, e)

Evaluation

24

Evaluation

• Improve correctness

24

Evaluation

• Improve correctness
• Save programmer effort

24

Evaluation

• Improve correctness
• Save programmer effort
• Match performance

24

Evaluation

• Improve correctness
• Save programmer effort
• Match performance

24

Case studies

25

Case studies

25

Analytics data indexed by timespan and by request ID

• Myria: analytics

Case studies

25

Analytics data indexed by timespan and by request ID Tracks map tiles in a least-recently-used cache

• ZTopo: tile cache
• Myria: analytics

Case studies

25

Analytics data indexed by timespan and by request ID Tracks map tiles in a least-recently-used cache Stores axis-aligned bounding boxes for fast collision detection

• ZTopo: tile cache
• Bullet: volume tree
• Myria: analytics

Case studies

25

Analytics data indexed by timespan and by request ID Tracks map tiles in a least-recently-used cache Stores axis-aligned bounding boxes for fast collision detection

• ZTopo: tile cache
• Bullet: volume tree
• Myria: analytics
• Sat4j: variable metadata

Tracks information about each variable in the formula

Case studies

25

Analytics data indexed by timespan and by request ID Tracks map tiles in a least-recently-used cache Stores axis-aligned bounding boxes for fast collision detection

• ZTopo: tile cache
• Bullet: volume tree
• Myria: analytics
• Sat4j: variable metadata

Tracks information about each variable in the formula

11 bugs 15 bugs 7 bugs

Specifications vs. Implementations

26

Myria ZTopo Sat4j Bullet Lines of code 11 292 25 22 23 1383 2582 269 Original Spec

Synthesis Time

27

Myria ZTopo Sat4j Bullet Time (s) Outline Synthesis Auto-Tuning 30 90 60

Performance

28

Original Synthesized

ZTopo

Data structures are nearly identical

Performance

28

Original Synthesized

Bullet

Binary search tree vs. space partitioning tree

ZTopo

Data structures are nearly identical

Performance

28

Original Synthesized

Bullet

Binary search tree vs. space partitioning tree

ZTopo

Data structures are nearly identical

Performance

28

Original Synthesized

Small overhead; performance dominated by other factors

Sat4j Bullet

Binary search tree vs. space partitioning tree

ZTopo

Data structures are nearly identical

Performance

28

Original Synthesized

Myria

Original implementation has worst-case linear time Small overhead; performance dominated by other factors

Sat4j Bullet

Binary search tree vs. space partitioning tree

ZTopo

Data structures are nearly identical

Performance

28

Original Synthesized

Myria

Original implementation has worst-case linear time Small overhead; performance dominated by other factors

Sat4j Bullet

Binary search tree vs. space partitioning tree

ZTopo

Data structures are nearly identical

Performance

28

Original Synthesized

Related Work

29

Related Work

• J. Earley: “High level iterators and a method for automatically

designing data structure representation” (1974)

• Hard-coded rewrite rules

29

Related Work

• J. Earley: “High level iterators and a method for automatically

designing data structure representation” (1974)

• Hard-coded rewrite rules
• S. Agrawal et al: “Automated selection of materialized views and

indexes in sql databases” (2000)

• Enumerate possible views & indexes based on query syntax and use

the planner to decide which ones to keep

29

Related Work

• J. Earley: “High level iterators and a method for automatically

designing data structure representation” (1974)

• Hard-coded rewrite rules
• S. Agrawal et al: “Automated selection of materialized views and

indexes in sql databases” (2000)

• Enumerate possible views & indexes based on query syntax and use

the planner to decide which ones to keep

• P. Hawkins et al: “Data representation synthesis” (2011)
• Enumerate representations and use a planner to implement retrieval
• perations; conjunctions of equalities only

29

http://cozy.uwplse.org

• Implementation outlines make the

problem tractable

• Synthesis completes < 90 seconds
• Cozy generates correct code, and

matches handwritten implementation performance

30

Special thanks to: Michael Ernst Emina Torlak

also Haoming Liu & Daniel Perelman

Impl. Impl. Impl. Outline Rep. Rep. Rep.

Inductive Synthesizer Verifier

Specification