Neural Program Synthesis Rishabh Singh, Google Brain Great - - PowerPoint PPT Presentation

▶
neural program synthesis
SMART_READER_LITE
LIVE PREVIEW

Neural Program Synthesis Rishabh Singh, Google Brain Great - - PowerPoint PPT Presentation

Sep 13, 2022 19 likes •1.45k views

Neural Program Synthesis Rishabh Singh, Google Brain Great Collaborators! Deep Learning and Evolutionary Progression Vision Speech Language Deep Learning and Evolutionary Progression Vision Speech Language Programming Deep Learning

slide-1
SLIDE 1

Neural Program Synthesis

Rishabh Singh, Google Brain

slide-2
SLIDE 2

Great Collaborators!

slide-3
SLIDE 3

Deep Learning and Evolutionary Progression

Vision Speech Language

slide-4
SLIDE 4

Deep Learning and Evolutionary Progression

Vision Speech Language Programming

slide-5
SLIDE 5

Deep Learning and Evolutionary Progression

Vision Speech Language Programming Perceptual Tasks Algorithmic Tasks

slide-6
SLIDE 6

Neural Program Learning

slide-7
SLIDE 7

Neural Program Learning More Complex Tasks

slide-8
SLIDE 8

Neural Program Learning More Complex Tasks Generalizability

slide-9
SLIDE 9

Neural Program Learning More Complex Tasks Generalizability Interpretability

slide-10
SLIDE 10

Human Programmers

slide-11
SLIDE 11

Human Programmers

Spec

I/O Examples Natural Language Partial programs

slide-12
SLIDE 12

Human Programmers

Spec

Logic Basics Experience Samples

I/O Examples Natural Language Partial programs

slide-13
SLIDE 13

Human Programmers

Spec

Logic Basics Experience Samples

I/O Examples Natural Language Partial programs

slide-14
SLIDE 14

Neural Programmers

Spec

I/O Examples Natural Language Partial programs

Logic Basics Experience Samples

slide-15
SLIDE 15

Some Properties of Neural Programmers

slide-16
SLIDE 16

Some Properties of Neural Programmers

Limited Search

slide-17
SLIDE 17

Some Properties of Neural Programmers

Limited Search Learn from few examples/test

slide-18
SLIDE 18

Some Properties of Neural Programmers

Limited Search Learn from few examples/test Make Mistakes

slide-19
SLIDE 19

Some Properties of Neural Programmers

Limited Search Learn from few examples/test Make Mistakes Improve over time

slide-20
SLIDE 20

Long term Vision

Agent to win programming contests

[T

  • pCoder]
slide-21
SLIDE 21

Long term Vision

Agent to win programming contests

Program Representations

[T

  • pCoder]
slide-22
SLIDE 22

Long term Vision

Agent to win programming contests

Program Representations Program Repair[ICSE’18, ICLR’18]

[T

  • pCoder]
slide-23
SLIDE 23

Long term Vision

Agent to win programming contests

Program Representations Program Repair[ICSE’18, ICLR’18] Fuzzing/Security Testing [ASE’17]

[T

  • pCoder]
slide-24
SLIDE 24

Long term Vision

Agent to win programming contests

Program Representations Program Repair[ICSE’18, ICLR’18] Fuzzing/Security Testing [ASE’17] Program Optimization

[T

  • pCoder]
slide-25
SLIDE 25

Neural Program Induction

slide-26
SLIDE 26

Differentiable Neural Computer [Graves et al. Nature 2016]

slide-27
SLIDE 27

Neural RAM [Kurach et al. ICLR 2016]

An LSTM Controller choosing modules and arguments

slide-28
SLIDE 28

Neural RAM [Kurach et al. ICLR 2016]

An LSTM Controller choosing modules and arguments

14 modules

slide-29
SLIDE 29

Neural RAM [Kurach et al. ICLR 2016]

An LSTM Controller choosing modules and arguments

14 modules

Differentiable Semantics

slide-30
SLIDE 30

Neural RAM [Kurach et al. ICLR 2016]

An LSTM Controller choosing modules and arguments

14 modules

Differentiable Semantics

slide-31
SLIDE 31

Neural Program Induction

Differentiable memory, stack Lots of Examples Single-task learning Non-Interpretable programs Examples: NTM, DNC, etc. Difficult to Generalize

slide-32
SLIDE 32

Neural Program Induction

Differentiable memory, stack Lots of Examples Single-task learning Non-Interpretable programs Examples: NTM, DNC, etc. Difficult to Generalize

Neural Program Synthesis

Functional Abstractions Lots of Examples Single-task learning Interpretable programs Examples: QuickSort Generalizes Better

slide-33
SLIDE 33

Neural Program Induction

Differentiable memory, stack Lots of Examples Single-task learning Non-Interpretable programs Examples: NTM, DNC, etc. Difficult to Generalize

Neural Program Synthesis

Functional Abstractions Lots of Examples Single-task learning Interpretable programs Examples: QuickSort Generalizes Better

Meta- Neural Program Synthesis

Functional Abstractions Few Examples Multi-task learning Interpretable programs Strong Generalization

slide-34
SLIDE 34

Neuro-Symbolic Program Synthesis [ICLR 2017]

Emilio Parisotto, Abdelrahman Mohamed, Rishabh Singh, Lihong Li, Dengyong Zhou, Pushmeet Kohli

slide-35
SLIDE 35

FlashFill in Excel 2013

Gulwani, Harris, Singh [CACM Research Highlight 2012]

slide-36
SLIDE 36

FlashFill in Excel 2013

Gulwani, Harris, Singh [CACM Research Highlight 2012]

slide-37
SLIDE 37

FlashFill DSL

slide-38
SLIDE 38

Example FlashFill Task

Input (v) Output William Henry Charles Charles, W. Larry Page Page, L. Sergey Brin Brin, S. Martha D. Saunders Saunders, M.

slide-39
SLIDE 39

Example FlashFill Task

Input (v) Output William Henry Charles Charles, W. Larry Page Page, L. Sergey Brin Brin, S. Martha D. Saunders Saunders, M. Concat(f1, ConstStr(“, ”), f2, ConstStr(“.”))

slide-40
SLIDE 40

Example FlashFill Task

Input (v) Output William Henry Charles Charles, W. Larry Page Page, L. Sergey Brin Brin, S. Martha D. Saunders Saunders, M. Concat(f1, ConstStr(“, ”), f2, ConstStr(“.”)) f1 = SubStr(v, (Word,-1,Start), (Word,-1,End))

slide-41
SLIDE 41

Example FlashFill Task

Input (v) Output William Henry Charles Charles, W. Larry Page Page, L. Sergey Brin Brin, S. Martha D. Saunders Saunders, M. Concat(f1, ConstStr(“, ”), f2, ConstStr(“.”)) f1 = SubStr(v, (Word,-1,Start), (Word,-1,End)) f2 = SubStr(v, CPos(0), CPos(1))

slide-42
SLIDE 42

General Methodology

DSL

Rishabh Singh, Pushmeet Kohli. Artificial Programming. SNAPL 2017

slide-43
SLIDE 43

General Methodology

DSL

Sampler – Training Data Neural Model

Rishabh Singh, Pushmeet Kohli. Artificial Programming. SNAPL 2017

slide-44
SLIDE 44

General Methodology

DSL

Sampler – Training Data Neural Model

Synthesizer

Rishabh Singh, Pushmeet Kohli. Artificial Programming. SNAPL 2017

slide-45
SLIDE 45

General Methodology

DSL

Sampler – Training Data Neural Model

Synthesizer

3 Key Properties

Syntax Semantics Executable

Rishabh Singh, Pushmeet Kohli. Artificial Programming. SNAPL 2017

slide-46
SLIDE 46

Synthetic Training Data

slide-47
SLIDE 47

Synthetic Training Data

slide-48
SLIDE 48

Synthetic Training Data

slide-49
SLIDE 49

Real-world Test Data

slide-50
SLIDE 50

Real-world Test Data

slide-51
SLIDE 51

Real-world Test Data

slide-52
SLIDE 52

Real-world Test Data

slide-53
SLIDE 53

Neural Architecture

I/O Encoder

Examples

slide-54
SLIDE 54

Neural Architecture

I/O Encoder

Examples

Tree Decoder

slide-55
SLIDE 55 S -> e + e

CFG/DSL:

S -> e + e e -> x e -> 1 e -> 0 Non-Terminals = {S, e} Terminals = {x, 1, 0, +}

S e e e

S -> e + e S -> e + e a1: S -> e + e a1: e -> x a2: e -> 1 a3: e -> 0 a4: e -> x a5: e -> 1 a6: e -> 0

a1 a5

e -> 1

1

e -> 1

1 x

e -> x

a1

a1: e -> x a2: e -> 1 a3: e -> 0

f(x) = x + 1 + + +

Key Idea: Guided Enumeration

slide-56
SLIDE 56 S -> e + e

CFG/DSL:

S -> e + e e -> x e -> 1 e -> 0 Non-Terminals = {S, e} Terminals = {x, 1, 0, +}

S e e e

S -> e + e S -> e + e a1: S -> e + e a1: e -> x a2: e -> 1 a3: e -> 0 a4: e -> x a5: e -> 1 a6: e -> 0

a1 a5

e -> 1

1

e -> 1

1 x

e -> x

a1

a1: e -> x a2: e -> 1 a3: e -> 0

f(x) = x + 1 + + +

Key Idea: Guided Enumeration

slide-57
SLIDE 57 S -> e + e

CFG/DSL:

S -> e + e e -> x e -> 1 e -> 0 Non-Terminals = {S, e} Terminals = {x, 1, 0, +}

S e e e

S -> e + e S -> e + e a1: S -> e + e a1: e -> x a2: e -> 1 a3: e -> 0 a4: e -> x a5: e -> 1 a6: e -> 0

a1 a5

e -> 1

1

e -> 1

1 x

e -> x

a1

a1: e -> x a2: e -> 1 a3: e -> 0

f(x) = x + 1 + + +

Key Idea: Guided Enumeration

slide-58
SLIDE 58 S -> e + e

CFG/DSL:

S -> e + e e -> x e -> 1 e -> 0 Non-Terminals = {S, e} Terminals = {x, 1, 0, +}

S e e e

S -> e + e S -> e + e a1: S -> e + e a1: e -> x a2: e -> 1 a3: e -> 0 a4: e -> x a5: e -> 1 a6: e -> 0

a1 a5

e -> 1

1

e -> 1

1 x

e -> x

a1

a1: e -> x a2: e -> 1 a3: e -> 0

f(x) = x + 1 + + +

Key Idea: Guided Enumeration

slide-59
SLIDE 59 S -> e + e

Problem

How to assign probabilities to each action ai such that the global tree state is taken into account?

S e e e

S -> e + e S -> e + e a1: S -> e + e a1: e -> x a2: e -> 1 a3: e -> 0 a4: e -> x a5: e -> 1 a6: e -> 0

a1 a5

e -> 1

1

e -> 1

1 x

e -> x

a1

a1: e -> x a2: e -> 1 a3: e -> 0

f(x) = x + 1 + + +

Key Idea: Guided Enumeration

slide-60
SLIDE 60

Neural- Guided Enumeration

f ( ,I-O) =

slide-61
SLIDE 61

Neural- Guided Enumeration

f ( ,I-O) =

slide-62
SLIDE 62

Neural- Guided Enumeration

f ( ,I-O) =

slide-63
SLIDE 63

Key Challenges

2

slide-64
SLIDE 64

Key Challenges

2

Program Representation

slide-65
SLIDE 65

Key Challenges

2

Program Representation Example Representation I-O

slide-66
SLIDE 66

Recursive-Reverse-Recursive Neural Network (R3NN)

slide-67
SLIDE 67
  • Recursive-Reverse-Recursive Neural

Network (R3NN)

slide-68
SLIDE 68

Recursive

slide-69
SLIDE 69

Input:

Distributed representations

  • f each leaf’s symbol.

Recursive

slide-70
SLIDE 70

Input:

Distributed representations

  • f each leaf’s symbol.

Recursive

slide-71
SLIDE 71

Input:

Distributed representations

  • f each leaf’s symbol.

Recursive

slide-72
SLIDE 72

Input:

Distributed representations

  • f each leaf’s symbol.

Recursive

slide-73
SLIDE 73

Input:

Distributed representations

  • f each leaf’s symbol.

Recursive

slide-74
SLIDE 74

Input:

Distributed representations

  • f each leaf’s symbol.

Output:

Global root representation.

Recursive

slide-75
SLIDE 75

Reverse-Recursive

slide-76
SLIDE 76

Input:

root representation from recursive pass

Reverse-Recursive

slide-77
SLIDE 77

Input:

root representation from recursive pass

Reverse-Recursive

slide-78
SLIDE 78

Input:

root representation from recursive pass

Reverse-Recursive

slide-79
SLIDE 79

Input:

root representation from recursive pass

Reverse-Recursive

slide-80
SLIDE 80

Input:

root representation from recursive pass

Output:

Global leaf representations.

Reverse-Recursive

slide-81
SLIDE 81

Input:

root representation from recursive pass

Output:

Global leaf representations.

Reverse-Recursive

slide-82
SLIDE 82

Input:

root representation from recursive pass

Output:

Global leaf representations.

Reverse-Recursive

slide-83
SLIDE 83

Cross-Correlation I/O Encoder

slide-84
SLIDE 84

Cross-Correlation I/O Encoder

slide-85
SLIDE 85

Cross-Correlation I/O Encoder

slide-86
SLIDE 86

Synthetic Data Results (< 13 AST)

slide-87
SLIDE 87

FlashFill Benchmarks

slide-88
SLIDE 88

FlashFill Benchmarks

Batching Trees for larger programs

slide-89
SLIDE 89

FlashFill Benchmarks

Batching Trees for larger programs R3NN for contextual program embeddings

slide-90
SLIDE 90
  • J. Devlin, J. Uesato, S. Bhuptiraju, R. Singh, A. Mohamed, P

. Kohli

RobustFill [ICML 2017]

slide-91
SLIDE 91
  • J. Devlin, J. Uesato, S. Bhuptiraju, R. Singh, A. Mohamed, P

. Kohli

RobustFill [ICML 2017]

slide-92
SLIDE 92
  • J. Devlin, J. Uesato, S. Bhuptiraju, R. Singh, A. Mohamed, P

. Kohli

RobustFill [ICML 2017]

slide-93
SLIDE 93
  • J. Devlin, J. Uesato, S. Bhuptiraju, R. Singh, A. Mohamed, P

. Kohli

RobustFill [ICML 2017]

slide-94
SLIDE 94

Multiple I/O Examples

slide-95
SLIDE 95

Multiple I/O Examples

slide-96
SLIDE 96

Multiple I/O Examples

slide-97
SLIDE 97

Extended DSL

slide-98
SLIDE 98

92% Generalization Accuracy

slide-99
SLIDE 99

Robustness with Noise

slide-100
SLIDE 100

Incorrect Generalization

slide-101
SLIDE 101

Program Induction Model

slide-102
SLIDE 102

Induction vs Synthesis

slide-103
SLIDE 103

Other Synthesis Domains

More Complex DSLs

slide-104
SLIDE 104

Other Synthesis Domains

FlashFill (Functional) More Complex DSLs

slide-105
SLIDE 105

Other Synthesis Domains

FlashFill (Functional) More Complex DSLs Karel (Imperative with Control Flow)

slide-106
SLIDE 106

Other Synthesis Domains

FlashFill (Functional) More Complex DSLs Karel (Imperative with Control Flow) Python & R Scripts (Stateful Variables)

slide-107
SLIDE 107

Other Synthesis Domains

FlashFill (Functional) More Complex DSLs Karel (Imperative with Control Flow) Python & R Scripts (Stateful Variables) Grammar Learning (CFG s & CSGs)

slide-108
SLIDE 108

Other Synthesis Domains

FlashFill (Functional) More Complex DSLs Karel (Imperative with Control Flow) Python & R Scripts (Stateful Variables) Grammar Learning (CFG s & CSGs) Specification Modalities

slide-109
SLIDE 109

Other Synthesis Domains

FlashFill (Functional) More Complex DSLs Karel (Imperative with Control Flow) Python & R Scripts (Stateful Variables) Grammar Learning (CFG s & CSGs) Specification Modalities Natural Language (NL2SQL)

slide-110
SLIDE 110

Other Synthesis Domains

FlashFill (Functional) More Complex DSLs Karel (Imperative with Control Flow) Python & R Scripts (Stateful Variables) Grammar Learning (CFG s & CSGs) Specification Modalities Natural Language (NL2SQL) Partial Programs (Sketching)

slide-111
SLIDE 111

Synthesizing Karel Programs

[NIPS 2017, ICLR 2018]

  • R. Bunel, M. Hausknecht, J. Devlin, R. Singh, P

. Kohli

slide-112
SLIDE 112

Karel the Robot

Input Output

slide-113
SLIDE 113

Karel the Robot

Input Output Program

slide-114
SLIDE 114

Karel DSL

slide-115
SLIDE 115

Synthesis Architecture

CNNs for Encoder, LSTMs for decoder

slide-116
SLIDE 116

Supervised Learning

Top-1 Top-5 Supervised 71.91 80.00

slide-117
SLIDE 117

Multiple Consistent Programs

Input Output

slide-118
SLIDE 118

Reinforcement Learning

slide-119
SLIDE 119

Reinforcement Learning

  • 1. First Supervised Training
slide-120
SLIDE 120

Reinforcement Learning

  • 1. First Supervised Training
  • 2. Sample Program from the model
slide-121
SLIDE 121

Reinforcement Learning

  • 1. First Supervised Training
  • 2. Sample Program from the model
  • 3. Run the program on I/O
slide-122
SLIDE 122

Reinforcement Learning

  • 1. First Supervised Training
  • 2. Sample Program from the model
  • 3. Run the program on I/O
  • 4. Positive Reward if Output matches
slide-123
SLIDE 123

Reinforcement Learning

Top-1 Top-5 Supervised 71.91 80.00 REINFORCE 71.99 74.11 Beam REINFORCE 77.68 82.73

  • 1. First Supervised Training
  • 2. Sample Program from the model
  • 3. Run the program on I/O
  • 4. Positive Reward if Output matches
slide-124
SLIDE 124

Stanford CS106a Test

7/16 problems = 43%

slide-125
SLIDE 125

Stanford CS106a Test

7/16 problems = 43%

slide-126
SLIDE 126

Stanford CS106a Test

7/16 problems = 43% Neural Symbolic

slide-127
SLIDE 127

Neural Representations for Program Understanding/Analysis

slide-128
SLIDE 128

Neural Program Repair

Sahil Bhatia, Pushmeet Kohli, Rishabh Singh.Neuro-Symbolic Program Corrector. ICSE 2018 Ke Wang, Rishabh Singh, Zhendong Su. Dynamic Program Embeddings. ICLR 2018

slide-129
SLIDE 129

Neural Program Repair

Sahil Bhatia, Pushmeet Kohli, Rishabh Singh.Neuro-Symbolic Program Corrector. ICSE 2018 Ke Wang, Rishabh Singh, Zhendong Su. Dynamic Program Embeddings. ICLR 2018

slide-130
SLIDE 130

Neural Program Repair

Sahil Bhatia, Pushmeet Kohli, Rishabh Singh.Neuro-Symbolic Program Corrector. ICSE 2018 Ke Wang, Rishabh Singh, Zhendong Su. Dynamic Program Embeddings. ICLR 2018

slide-131
SLIDE 131

Dynamic Runtime Traces

Ke Wang, Rishabh Singh, Zhendong Su. Dynamic Program Embeddings. ICLR 2018

slide-132
SLIDE 132

Dynamic Runtime Traces

Ke Wang, Rishabh Singh, Zhendong Su. Dynamic Program Embeddings. ICLR 2018

slide-133
SLIDE 133

Embedding Program Traces

Ke Wang, Rishabh Singh, Zhendong Su. Dynamic Program Embeddings. ICLR 2018

slide-134
SLIDE 134

Embedding Program Traces

Ke Wang, Rishabh Singh, Zhendong Su. Dynamic Program Embeddings. ICLR 2018

slide-135
SLIDE 135

Fuzzing for Security Bugs

Seed Input

slide-136
SLIDE 136

Fuzzing for Security Bugs

Random Mutations Seed Input

slide-137
SLIDE 137

Fuzzing for Security Bugs

Random Mutations

Crash!

Execute Binary Seed Input

slide-138
SLIDE 138

Fuzzing for Security Bugs

Random Mutations

Crash!

Execute Binary Seed Input Coverage guided — AFL

slide-139
SLIDE 139

Neural Grammar-based Fuzzing

Patrice Godefroid, Hila Peleg Rishabh Singh. Learn&Fuzz: Machine Learning for Input Fuzzing. ASE 2017

slide-140
SLIDE 140

Neural Grammar-based Fuzzing

Patrice Godefroid, Hila Peleg Rishabh Singh. Learn&Fuzz: Machine Learning for Input Fuzzing. ASE 2017

slide-141
SLIDE 141

Neural Grammar-based Fuzzing

Patrice Godefroid, Hila Peleg Rishabh Singh. Learn&Fuzz: Machine Learning for Input Fuzzing. ASE 2017

More coverage, Bugs!

slide-142
SLIDE 142

Neural Programmer

Natural Language Input/Output Examples Partial Programs

Neural Synthesis [ICLR2017, ICML2017] Neural Repair [ICSE2018, ICLRW 2018] Program Induction [NIPS2017] Neural Fuzzing [ASE2017, arxiv2017]

Neural Architectures for Program and Spec Representation Rishabh Singh, rising@google.com