SLIDE 1
Neural Program Synthesis from Diverse Demonstration Videos
1University of Southern California 2Pohang University of Science and Technology
*Equal contribution
Shao-Hua Sun*1 Hyeonwoo Noh*2 Sriram Somasundaram1 Joseph J. Lim1
Neural Program Synthesis from Diverse Demonstration Videos Sriram - - PowerPoint PPT Presentation
Neural Program Synthesis from Diverse Demonstration Videos Sriram - - PowerPoint PPT Presentation
Neural Program Synthesis from Diverse Demonstration Videos Sriram Shao-Hua Sun* 1 Hyeonwoo Noh* 2 Joseph J. Lim 1 Somasundaram 1 1 University of Southern California 2 Pohang University of Science and Technology *Equal contribution A program is an
SLIDE 2
SLIDE 3
A program is an interpretable and executable way to describe behaviors
SLIDE 4
A program is an interpretable and executable way to describe behaviors
Human
SLIDE 5
A program is an interpretable and executable way to describe behaviors
Human Robot
SLIDE 6
Program Synthesis
Hey, can you…
SLIDE 7
Motivation
Devlin et al. “Robustfill: Neural program learning under noisy i/o.” ICML 2017 Balog, et al. "Deepcoder: Learning to write programs." ICLR 2017 Rudy R et al. “Leveraging grammar and reinforcement learning for neural program synthesis.” ICLR 2018
Program Synthesis
Input/output pairs Task specification
( ) , ( ) ,
SLIDE 8
Motivation
Program Synthesis
Devlin et al. “Robustfill: Neural program learning under noisy i/o.” ICML 2017 Balog, et al. "Deepcoder: Learning to write programs." ICLR 2017 Rudy R et al. “Leveraging grammar and reinforcement learning for neural program synthesis.” ICLR 2018
def run() if frontIsClear(): move() else: turnLeft() move() turnLeft repeat(2): turnRight() putMarker()
Program Input/output pairs Task specification
( ) , ( ) ,
SLIDE 9
Motivation
Program Synthesis
Devlin et al. “Robustfill: Neural program learning under noisy i/o.” ICML 2017 Balog, et al. "Deepcoder: Learning to write programs." ICLR 2017 Rudy R et al. “Leveraging grammar and reinforcement learning for neural program synthesis.” ICLR 2018
def run() if frontIsClear(): move() else: turnLeft() move() turnLeft repeat(2): turnRight() putMarker()
Program Input/output pairs Task specification
( ) , ( ) ,
SLIDE 10
def run() if frontIsClear(): move() else: turnLeft() move() turnLeft repeat(2): turnRight() putMarker()
Demonstration Sequences … … Input/output pairs Task specification
( ) , ( ) ,
Program Synthesis
Program
SLIDE 11
Problem Formulation
Program
?
Input a set of demo videos
SLIDE 12
Program
?
Output a program describing the demonstrated behavior Input a set of demo videos
Problem Formulation
SLIDE 13
Program
?
Output a program describing the demonstrated behavior Input a set of demo videos
Problem Formulation
- Extracting unique behaviors in each demo
- Summarizing diverse behaviors as a program
Challenges
SLIDE 14
Program
?
Output a program describing the demonstrated behavior Input a set of demo videos
Problem Formulation
- Extracting unique behaviors in each demo
- Summarizing diverse behaviors as a program
Challenges
Reviewer module Relation module
SLIDE 15
Program vector
Model Overview
Encoder
Demos
Encoder Encoder
Reviewer Module
Decoder
Program
Relation Module
Extract unique behaviors Demo features
SLIDE 16
Model Overview
Encoder
Demos
Encoder Encoder
Reviewer Module
Decoder
Program
Relation Module
Extract unique behaviors Summarize Demo features Program vector
SLIDE 17
Demo features
Encoder
Demos
Encoder Encoder
Reviewer Module
Decoder
Program Program vector
Relation Module
Model Overview
Extract unique behaviors Summarize Decode
SLIDE 18
Program vector
Model Overview
Encoder
Demos
Encoder Encoder
Reviewer Module
Decoder
Program
Relation Module
Extract unique behaviors Demo features Summarize Decode
SLIDE 19
Demo1
LSTM
t=1 t=2 t=T
…
CNN CNN CNN
…
Demo1 Encoder
Reviewer Module
SLIDE 20
Demo2
LSTM
t=1 t=2 t=T
…
CNN CNN CNN
Encoder
…
Demo1 Demo2
Reviewer Module
SLIDE 21
Demok
LSTM
t=1 t=2 t=T
…
CNN CNN CNN
Encoder
…
Demo1 Demo2 Demok
…
Reviewer Module
SLIDE 22
Demok
LSTM
t=1 t=2 t=T
…
CNN CNN CNN
Encoder
…
Demo1 Demo2 Demok
…
Overall Tendency
Reviewer Module
SLIDE 23
Demo1
LSTM
t=1 t=2 t=T
…
CNN CNN CNN
…
Encoder Demo1 Demo2 Demok
…
Overall Tendency
Reviewer Module
Review each demo
SLIDE 24
Reviewer LSTM
Demo1
LSTM
t=1 t=2 t=T
…
CNN CNN CNN
…
Encoder Demo1 Demo2 Demok
…
Overall Tendency Overall Tendency
Reviewer Module
SLIDE 25
Reviewer LSTM
Overall Tendency Demo1
LSTM
t=1 t=2 t=T
…
CNN CNN CNN
…
Encoder
Reviewer Module
SLIDE 26
Reviewer LSTM
Overall Tendency Demo1
LSTM
t=1 t=2 t=T
…
CNN CNN CNN
…
Encoder
…
Demo feature1
Reviewer Module
SLIDE 27
Model Overview
Encoder
Demos
Encoder Encoder
Reviewer Module
Demo features
SLIDE 28
Model Overview
Encoder
Demos
Encoder Encoder
Reviewer Module
Decoder
Program
Relation Module
Extract unique behaviors Summarize Demo features Program vector Decode
SLIDE 29
Relation Module
if frontIsClear(): move() else: turnLeft()
Demo1 Demo2
…
Reviewer Module
SLIDE 30
Relation Module
if frontIsClear(): move() else: turnLeft()
Demo1 Demo2
Compare demo pairs to infer branching conditions
…
Reviewer Module
SLIDE 31
Relation Module
…
Santoro et al. "A simple neural network module for relational reasoning.” NIPS 2017
Demo pairs
{ {
…
Reviewer Module
Relation Module
SLIDE 32
Relation Module
Program vector
…
Santoro et al. "A simple neural network module for relational reasoning.” NIPS 2017
gθ
gθ
gθ
Demo pairs
{ {
…
Reviewer Module
Relation Module
SLIDE 33
Demo features
Encoder
Demos
Encoder Encoder
Reviewer Module
Decoder
Program Program vector
Relation Module
Model Overview
Extract unique behaviors Summarize Decode
SLIDE 34
Decode a Program
LSTM def run() move() if <end>
Program vector
SLIDE 35
Model Overview
Demo features
Encoder
Demos
Encoder Encoder
Reviewer Module
Decoder
Program Program vector
Relation Module
Extract unique behaviors Summarize Decode
SLIDE 36
Experiments
SLIDE 37
Environments
Richard E Pattis. “Karel the robot: a gentle introduction to the art of programming.” John Wiley & Sons, Inc., 1981
def run() while(inTarget( HellKnight)): attack() moveForward() if isThere(Demon): moveRight() else: moveLeft() moveBackward() def run() if frontIsClear(): move() else: turnLeft() move() turnLeft repeat(2): turnRight() putMarker()
Kempka et al., “Vizdoom: A doom-based ai research platform for visual reinforcement learning.” In CIG, 2016
Karel ViZDoom
SLIDE 38
Baselines
- Program synthesis baseline
- Program induction baseline
Decoder Encoder
Program
Encoder Encoder
Average vector Demos
Without reviewer and relation modules
SLIDE 39
Baselines
- Program synthesis baseline
- Program induction baseline
Decoder
A testing demo frame
Encoder
Input Demos
Encoder Encoder
Average vector
State s Action a
move()
SLIDE 40
Evaluation Metrics
Example Result: Karel
def run(): if frontIsClear(): move() else: turnLeft() move() repeat(2): turnRight() putMarker()
Ground truth
def run(): move() move() turnRight() putMarker() turnRight() putMarker()
Synthesis baseline
Miss the if-else statement
SLIDE 41
Evaluation Metrics
def run(): if frontIsClear(): move() else: turnLeft() move() turnRight() putMarker() turnRight() putMarker()
Ours
def run(): if frontIsClear(): move() else: turnLeft() move() repeat(2): turnRight() putMarker()
Ground truth
def run(): move() move() turnRight() putMarker() turnRight() putMarker()
Synthesis baseline
Example Result: Karel
SLIDE 42
Evaluation Metrics
def run(): if inTarget(Demon): attack() moveLeft() else: moveRight() if isThere(Demon): attack() moveLeft()
Ground truth
def run(): while(inTarget( HellKnight)): attack() if isThere(Demon): moveRight() attack() else: moveLeft()
Synthesis baseline
Example Result: ViZDoom
SLIDE 43
Evaluation Metrics
def run(): if inTarget(Demon): attack() moveLeft() else: moveRight() if isThere(Demon): attack() moveLeft()
Ours Ground truth Synthesis baseline
def run(): if inTarget(Demon): attack() moveLeft() else: moveRight() if isThere(Demon): attack() moveLeft() def run(): while(inTarget( HellKnight)): attack() if isThere(Demon): moveRight() attack() else: moveLeft()
Example Result: ViZDoom
SLIDE 44
Environments and Results
Quantitative Result: Infer Programs
SLIDE 45
Evaluation Metrics
Sequence Accuracy
- Measure the accuracy based on code sequences
def run(): if A(): x() else: while(B()): y() z() def run(): if A(): x() else: repeat(5): y() z()
Synthesized program Ground truth program
def run(): if A(): x() else: y() def run(): if not A(): y() else: x()
Synthesized program Ground truth program
def run(): if A(): x() else: y() def run(): if A(): x() else: y()
Synthesized program Ground truth program
SLIDE 46
Evaluation Metrics
Sequence Accuracy
- Measure the accuracy based on code sequences
def run(): if A(): x() else: while(B()): y() z() def run(): if A(): x() else: repeat(5): y() z()
Synthesized program Ground truth program
def run(): if A(): x() else: y() def run(): if not A(): y() else: x()
Synthesized program Ground truth program
def run(): if A(): x() else: y() def run(): if A(): x() else: y()
Synthesized program Ground truth program
SLIDE 47
def run(): if A(): x() else: while(B()): y() z() def run(): if A(): x() else: repeat(5): y() z()
Synthesized program Ground truth program
def run(): if A(): x() else: y() def run(): if not A(): y() else: x()
Synthesized program Ground truth program
def run(): if A(): x() else: y() def run(): if A(): x() else: y()
Synthesized program Ground truth program
Evaluation Metrics
Sequence Accuracy
- Measure the accuracy based on code sequences
Program aliasing
Different codes with identical program semantics
SLIDE 48
Evaluation Metrics
Program Accuracy
def run(): if A(): x() else: while(B()): y() z() def run(): if A(): x() else: repeat(5): y() z()
Synthesized program Ground truth program
def run(): if A(): x() else: y() def run(): if not A(): y() else: x()
Synthesized program Ground truth program
def run(): if A(): x() else: y() def run(): if A(): x() else: y()
Synthesized program Ground truth program
- Compare programs in the program semantic space
- With some assumptions
- ex. termination of loops
SLIDE 49
25% 30% 35% 40% 45% 50% 55%
Sequence Accuracy Program Accuracy
48.9 41.0 42.4 35.7
Synthesis baseline Ours-full
Quantitative Result: Infer Programs
SLIDE 50
Environments and Results
Quantitative Result: Execution
SLIDE 51
Quantitative Result: Execution
Execution Accuracy
30% 40% 50% 60% 70% 80%
Karel ViZDoom
78.4 72.1 48.2 64.1 35.1 62.8
Induction baseline Synthesis baseline Ours (with the reviewer and relation modules)
SLIDE 52
Execution Accuracy
30% 40% 50% 60% 70% 80%
Karel ViZDoom
78.4 72.1 48.2 64.1 35.1 62.8
Induction baseline Synthesis baseline Ours (with the reviewer and relation modules)
Quantitative Result: Execution
SLIDE 53
Execution Accuracy
30% 40% 50% 60% 70% 80%
Karel ViZDoom
78.4 72.1 48.2 64.1 35.1 62.8
Induction baseline Synthesis baseline Ours (with the reviewer and relation modules)
Synthesize programs to describe behaviors
Quantitative Result: Execution
SLIDE 54
Execution Accuracy
30% 40% 50% 60% 70% 80%
Karel ViZDoom
78.4 72.1 48.2 64.1 35.1 62.8
Induction baseline Synthesis baseline Ours (with the reviewer and relation modules)
Reviewer module Relation module Synthesize programs to describe behaviors
Quantitative Result: Execution
SLIDE 55
Environments and Results
More Results
SLIDE 56
Environments and Results
0.1 0.3 0.5 0.7 0.9 5 10 15 20 25 30 35 40
Inudction baseline Synthesis baseline Ours (with the reviewer and relation module)
Execution Accuracy Number of input demos
Effect of Additional Demos
SLIDE 57
Environments and Results
Incomplete Seen Denominations
The seen demos do not completely capture the underlying behavior
SLIDE 58
Each program: single if-else statement with two branching consequences
Environments and Results
If-else experiment
20% 30% 40% 50% 60% 70% 80% 90%
Sequence Accuracy Program Accuracy Execution Accuracy
89.4 69.1 58.8 59.9 44.4 36.1 26.5
Inudction baseline Synthesis baseline Ours-full
SLIDE 59
Environments and Results
Conclusion
SLIDE 60
def run() if frontIsClear(): move() else: turnLeft() move() turnLeft repeat(2): turnRight() putMarker()
Neural Program Synthesis from Diverse Demonstration Videos
SLIDE 61
Neural Program Synthesis from Diverse Demonstration Videos
Demo features
Encoder
Demos
Encoder Encoder
Reviewer Module
Decoder
Program Program vector
Relation Module
Extract unique behaviors Summarize Decode
SLIDE 62
Codes, datasets, and checkpoints are available at
SLIDE 63