Joint work with Marc Brockschmidt, Alex Gaunt, Alex Polozov, Patrick - - PowerPoint PPT Presentation

▶
joint work with marc brockschmidt alex gaunt alex polozov
SMART_READER_LITE
LIVE PREVIEW

Joint work with Marc Brockschmidt, Alex Gaunt, Alex Polozov, Patrick - - PowerPoint PPT Presentation

Nov 23, 2022 448 likes •958 views

Microsoft Research Cambridge Joint work with Marc Brockschmidt, Alex Gaunt, Alex Polozov, Patrick Fernandes, Mahmoud Khademi Deep Understands images/language/speech Learning Finds patterns in noisy data Requires many samples -

slide-1
SLIDE 1

Microsoft Research Cambridge

Joint work with Marc Brockschmidt, Alex Gaunt, Alex Polozov, Patrick Fernandes, Mahmoud Khademi

slide-2
SLIDE 2

Program Structure Deep Learning

✓ Understands images/language/speech ✓ Finds patterns in noisy data

  • Requires many samples
  • Handling structured data is hard

✓ Interpretable ✓ Generalisation verifiable

  • Manual effort
  • Limited to specialists

DPU

slide-3
SLIDE 3
slide-4
SLIDE 4

https://visualstudio.microsoft.com/services/intellicode/ http://www.eclipse.org/recommenders/

slide-5
SLIDE 5
slide-6
SLIDE 6

Defined Types

string string

slide-7
SLIDE 7

Variable Misuse

Allamanis et al. “Learning to Represent Programs with Graphs”. 2018

slide-8
SLIDE 8

Research in ML+Code

slide-9
SLIDE 9
slide-10
SLIDE 10
slide-11
SLIDE 11

B A E G D C F

Li et al (2015). Gated Graph Sequence Neural Networks.

B A E G D C F Gilmer et al (2017). Neural Message Passing for Quantum Chemistry.

slide-12
SLIDE 12

F D E

E F D F

F

slide-13
SLIDE 13

F F D E

E F D F

Li et al (2015). Gated graph sequence neural networks.

slide-14
SLIDE 14
slide-15
SLIDE 15
slide-16
SLIDE 16

B A E G D C F

slide-17
SLIDE 17

B A E G D C F B A E G D C F B A E G D C F B A E G D C F B A E G D C F B A E G D C F B A E G D C F

slide-18
SLIDE 18

Li et al (2015). Gated Graph Sequence Neural Networks. Gilmer et al (2017). Neural Message Passing for Quantum Chemistry.

  • node selection
  • node classification
  • graph classification

https://github.com/Microsoft/gated-graph-neural-network-samples

slide-19
SLIDE 19
slide-20
SLIDE 20

int int int int return for (int i =0; < ; ++) if ( [ ]>0) += [ ]; int int int int return for (int i = 0; i < lim; i++) if (arr[i] > 0) sum += arr[i];

slide-21
SLIDE 21

Assert.NotNull(clazz);

Assert . ( NotNull … ExpressionStatement InvocationExpression MemberAccessExpression ArgumentList Next Token AST Child

slide-22
SLIDE 22

(x, y) = Foo(); while (x > 0) x = x + y;

Last Write Last Use Computed From

slide-23
SLIDE 23

int int int int return for (int i =0; < ; ++) if ( [ ]>0) += [ ]; ~900 nodes/graph ~8k edges/graph

slide-24
SLIDE 24
slide-25
SLIDE 25
slide-26
SLIDE 26

B A E G D C F

slide-27
SLIDE 27

B A E G D C F

slide-28
SLIDE 28
slide-29
SLIDE 29

Seen Projects: 24 F/OSS C# projects (2060 kLOC): Used for train and test 3.8 type-correct alternative variables per slot (median 3, σ= 2.6)

Accuracy (%) BiGRU BiGRU+Dataflow GGNN Seen Projects 50.0 73.7 85.5

slide-30
SLIDE 30

Accuracy (%) BiGRU BiGRU+Dataflow GGNN Seen Projects 50.0 73.7 85.5 Unseen Projects 28.9 60.2 78.2

Seen Projects: 24 F/OSS C# projects (2060 kLOC): Used for train and test Unseen Projects: 3 F/OSS C# projects (228 kLOC): Used only for test 3.8 type-correct alternative variables per slot (median 3, σ= 2.6)

slide-31
SLIDE 31
slide-32
SLIDE 32

int int int int return for (int i = 0; i < lim; i++) if (arr[i]>0) sum += arr[i]; Returns the sum of the positive numbers in an array

slide-33
SLIDE 33

B A E G D C F

slide-34
SLIDE 34

C# Documentation F1 ROUGE-2 ROUGE-L biRNN -> RNN 35.2 20.8 36.7 GNN -> RNN 38.9 25.6 37.1 biRNN + GNN -> RNN 45.4 28.3 41.1 Java Method Naming F1 ROUGE-2 ROUGE-L Alon et al. * 43.0

  • biRNN -> RNN

35.8 17.9 39.7 biRNN + GNN -> RNN 44.7 21.1 43.1

* Alon et al. (2018). Code2seq: Generating sequences from structured representations of code.

slide-35
SLIDE 35
slide-36
SLIDE 36
slide-37
SLIDE 37

Adding Edges During Generation

Expression Expression

  • Expression

i Expression Expression Expression + 1 j i j

slide-38
SLIDE 38

Neural Attribute Grammars (NAG)

Expression Inherited Synthesized

slide-39
SLIDE 39

Information Propagation on Graph Generation

Expression Expression

  • Expression

i i j

slide-40
SLIDE 40

Information Propagation on Graph Generation

Expression1 Expression2

  • Expression3

i i j i j

Expression1

Expression2

Child

i

Child

Expression2

Parent

  • Child

NextSibling NextToken InheritedToSynthesized

T=1 T=2 T=3 T=4

slide-41
SLIDE 41

Model PPL Type-Correct Match@1 Match @5 Seq → NAG 8.38 40.4 8.4 15.8 Graph → Tree 5.37 41.2 19.9 36.8 Graph → Syntax Networks 3.03 74.7 32.4 48.1 Graph → Sequentalised Tree 3.48 84.5 36.0 52.7 Graph → Neural Attr. Gram. 3.07 84.5 38.8 57.0

Training data: 479 C# projects from GitHub Test data: 114 C# projects from GitHub (~100 000 samples)

slide-42
SLIDE 42
slide-43
SLIDE 43

Gunshots were fired at rapper Lil Wayne ’s tour bus No one was injured in the shooting, and no arrests have been made

Rapper Lil Wayne not injured after shots fired at his tour bus

  • n an Atlanta interstate, police say. No one has been

arrested in the shooting.

slide-44
SLIDE 44

signed New Zealand international Francis Saili

  • n

… …

Utility back Saili

…

Sentence 1 Sentence 2 PERSON LOCATION

slide-45
SLIDE 45

CNN/DailyMail ROUGE-1 ROUGE-2 ROUGE-L RNN -> RNN * 31.3 11.8 28.8 RNN + GNN -> RNN 33.0 13.3 28.3

* See et al. (2015). Get To The Point: Summarization with Pointer-Generator Networks

slide-46
SLIDE 46

CNN/DailyMail ROUGE-1 ROUGE-2 ROUGE-L RNN -> RNN * 31.3 11.8 28.8 RNN + GNN -> RNN 33.0 13.3 28.3 RNN -> RNN + pointer * 36.4 15.7 33.4 RNN + GNN -> RNN + pointer 38.1 16.1 33.2

* See et al. (2017). Get To The Point: Summarization with Pointer-Generator Networks

slide-47
SLIDE 47

Procedural Text Semantic Parsing Grounded Language

slide-48
SLIDE 48

Reasoning Rich Structure

slide-49
SLIDE 49

Joint work with M. Brockschmidt, A.L. Gaunt, A. Polozov, P . Fernandes, M. Khademi