Joint work with Marc Brockschmidt, Alex Gaunt, Alex Polozov, Patrick - - PowerPoint PPT Presentation

▶
joint work with marc brockschmidt alex gaunt alex polozov
SMART_READER_LITE
LIVE PREVIEW

Joint work with Marc Brockschmidt, Alex Gaunt, Alex Polozov, Patrick - - PowerPoint PPT Presentation

Mar 14, 2023 754 likes •1.32k views

Microsoft Research Cambridge Joint work with Marc Brockschmidt, Alex Gaunt, Alex Polozov, Patrick Fernandes, Mahmoud Khademi Deep Understands images/language/speech Learning Finds patterns in noisy data Requires many samples -

slide-1
SLIDE 1

Microsoft Research Cambridge

Joint work with Marc Brockschmidt, Alex Gaunt, Alex Polozov, Patrick Fernandes, Mahmoud Khademi

slide-2
SLIDE 2

Program Structure Deep Learning

✓ Understands images/language/speech ✓ Finds patterns in noisy data

  • Requires many samples
  • Handling structured data is hard

✓ Interpretable ✓ Generalisation verifiable

  • Manual effort
  • Limited to specialists

DPU

slide-3
SLIDE 3
slide-4
SLIDE 4

https://visualstudio.microsoft.com/services/intellicode/ http://www.eclipse.org/recommenders/

slide-5
SLIDE 5
slide-6
SLIDE 6
slide-7
SLIDE 7

B A E G D C F

Li et al (2015). Gated Graph Sequence Neural Networks.

B A E G D C F Gilmer et al (2017). Neural Message Passing for Quantum Chemistry.

slide-8
SLIDE 8

F D E

E F D F

F

slide-9
SLIDE 9

F F D E

E F D F

Li et al (2015). Gated graph sequence neural networks.

slide-10
SLIDE 10
slide-11
SLIDE 11
slide-12
SLIDE 12

B A E G D C F

slide-13
SLIDE 13

B A E G D C F B A E G D C F B A E G D C F B A E G D C F B A E G D C F B A E G D C F B A E G D C F

slide-14
SLIDE 14

Li et al (2015). Gated Graph Sequence Neural Networks. Gilmer et al (2017). Neural Message Passing for Quantum Chemistry.

  • node selection
  • node classification
  • graph classification

https://github.com/Microsoft/gated-graph-neural-network-samples

slide-15
SLIDE 15
slide-16
SLIDE 16

int int int int return for (int i =0; < ; ++) if ( [ ]>0) += [ ]; int int int int return for (int i = 0; i < lim; i++) if (arr[i] > 0) sum += arr[i];

slide-17
SLIDE 17

Assert.NotNull(clazz);

Assert . ( NotNull … ExpressionStatement InvocationExpression MemberAccessExpression ArgumentList Next Token AST Child

slide-18
SLIDE 18

(x, y) = Foo(); while (x > 0) x = x + y;

Last Write Last Use Computed From

slide-19
SLIDE 19
slide-20
SLIDE 20
slide-21
SLIDE 21

int int int int return for (int i =0; < ; ++) if ( [ ]>0) += [ ]; ~900 nodes/graph ~8k edges/graph

slide-22
SLIDE 22
slide-23
SLIDE 23
slide-24
SLIDE 24

B A E G D C F

slide-25
SLIDE 25
slide-26
SLIDE 26
slide-27
SLIDE 27

B A E G D C F

slide-28
SLIDE 28

B A E G D C F

slide-29
SLIDE 29
slide-30
SLIDE 30

Seen Projects: 24 F/OSS C# projects (2060 kLOC): Used for train and test 3.8 type-correct alternative variables per slot (median 3, σ= 2.6)

Accuracy (%) BiGRU BiGRU+Dataflow GGNN Seen Projects 50.0 73.7 85.5

slide-31
SLIDE 31

Accuracy (%) BiGRU BiGRU+Dataflow GGNN Seen Projects 50.0 73.7 85.5 Unseen Projects 28.9 60.2 78.2

Seen Projects: 24 F/OSS C# projects (2060 kLOC): Used for train and test Unseen Projects: 3 F/OSS C# projects (228 kLOC): Used only for test 3.8 type-correct alternative variables per slot (median 3, σ= 2.6)

slide-32
SLIDE 32

bool string string

  • ut string

var while null if return true null return false

slide-33
SLIDE 33
slide-34
SLIDE 34
slide-35
SLIDE 35
slide-36
SLIDE 36

Expression Expression

  • Expression

i Expression Expression Expression + 1 j i j

But how do we generate graphs?

slide-37
SLIDE 37

Adding Edges During Generation

Expression Expression

  • Expression

i Expression Expression Expression + 1 j i j

slide-38
SLIDE 38

Neural Attribute Grammars (NAG)

Expression Inherited Synthesized

slide-39
SLIDE 39

Information Propagation on Graph Generation

Expression Expression

  • Expression

i i j

slide-40
SLIDE 40

Information Propagation on Graph Generation

Expression1 Expression2

  • Expression3

i i j i j

Expression1

Expression2

Child

i

Child

Expression2

Parent

  • Child

NextSibling NextToken InheritedToSynthesized

T=1 T=2 T=3 T=4

slide-41
SLIDE 41
slide-42
SLIDE 42
slide-43
SLIDE 43

Model PPL Type-Correct Match@1 Match @5 Seq → NAG 8.38 40.4 8.4 15.8 Graph → Tree 5.37 41.2 19.9 36.8 Graph → Syntax Networks 3.03 74.7 32.4 48.1 Graph → Sequentalised Tree 3.48 84.5 36.0 52.7 Graph → Neural Attr. Gram. 3.07 84.5 38.8 57.0

Training data: 479 C# projects from GitHub Test data: 114 C# projects from GitHub (~100 000 samples)

slide-44
SLIDE 44
slide-45
SLIDE 45

int int int int return for (int i = 0; i < lim; i++) if (arr[i]>0) sum += arr[i]; Returns the sum of the positive numbers in an array

slide-46
SLIDE 46

B E G D F I H L J C A

slide-47
SLIDE 47

B A E G D C F

slide-48
SLIDE 48

C# Documentation F1 ROUGE-2 ROUGE-L biRNN -> RNN 35.2 20.8 36.7 GNN -> RNN 38.9 25.6 37.1 biRNN + GNN -> RNN 45.4 28.3 41.1 Java Method Naming F1 ROUGE-2 ROUGE-L Alon et al. * 43.0

  • biRNN -> RNN

35.8 17.9 39.7 biRNN + GNN -> RNN 44.7 21.1 43.1

* Alon et al. (2018). Code2seq: Generating sequences from structured representations of code.

slide-49
SLIDE 49

Gunshots were fired at rapper Lil Wayne ’s tour bus No one was injured in the shooting, and no arrests have been made

Rapper Lil Wayne not injured after shots fired at his tour bus

  • n an Atlanta interstate, police say. No one has been

arrested in the shooting.

slide-50
SLIDE 50

signed New Zealand international Francis Saili

  • n

… …

Utility back Saili

…

Sentence 1 Sentence 2 PERSON LOCATION

slide-51
SLIDE 51

CNN/DailyMail ROUGE-1 ROUGE-2 ROUGE-L RNN -> RNN * 31.3 11.8 28.8 RNN + GNN -> RNN 33.0 13.3 28.3 RNN -> RNN + pointer * 36.4 15.7 33.4 RNN + GNN -> RNN + pointer 38.1 16.1 33.2

* See et al. (2017). Get To The Point: Summarization with Pointer-Generator Networks

slide-52
SLIDE 52

Encoding of Procedural Knowledge Graph Neural Networks

slide-53
SLIDE 53

Joint work with M. Brockschmidt, A.L. Gaunt, A. Polozov, P . Fernandes, M. Khademi

slide-54
SLIDE 54

Research in ML+Code