Learning Execution through Neural Code Fusion Zhan Shi, Kevin - - PowerPoint PPT Presentation

Learning Execution through Neural Code Fusion

Zhan Shi, Kevin Swersky, Danny Tarlow, Paruhasarathy Ranganathan, Milad Hashemi

Confidential + Proprietary Proprietary + Confidential

Overview

• Motivation
• Background
• Neural Code Fusion
• Experimental Results
• Conclusion

2

Confidential + Proprietary Proprietary + Confidential

Motivation

3

Proprietary + Confidential Proprietary + Confidential

2% Pergormance/Year is the New Normal

4

Source: Paruhasarathy Ranganathan, More Moore: Thinking Outside the (Server) Box

Confidential + Proprietary Proprietary + Confidential

• Dynamic speculative execution

○ Branch prediction, value prediction, cache replacement, prefetching...

Motivation

5

Confidential + Proprietary Proprietary + Confidential

• Dynamic speculative execution

○ Branch prediction, value prediction, cache replacement, prefetching...

• Static source code

○ Variable naming, fjnding bugs, algorithm classifjcation, program synthesis… ○ Pergormance-related tasks: device mapping, thread coarsening, throughput prediction...

Motivation

6

Confidential + Proprietary Proprietary + Confidential

• Dynamic speculative execution

○ Branch prediction, value prediction, cache replacement, prefetching...

• Static source code

○ Variable naming, fjnding bugs, algorithm classifjcation, program synthesis… ○ Pergormance-related tasks: device mapping, thread coarsening, throughput prediction...

• Both views provide useful features

Motivation

7

Confidential + Proprietary Proprietary + Confidential

for (i = 0; i < k; i++) { }

Example: a “Simple” Case for Branch Prediction

8

Confidential + Proprietary Proprietary + Confidential

for (i = 0; i < k; i++) { }

Example: a “Simple” Case for Branch Prediction

9

Highly biased

Confidential + Proprietary Proprietary + Confidential

for (i = 0; i < k; i++) { }

Example: a “Simple” Case for Branch Prediction

10

Highly biased Branch history doesn’t help

Confidential + Proprietary Proprietary + Confidential

while(...){ generate k; for (i = 0; i < k; i++) { } }

Example: a “Simple” Case for Branch Prediction

11

Highly biased Branch history doesn’t help

Confidential + Proprietary Proprietary + Confidential

while(...){ generate k; for (i = 0; i < k; i++) { } }

Example: a “Simple” Case for Branch Prediction

12

Highly biased Branch history doesn’t help

• Jump out when “close enough”
• Predictable if we knew the relation

[Static] i and k are compared [Dynamic] values of i and k

Confidential + Proprietary Proprietary + Confidential

Background: Graph Neural Networks

13

Confidential + Proprietary Proprietary + Confidential

Background: Graph Neural Networks

• Typical deep learning operates
• n IID data points.

14

Confidential + Proprietary Proprietary + Confidential

Background: Graph Neural Networks

• What if the data points had relational information?

Battaglia et al., 2018

15

Confidential + Proprietary Proprietary + Confidential

Background: Graph Neural Networks

• Message passing

16

Input graph

Confidential + Proprietary Proprietary + Confidential

Background: Graph Neural Networks

• Message passing

17

Input graph Step 0

Confidential + Proprietary Proprietary + Confidential

Background: Graph Neural Networks

• Message passing

18

Input graph Step 0 Step 1

Confidential + Proprietary Proprietary + Confidential

Background: Graph Neural Networks

• Message passing

19

Input graph Step 0 Step 1 Step 2

Confidential + Proprietary Proprietary + Confidential

Background: Graph Neural Networks

• Message passing

20

Input graph Step 0 Step 1 Step 2

GRU GRU

Confidential + Proprietary Proprietary + Confidential

Programs as Graphs

Allamanis et al., 2017

21

Confidential + Proprietary Proprietary + Confidential

Representing Static and Dynamic Information

• Graphs are an efgective representation for static code
• How do we generally represent dynamic information in a

model?

22

Confidential + Proprietary Proprietary + Confidential

Neural Code Fusion

23

Confidential + Proprietary Proprietary + Confidential

Full System

24

Confidential + Proprietary Proprietary + Confidential

Assembly vs Source Code

• Highly structured

25

Confidential + Proprietary Proprietary + Confidential

Assembly vs Source Code

• Highly structured

26

Confidential + Proprietary Proprietary + Confidential

Assembly vs Source Code

• Highly structured
• Directly relate data to program semantics

27

Confidential + Proprietary Proprietary + Confidential

Assembly vs Source Code

• Highly structured
• Directly relate data to program semantics
• Easy to use for architecture tasks

28

Confidential + Proprietary Proprietary + Confidential

Code Fusion Graph Representation

29

Confidential + Proprietary Proprietary + Confidential

Dynamic Tasks: Control Flow and Data Flow

• Control fmow (branch prediction)
• predict whether a branch statement will be taken or not taken.
• Set branch instruction node to be the target node.
• Binary classifjcation

30

Confidential + Proprietary Proprietary + Confidential

Dynamic Tasks: Control Flow and Data Flow

• Control fmow (branch prediction)
• predict whether a branch statement will be taken or not taken.
• Set branch instruction node to be the target node.
• Binary classifjcation
• Data fmow (prefetching)
• predict which address will be accessed next.
• Set src node to be the target node.
• Predict 64-bit address

31

Confidential + Proprietary Proprietary + Confidential

Multi-Task Representation

• Many other static/dynamic tasks can be defjned on the

graph simultaneously

○ Value prediction, indirect branch prediction, memory disambiguation, caching…

32

Confidential + Proprietary Proprietary + Confidential

Dynamic Snapshots

• Snapshots

○ The values of the set of variable nodes ○ Captured during program execution

• Used to initialize the graph neural network

33

Confidential + Proprietary Proprietary + Confidential

Representation Study

• Number “3” in difgerent representations

○ Categorical: [1, 0, 0, 0] ○ Scalar: 3 ○ Binary: 11

34

Confidential + Proprietary Proprietary + Confidential

Representation Study

• Correctly predict when to jump out
• Sample k values as training data

35

for(k=0; k < n; k+=3){ for (i = 0; i < k; i++) { } }

Confidential + Proprietary Proprietary + Confidential

Representation Study:

• Results

○ Binary > scalar > categorical

36

Confidential + Proprietary Proprietary + Confidential

Experimental Results

37

Confidential + Proprietary Proprietary + Confidential

Experimental Setup

• Benchmarks

○ SPEC06 INT

• Tasks

○ Dynamic: control fmow (branch prediction) and data fmow (prefetching) ○ Static: algorithm classifjcation

• Offmine evaluation for both NCF and baselines

○ 70% training ○ 30% testing

38

Confidential + Proprietary Proprietary + Confidential

Control-fmow (Branch Prediction) and Data-fmow (Prefetching)

39

Confidential + Proprietary Proprietary + Confidential

Algorithm Classifjcation

• Test the usefulness of the learned representation
• We pre-train our GNN on the control-fmow task
• A simple linear SVM model
• We get 96% vs 95.3% (50M lines of LLVM IR ) using 200k lines of

assembly with no external data sources.

40

Confidential + Proprietary Proprietary + Confidential

Summary

• NCF combining static and dynamic information

○ creates useful representations

• Difgerent from the traditional dynamic models in architecture

○ Data is usually purely dynamic ○ Model is history-based

• Enhances static models with dynamic program behavior

○ Learned representation can also transfer to a unseen static task

41

Thank you! Questions?