Execu&on Templates: Caching Control Plane Decisions for Strong - PowerPoint PPT Presentation

Execu&on Templates: Caching Control Plane Decisions for Strong Scaling of Data Analy&cs Omid Mashayekhi Hang Qu Chinmayee Shah Philip Levis July 13, 2017

2 2

Cloud Frameworks Machine Graph SQL Streaming Learning Cloud Framework ... ... ... ... Cloud frameworks abstract away the complexi&es of the cloud infrastructure from the applica&on developers: 1. Automa&c distribu&on 2. Elas&c scalability 3. Mul&tenant applica&ons 4. Load balancing 5. Fault tolerance 3

Cloud Frameworks Job SQL Control Plane ... ... ... ... Task • Job is an instance of the applica&on running in the framework. • Task is the unit of computa&on for the job. • Control plane par&&ons job in to tasks, schedules task, and recovers from faults. 4

Evolu&on of Cloud Frameworks 2004 I/O-bound data analy&cs MapReduce Hadoop 10s 1s 100ms 10ms 1ms Task Length 5

Evolu&on of Cloud Frameworks 2004 2012 I/O-bound In-memory data analy&cs data analy&cs MapReduce Spark Hadoop Naiad 10s 1s 100ms 10ms 1ms Task Length 6

Evolu&on of Cloud Frameworks 2004 2012 2016 I/O-bound In-memory Op&mized data analy&cs data analy&cs data analy&cs Spark 2.0 MapReduce Spark Common IL Hadoop Naiad C++ 10s 1s 100ms 10ms 1ms Task Length 7

Individual tasks are ge]ng faster. But does it necessarily mean that job comple&on &me is ge]ng shorter? 8

Control Plane The New Boaleneck • Logis&c regression over a data set of size 100GB. • Classic Spark used to be CPU-bound . 9

Control Plane The New Boaleneck • Logis&c regression over a data set of size 100GB. • Spark 2.0 with Scala implementa&on is already control-bound . 10

Control Plane The New Boaleneck • Logis&c regression over a data set of size 100GB. • Spark-opt: hypothe&cal case where Spark runs tasks as fast as C++. 11

Control plane is the emerging boaleneck for the cloud compu&ng frameworks. 12

Control Plane Design Scope Control Plane Example Task Throughput Scheduling Cost Design Framework (task per sec) (per task) MapReduce Centralized Hadoop ≈ 1 , 000 ≈ 100 µ s Spark Naiad Distributed ≈ 100 , 000 ≈ 100 , 000 µ s TensorFlow Centralized w/ • Centralized controller adapts to scheduling changes reac&vely with a low cost, but has limited task throughput and boalenecks at scale. • Distributed controller scales well, but any scheduling change requires stopping all nodes and installing new data flow with high latency. 13

Execu>on Templates is an abstrac&on for the control plane of cloud compu&ng frameworks, that enables orders of magnitude higher task throughput, while keeping the fine-grained, flexible scheduling with low cost. 14

Control Plane The New Boaleneck • Logis&c regression over a data set of size 100GB. • Nimbus with execu>on templates scales almost linearly, with low cost scheduling. 15

Repe&&ve Paaerns • Advanced data analy&cs are itera&ve in nature. – Machine learning, graph processing, image recogni&on, etc. • This results in repe&&ve paaerns in the control plane. – Similar tasks execute with minor differences. 16

Execu&on Model Driver Program Controller Data Task Graph Data flow Map Reduce Data Objects Data Objects Worker Worker 17

Execu&on Model Driver Program Controller Data Task Graph Data flow Map Reduce C Data Objects Data Objects Worker Worker 18

Execu&on Model Driver Program Controller Data Task Graph Data flow Map Reduce Task id Data list C Dep. list Function Parameter Data Objects Data Objects Worker Worker 19

Execu&on Model Driver Program Controller Data Task Graph Data flow Map Reduce Task id Data Objects Data Objects Data list Dep. list Function Data Exchange Parameter C Worker Worker 20

Repe&&ve Paaerns Controller Task Graph Data Objects Data Objects Worker Worker 21

Repe&&ve Paaerns Controller Task Graph Task id Data list C Dep. list Function Parameter Data Objects Data Objects Worker Worker 22

Repe&&ve Paaerns Controller Task Graph Task id Data Objects Data Objects Data list Dep. list Function Data Exchange Parameter C Worker Worker 23

Repe&&ve Paaerns Controller Task Graph Task id Data list C Dep. list Function Parameter Data Objects Data Objects Worker Worker 24

Repe&&ve Paaerns Controller Task Graph Task id Data Objects Data Objects Data list Dep. list Function Data Exchange Parameter C Worker Worker 25

Execu&on Templates • Tasks are cached as parameterizable blocks on nodes. • Instead of assigning the tasks from scratch, templates are instan>ated by filling in only changing parameters. Task id Data list Task id Dep. list Data list Function Task id Dep. list Parameter Data list Function Dep. list Parameter Function Parameter 26

Execu&on Templates • Tasks are cached as parameterizable blocks on nodes. • Instead of assigning the tasks from scratch, templates are instan>ated by filling in only changing parameters. Task id Load New Data list T 2 Task id T 1 Task ids Dep. list P 2 Data list P 1 Parameters Function Task id Dep. list Parameter Data list Function Dep. list Parameter T 3 Function P 3 Parameter 27

Execu&on Templates Mechanisms Summary • Instan>a>on : spawn a block of tasks without processing each task individually from scratch. It helps increase the task throughput . • Edits : modifies the content of each template at the granularity of tasks. It enables fine-grained, dynamic scheduling . • Patches : In case the state of the worker does not match the precondi&ons of the template. It enables dynamic control flow . 28

Execu&on Templates Instan&a&on Controller Task Graph C Data Objects Data Objects Worker Worker 29

Execu&on Templates Instan&a&on Controller Task Graph Data Objects Data Objects Template Template C C Worker Worker 30

Execu&on Templates Instan&a&on Controller Task Graph Data Objects Data Objects Template Template C Worker Worker 31

Execu&on Templates Instan&a&on Controller Task Graph Instantiate<params> Instantiate<params> Data Objects Data Objects Template Template C Worker Worker 32

Execu&on Templates Instan&a&on Controller Task Graph Data Objects Data Objects Template Template C C Worker Worker 33

Execu&on Templates Caching tasks implies sta&c behavior; how could templates allow dynamic scheduling ? • Reac&ve scheduling changes for load balancing. • Scheduling changes at the task granularity. 34

Execu&on Templates Edits • If scheduling changes, even slightly, the templates are obsolete. – For example rescheduling a task from one worker to another. • Instead of paying the substan&al cost of installing templates for every changes, templates allow edit , to change their structure. • Edits enable adding or removing tasks from the template and modifying the template content, in-place. • Controller has the general view of the task graph so it can update the dependencies properly, needed by the edits. 35

Execu&on Templates Edits Controller Task Graph Data Objects Data Objects Reschedule one task Template Template C Worker Worker 36

Execu&on Templates Edits Controller Task Graph Edit<add > Edit<remove > Data Objects Data Objects Template Template C Worker Worker 37

Execu&on Templates Edits Controller Task Graph Data Objects Data Objects Template Template C Worker Worker 38

Execu&on Templates Edits Controller Task Graph Instantiate<params> Instantiate<params> Data Objects Data Objects Template Template C Worker Worker 39

Execu&on Templates Caching tasks implies sta&c behavior; how could templates allow dynamic control flow ? • Need to support nested loops. • Need to support data dependent branches. 40

Execu&on Templates Patching • Execu&on templates operates at the granularity of basic blocks : – A code block with single entry and no branches except at the end. • Each template has a set of precondi>ons that need to be sa&sfied. – For example the set of data objects in memory, accessed by the tasks. • Worker state might not match the precondi&ons of the template in all circumstances. • Controller patches the worker state before template instan&a&on, to sa&sfy the precondi&ons. 41

Execu&on Templates Patching Controller Task Graph Data Objects Data Objects Precondi&ons Precondi&ons Template Template C Worker Worker 42

Execu&on Templates Patching Controller Task Graph Patch< load > Data Objects Data Objects Precondi&ons Precondi&ons Template Template C Worker Worker 43

Execu&on Templates Patching Controller Task Graph Data Objects Data Objects Precondi&ons Precondi&ons Template Template C Worker Worker 44

Execu&on Templates Patching Controller Task Graph Instantiate<params> Instantiate<params> Data Objects Data Objects Precondi&ons Precondi&ons Template Template C Worker Worker 45