Sol Fast Distributed Computation Over Slow Networks Fan Lai , Jie - - PowerPoint PPT Presentation

Sol

Fast Distributed Computation Over Slow Networks

Fan Lai, Jie You, Xiangfeng Zhu Harsha V. Madhyastha, Mosharaf Chowdhury

1

• Distributed computation in Local-Area Networks (LAN)
• To accelerate executions within a single cluster

Distributed Data Processing is Ubiquitous

2

Efforts for Computation in LAN

• Computation over Wide-Area Networks (WAN)
• To reduce data transfers, mitigate privacy risks
• Distributed computation in Local-Area Networks (LAN)
• To accelerate executions within a single cluster

Distributed Data Processing is Ubiquitous

2

Google Spanner

Iridium CLARINEt

Efforts for Computation over WAN

Azure Cosmos DB

Tetrium

Efforts for Computation in LAN

Execution Engine: Core of Big Data Stack

SQL Queries AI/ML

Select * FROM …; K-means, SVM

Stream Processing

… …

WordCount, TopKCount

3

Execution Engine: Core of Big Data Stack

SQL Queries AI/ML Execution Planner

Select * FROM …; K-means, SVM

Stream Processing

… …

WordCount, TopKCount

3

Execution Engine: Core of Big Data Stack

SQL Queries AI/ML Execution Planner

Select * FROM …; K-means, SVM

Stream Processing

… …

WordCount, TopKCount

Typical job execution plans

Job 2 Job 1

3

Execution Engine: Core of Big Data Stack

SQL Queries AI/ML Execution Planner

Select * FROM …; K-means, SVM

Coordinator Worker1 Worker2 WorkerN

…

Execution Engine Stream Processing

… …

WordCount, TopKCount

3

Execution Engine: Core of Big Data Stack

SQL Queries AI/ML Execution Planner Storage System Resource Scheduler

Select * FROM …; K-means, SVM

Coordinator Worker1 Worker2 WorkerN

…

Execution Engine Stream Processing

… …

WordCount, TopKCount

3

Google Spanner

Execution Engine: Core of Big Data Stack

SQL Queries AI/ML Execution Planner Storage System Resource Scheduler

Select * FROM …; K-means, SVM

Coordinator Worker1 Worker2 WorkerN

…

Execution Engine Stream Processing

… …

WordCount, TopKCount

Efforts for Computation in LAN

CLARINEt

Efforts for Computation over WAN

Azure Cosmos DB

Tetrium

3

Execution Engine: Core of Big Data Stack

SQL Queries AI/ML Execution Planner Storage System Resource Scheduler

Select * FROM …; K-means, SVM

Coordinator Worker1 Worker2 WorkerN

…

Execution Engine Stream Processing

… …

WordCount, TopKCount

Efforts for Computation in LAN

Efforts for Computation over WAN

Tetrium

3

Execution Engine: Core of Big Data Stack

SQL Queries AI/ML Execution Planner Storage System Resource Scheduler

Select * FROM …; K-means, SVM

Coordinator Worker1 Worker2 WorkerN

…

Execution Engine Stream Processing

… …

WordCount, TopKCount

Efforts for Computation in LAN

Efforts for Computation over WAN

Tetrium

3

While network conditions are diverse in real, execution engines remain the same

Outline

4

• Today’s Execution Engines
• Sol Architecture
• Control Plane Design
• Data Plane Design
• Evaluation

50 100 150 Query Completion Time (s) 0.00 0.25 0.50 0.75 1.00 CDF across Queries 10 Gbps, O(1) ms 1 Gbps, O(1) ms 10 Gbps, O(100) ms 1 Gbps, O(100) ms

Impact of Networks on Latency-sensitive Jobs

5

Queries from 100 GB TPC Benchmarks

Job Completion Time (s) CDF

50 100 150 Query Completion Time (s) 0.00 0.25 0.50 0.75 1.00 CDF across Queries 10 Gbps, O(1) ms 1 Gbps, O(1) ms 10 Gbps, O(100) ms 1 Gbps, O(100) ms

Impact of Networks on Latency-sensitive Jobs

5

Queries from 100 GB TPC Benchmarks

Job Completion Time (s) CDF

50 100 150 Query Completion Time (s) 0.00 0.25 0.50 0.75 1.00 CDF across Queries 10 Gbps, O(1) ms 1 Gbps, O(1) ms 10 Gbps, O(100) ms 1 Gbps, O(100) ms

Impact of Networks on Latency-sensitive Jobs

5

Queries from 100 GB TPC Benchmarks

4.9X Job Completion Time (s) CDF

Slow job execution in high-latency networks

Problem #1

50 100 150 Query Completion Time (s) 0.00 0.25 0.50 0.75 1.00 CDF across Queries 10 Gbps, O(1) ms 1 Gbps, O(1) ms 10 Gbps, O(100) ms 1 Gbps, O(100) ms

Impact of Networks on Latency-sensitive Jobs

5

Queries from 100 GB TPC Benchmarks

4.9X Job Completion Time (s) CDF

Slow job execution in high-latency networks

Problem #1

Control Plane Inefficiency Due to High Latency

6

Coordinator Worker Busy

O(1) ms

Time

Tasks Launch(■)

Slow job execution in high-latency networks

Problem #1

Control Plane Inefficiency Due to High Latency

6

Coordinator Worker Busy

O(1) ms

Time Busy

Tasks Tasks Launch(■) Complete(■) Launch(■) Complete(■)

Slow job execution in high-latency networks

Problem #1

Control Plane Inefficiency Due to High Latency

7

Coordinator Worker Busy

C

• m

p l e t e ( ■ ) L a u n c h ( ■ )

Time Busy

L a u n c h ( ■ ) Complete(■)

Idle

Tasks Tasks O(100) ms

Late-binding of tasks postpones scheduling

Impact of Networks on Bandwidth-intensive Jobs

8

Stage 1 Stage 2

Data transfers

• ver networks

Stage 3

Query25 on 1TB TPC benchmark

Impact of Networks on Bandwidth-intensive Jobs

8

Stage 1 Stage 2

Data transfers

• ver networks

Stage 3

Query25 on 1TB TPC benchmark

Resource utilization throughout the job

50 100 150 200 250 Time (s) 25 50 75 100 Percentage of the Total (%) Occupied CPUs CPU Util. B/w Util. Stage 1 Stage 2 Stage 3

Time (s)

Impact of Networks on Bandwidth-intensive Jobs

8

Stage 1 Stage 2

Data transfers

• ver networks

Stage 3

Query25 on 1TB TPC benchmark

50 100 150 200 250 Time (s) 25 50 75 100 Percentage of the Total (%) Occupied CPUs CPU Util. B/w Util.

Resource utilization throughout the job

Stage 1 Stage 2 Stage 3

Time (s)

Low CPU util.

Resource utilization throughout the job

50 100 150 200 250 Time (s) 25 50 75 100 Percentage of the Total (%) Occupied CPUs CPU Util. B/w Util. Stage 1 Stage 2 Stage 3

Time (s)

50 100 150 200 250 Time (s) 25 50 75 100 Percentage of the Total (%) Occupied CPUs CPU Util. B/w Util.

Resource utilization throughout the job

Stage 1 Stage 2 Stage 3

Time (s)

CPU underutilization in low-bandwidth networks

Data Plane Inefficiency Due to Low Bandwidth

9

Stage 1 Stage 2

Data transfers

• ver networks

Stage 3

Query25 on 1TB TPC benchmark

Tasks hog CPUs throughout the lifespan

Problem #2

Outline

10

• Today’s Execution Engines
• Sol Architecture
• Control Plane Design
• Data Plane Design
• Evaluation

Problem #1

High latency → Idleness of workers

Problem #2

Low b/w → CPU underutilization

Outline

11

• Today’s Execution Engines
• Sol Architecture
• Control Plane Design
• Data Plane Design
• Evaluation

A federated execution engine for diverse network conditions w/

• faster job execution
• higher resource utilization

Sol

LAN

Sol: A Federated Execution Engine

12

Sol Architecture

• Central Coordinator
• Coordinate inter-site executions

WAN

O(100) ms

Sol Coordinator

Task Arrivals Site 2 Site 3

WAN

O(100) ms

Site 1

LAN

Sol: A Federated Execution Engine

12

Sol Architecture

• Central Coordinator
• Coordinate inter-site executions
• Site Manager
• Coordinate local workers
• Manage queued tasks

WAN

O(100) ms

Sol Coordinator

Task Arrivals Site 2 Site 3

WAN

O(100) ms

Site Manager

LAN

LAN

Sol: A Federated Execution Engine

12

Sol Architecture

• Central Coordinator
• Coordinate inter-site executions
• Site Manager
• Coordinate local workers
• Manage queued tasks

WAN

O(100) ms

Sol Coordinator

Task Arrivals Site 2 Site 3

WAN

O(100) ms

Site Manager

LAN

LAN

Sol: A Federated Execution Engine

12

Sol Architecture

• Central Coordinator
• Coordinate inter-site executions
• Site Manager
• Coordinate local workers
• Manage queued tasks
• Task Manager
• Manage worker resource

WAN LAN

O(100) ms

Sol Coordinator

Task Arrivals Site 2 Site 3

WAN

O(100) ms

Worker

Task Manager

Worker

Task Manager

Site Manager

LAN

Outline

13

• Today’s Execution Engines
• Sol Architecture
• Control Plane Design
• Data Plane Design
• Evaluation

Push tasks proactively to reduce worker idle time Problem #1

High latency → Idleness of workers

Task Early-binding in Control Plane

14

Coordinator Worker Time

C

• m

p l e t e ( ■ )

Busy O(100) ms

Launch(■)

Idle

Launch(■)

Existing designs

Tasks Tasks

Task Early-binding in Control Plane

15

Coordinator Worker Time Site Manager O(100) ms O(1) ms

Tasks

Task Early-binding in Control Plane

15

Coordinator Worker Time Site Manager O(100) ms O(1) ms

Launch(■ ■) Tasks

Task Early-binding in Control Plane

15

Coordinator Worker Time Site Manager O(100) ms O(1) ms

Launch(■ ■) Launch(■) Complete(■) Busy Tasks

• Coordinator ⟷ Site Manager
• Inter-site operations are early-binding

→ Guarantee high utilization

Task Early-binding in Control Plane

15

Coordinator Worker Time

C

• m

p l e t e ( ■ )

Site Manager

Launch(■)

Idle Busy O(100) ms O(1) ms

Launch(■ ■) Launch(■) Complete(■) Busy Launch(■) Tasks Tasks

• Coordinator ⟷ Site Manager
• Inter-site operations are early-binding

→ Guarantee high utilization

Task Early-binding in Control Plane

15

Coordinator Worker Time

C

• m

p l e t e ( ■ )

Site Manager

Launch(■)

Idle Busy O(100) ms O(1) ms

Launch(■ ■) Launch(■) Complete(■) Busy Launch(■)

• Site Manager ⟷ Worker
• Intra-site operations are late-binding

→ Retain precise views

Tasks Tasks

Challenge 1.1: How Many Tasks to Push?

16

Coordinator Site Manager RTT

Tasks Tasks

Challenge 1.1: How Many Tasks to Push?

16

• Queue up too few
• Not enough work → Underutilization
• Queue up too many
• Scheduling too early → Suboptimal placement

Coordinator Site Manager RTT

Tasks Tasks

Challenge 1.1: How Many Tasks to Push?

16

• Queue up too few
• Not enough work → Underutilization
• Queue up too many
• Scheduling too early → Suboptimal placement
• Target:
• Total duration of queued tasks ≃ Round-Trip Time(RTT)

Coordinator Site Manager RTT

Tasks Tasks

• Sol works well w/o precise knowledge of task duration
• Hoeffding-bound (details in paper)

Challenge 1.1: How Many Tasks to Push?

16

• Queue up too few
• Not enough work → Underutilization
• Queue up too many
• Scheduling too early → Suboptimal placement
• Target:
• Total duration of queued tasks ≃ Round-Trip Time(RTT)

Coordinator Site Manager RTT

Tasks Tasks

Challenge 1.2: How to Push Tasks w/ Dependencies?

17

• Task placements depend on upstream outputs
• In order to reduce data transfers over networks

Challenge 1.2: How to Push Tasks w/ Dependencies?

17

• Task placements depend on upstream outputs
• In order to reduce data transfers over networks

C S1 S2

Time

T1 T2

Design in Existing Engines

T1 T2 T3

Task Dependencies

Challenge 1.2: How to Push Tasks w/ Dependencies?

17

• Task placements depend on upstream outputs
• In order to reduce data transfers over networks

Complete(T1)

C S1 S2

Time

T1 T2 Complete(T2)

Design in Existing Engines

T1 T2 T3

Task Dependencies

Output1 Output2

Challenge 1.2: How to Push Tasks w/ Dependencies?

17

• Task placements depend on upstream outputs
• In order to reduce data transfers over networks

Complete(T1)

C S1 S2

Time

T1 T2 Complete(T2) Start(T3) Launch(T3)

Design in Existing Engines

T1 T2 T3

Task Dependencies

Output1 Output2

Challenge 1.2: How to Push Tasks w/ Dependencies?

17

• Task placements depend on upstream outputs
• In order to reduce data transfers over networks

Complete(T1)

C S1 S2

Time

T1 T2 Complete(T2) Start(T3) Launch(T3)

Design in Existing Engines

T1 T2 T3

Task Dependencies

Output1 Output2

W/o full knowledge, pushing leads to tradeoff

Challenge 1.2: How to Push Tasks w/ Dependencies?

18

• 1. Sol improves utilization by pushing with speculation
• E.g., historical information

Complete(T1)

C S1 S2

Time

T1 T2 Complete(T2) Start(T3) Launch(T3)

Design in Existing Engines

Challenge 1.2: How to Push Tasks w/ Dependencies?

18

C S1 S2

Time

T1 T2

Push w/ Correct Speculations

T3

• 1. Sol improves utilization by pushing with speculation
• E.g., historical information

Complete(T1)

C S1 S2

Time

T1 T2 Complete(T2) Start(T3) Launch(T3)

Design in Existing Engines

Challenge 1.2: How to Push Tasks w/ Dependencies?

18

Complete(T1) Complete(T2)

C S1 S2

Time

T1 T2

Push w/ Correct Speculations

T3

Sol saves RTTs

• 1. Sol improves utilization by pushing with speculation
• E.g., historical information

Complete(T1)

C S1 S2

Time

T1 T2 Complete(T2) Start(T3) Launch(T3)

Design in Existing Engines

Activate(T3)

Challenge 1.2: How to Push Tasks w/ Dependencies?

19

• 2. In case of mistakes, Sol retains good scheduling by recovering
• With worker-initiated re-scheduling

Complete(T1)

C S1 S2

Time

T1 T2 Complete(T2) Start(T3) Launch(T3)

Design in Existing Engines

C S1 S2

Time

T1 T2

Challenge 1.2: How to Push Tasks w/ Dependencies?

19

Push under Mispredictions

• 2. In case of mistakes, Sol retains good scheduling by recovering
• With worker-initiated re-scheduling

Complete(T1)

C S1 S2

Time

T1 T2 Complete(T2) Start(T3) Launch(T3)

Design in Existing Engines

T3

C S1 S2

Time

T1 T2

Challenge 1.2: How to Push Tasks w/ Dependencies?

19

Push under Mispredictions

• 2. In case of mistakes, Sol retains good scheduling by recovering
• With worker-initiated re-scheduling

Complete(T1) Complete(T1)

C S1 S2

Time

T1 T2 Complete(T2) Start(T3) Launch(T3)

Design in Existing Engines

T3

C S1 S2

Time

T1 T2

Challenge 1.2: How to Push Tasks w/ Dependencies?

19

Push under Mispredictions

Cancel(T3)

• 2. In case of mistakes, Sol retains good scheduling by recovering
• With worker-initiated re-scheduling

Complete(T1) Complete(T1)

C S1 S2

Time

T1 T2 Complete(T2) Start(T3) Launch(T3)

Design in Existing Engines

C S1 S2

Time

T1 T2

Challenge 1.2: How to Push Tasks w/ Dependencies?

19

Complete(T2)

Push under Mispredictions

Start(T3) Re-Schedule(T3) Cancel(T3)

Sol does not make things worse

• 2. In case of mistakes, Sol retains good scheduling by recovering
• With worker-initiated re-scheduling

Complete(T1) Complete(T1)

C S1 S2

Time

T1 T2 Complete(T2) Start(T3) Launch(T3)

Design in Existing Engines

C S1 S2

Time

T1 T2

Task Early-binding in Control Plane

20

Complete(T2)

Push under Mispredictions

Start(T3) Re-Schedule(T3) Cancel(T3)

Sol retains good scheduling quality

• Sol improves utilization while retaining good scheduling quality

Complete(T1) T3 Complete(T1) Complete(T2)

C S1 S2

Time

T1 T2

Push w/ Correct Speculations

T3

Sol improves utilization

Activate(T3)

Outline

21

• Today’s Execution Engines
• Sol Architecture
• Control Plane Design
• Data Plane Design
• Evaluation

Decouple resource provisioning to improve CPU utilization Problem #2

Low b/w → CPU underutilization

Resource Decoupling in Data Plane

22

• Decouple the resource provisioning internally with
• Communication task: prepare data over networks

Resource Decoupling in Data Plane

22

• Decouple the resource provisioning internally with
• Communication task: prepare data over networks
• Computation task: perform computation on input

Resource Decoupling in Data Plane

22

• Decouple the resource provisioning internally with
• Communication task: prepare data over networks

Sol scales down CPU requirements and reclaims unused CPUs

• Computation task: perform computation on input

Challenge 2: How to Manage Jobs?

23

For bandwidth-intensive task

Challenge 2: How to Manage Jobs?

23

Incoming tasks Create

• comm. task

Y Large remote read?

Control flow of decoupling

For bandwidth-intensive task

Challenge 2: How to Manage Jobs?

23

Incoming tasks Create

• comm. task

Y Complete

• comm. ?

Large remote read?

Control flow of decoupling

For bandwidth-intensive task

Challenge 2: How to Manage Jobs?

23

• How many communication tasks to create?
• Too few → Network is not saturated
• Too many → CPUs are not saturated

Incoming tasks Create

• comm. task

Y Complete

• comm. ?

Large remote read?

Control flow of decoupling

For bandwidth-intensive task

Challenge 2: How to Manage Jobs?

23

• How many communication tasks to create?
• Too few → Network is not saturated
• Too many → CPUs are not saturated } Adapt to available bandwidth

Incoming tasks Create

• comm. task

Y Complete

• comm. ?

Large remote read?

Control flow of decoupling

For bandwidth-intensive task

Challenge 2: How to Manage Jobs?

24

Incoming tasks Create

• comm. task

Y Complete

• comm. ?

Control flow of decoupling

• How to manage the computation tasks?

Large remote read?

For bandwidth-intensive task

Challenge 2: How to Manage Jobs?

24

Incoming tasks Create

• comm. task

Y Complete

• comm. ?

Control flow of decoupling

Available CPUs N

• How to manage the computation tasks?

Pending tasks Large remote read?

For bandwidth-intensive task

Challenge 2: How to Manage Jobs?

24

Incoming tasks Create

• comm. task

Y Complete

• comm. ?

Control flow of decoupling

Available CPUs N

• How to manage the computation tasks?

Y Pending tasks Large remote read?

For bandwidth-intensive task

Challenge 2: How to Manage Jobs?

24

Incoming tasks Create

• comm. task

Y Complete

• comm. ?

Control flow of decoupling

Available CPUs N

• How to manage the computation tasks?
• Prioritize them when data is ready

Y Pending tasks Large remote read?

For bandwidth-intensive task

Challenge 2: How to Manage Jobs?

24

Incoming tasks Create

• comm. task

Y Complete

• comm. ?

Control flow of decoupling

Available CPUs N

• How to manage the computation tasks?
• Prioritize them when data is ready

Y Pending tasks Large remote read?

25

Evaluation

With a prototype supporting generic data processing

• Environment
• 10-site deployment in EC2
• 4 m4.4xlarge VMs in each site

Deployment over WAN

25

Evaluation

With a prototype supporting generic data processing 1. compared to existing engines? 2. across design space? 3. under uncertainties?

How does Sol perform:

• Environment
• 10-site deployment in EC2
• 4 m4.4xlarge VMs in each site

Deployment over WAN

Sol Improves Job Performance and Resource Util. (WAN)

26

Benchmark — multi-job execution

• Latency-sensitive TPC queries
• Bandwidth-intensive TeraSort

Baseline

• Apache Spark

Sol Improves Job Performance and Resource Util. (WAN)

26

Benchmark — multi-job execution

• Latency-sensitive TPC queries
• Bandwidth-intensive TeraSort

Baseline

• Apache Spark

100 101 102 103 Job Completion Time (s) 0.00 0.25 0.50 0.75 1.00 CDF Sol Spark

16.4x improvement on average

Sol Improves Job Performance and Resource Util. (WAN)

27

100 101 102 103 Job Completion Time (s) 0.00 0.25 0.50 0.75 1.00 CDF Sol Sol – Spark

16.4x improvement on average

Control-plane benefits (2.6x on avg.)

16.4x improvement on average

100 101 102 103 Job Completion Time (s) 0.00 0.25 0.50 0.75 1.00 CDF Sol Sol – Spark

Control plane benefits (2.6x on avg.)

Job Performance

Data plane benefits

Control Plane:

Early-binding → Less idle time

Sol w/o decoupling

Sol Improves Job Performance and Resource Util. (WAN)

27

100 101 102 103 Job Completion Time (s) 0.00 0.25 0.50 0.75 1.00 CDF Sol Sol – Spark

16.4x improvement on average

Data-plane benefits Control-plane benefits (2.6x on avg.)

16.4x improvement on average

100 101 102 103 Job Completion Time (s) 0.00 0.25 0.50 0.75 1.00 CDF Sol Sol – Spark

Control plane benefits (2.6x on avg.)

Job Performance

Data plane benefits

Control Plane:

Early-binding → Less idle time

Data Plane:

Decoupling → Less under-util.

Sol w/o decoupling

Sol Improves Job Performance and Resource Util. (WAN)

27

100 101 102 103 Job Completion Time (s) 0.00 0.25 0.50 0.75 1.00 CDF Sol Sol – Spark

16.4x improvement on average

Data-plane benefits Control-plane benefits (2.6x on avg.)

16.4x improvement on average

100 101 102 103 Job Completion Time (s) 0.00 0.25 0.50 0.75 1.00 CDF Sol Sol – Spark

Control plane benefits (2.6x on avg.)

Job Performance

Data plane benefits

Control Plane:

Early-binding → Less idle time

Data Plane:

Decoupling → Less under-util.

16.4x better job completion

+

1.8x better CPU util.

Sol w/o decoupling

Sol Performs Well Across Design Space (LAN)

28

High-bandwidth setting (10 Gbps)

1.3x improvement on average

100 101 102

Job Completion Time (s) 0.00 0.25 0.50 0.75 1.00 CDF Spark Sol

3.9x improvement on average

100 101 102

Job Completion Time (s) 0.00 0.25 0.50 0.75 1.00 CDF Spark Sol

Low-bandwidth setting (1 Gbps)

Sol

https://github.com/SymbioticLab/Sol

A federated execution engine for diverse network conditions with

• Faster job execution
• Higher resource utilization

Improve CPU util.{

before task executions → during task executions → Early-binding of tasks Decoupling of resource provisioning

29