Load Balancing for Interdependent IoT Microservices Ruozhou Yu, - - PowerPoint PPT Presentation

Load Balancing for Interdependent IoT Microservices

Ruozhou Yu, Vishnu Teja Kilari, Guoliang Xue Arizona State University Dejun Yang Colorado School of Mines

Outlines

2

Background and Motivation System Modeling Algorithm Design and Analysis Performance Evaluation Discussions, Future Work and Conclusions

IoT: The Future Internet

3

q IoT is the future Internet that connects every aspect of our work and life.

Environment Agriculture Shopping Manufacturing Transportation Home Healthcare Travel Security

Example IoT Applications

4

Traffic Analysis Image Processing Object Detection Temporal Analysis Traffic Prediction

Monolithic Applications

5

Traffic Analysis Image Processing Object Detection Temporal Analysis Traffic Prediction Crowd Detection Signal Processing Device Matching Temporal Analysis Population Estimation Criminal Searching Image Processing Object Detection Face Recognition Face Comparison

Temporal Analysis Face Comparison Traffic Prediction Population Estimation Device Matching Signal Processing

Microservices

6

Traffic Analysis Crowd Detection Criminal Searching Image Processing Object Detection Face Recognition

Microservices vs. Edge Computing

7

High delay High bw overhead Single point of failure Security concerns Responsive Low bw overhead Distributed Secure

Cloud-based Edge-based

The Microservice Load Balancing Problem

q Edge-based microservices can be easily saturated.

8

Image Processing

Image Processing 1 Image Processing

The Microservice Load Balancing Problem

q Edge-based microservices can be easily saturated.

9

Image Processing 2

The Microservice Load Balancing Problem

q Edge-based microservices can be easily saturated. q Challenge: interdependent microservices.

10

Image Processing 1 Image Processing 2 Temporal Analysis 1 Traffic Prediction Object Detection 1 Object Detection 2 Object Detection 3 Temporal Analysis 2

Our Approach: Overview

11

Problem Modeling 1) DAG-based interdependency graph (App-Graph). 2) Compactly modeled infrastructure (Inf-Graph). 3) Flexible application instantiation (Real-Graph). 4) Joint instantiation finding & load allocation. 5) Application QoS requirements. Algorithmic Results 1) Optimal algorithm for QoS-agnostic problem. 2) NP-hardness for QoS-aware problem. 3) FPTAS for QoS-aware problem. Next Steps (Future Work) 1) Network-aware load balancing. 2) Reliability and security. 3) Economics-aware microservice composition.

Outlines

12

Background and Motivation System Modeling Algorithm Design and Analysis Performance Evaluation Discussions, Future Work and Conclusions

Application with Interdependent Microservices

q General DAG-based application graph (App-Graph).

v Captures complex interdependencies, unlike existing line graph-based models.

13

Cloud Storage Cloud Storage Video Proc. Video Proc. Motion Detect. Motion Detect. Anomaly Detect. Anomaly Detect. Smart Home Control Smart Home Control Data Proc. Data Proc. Cmd. Center Cmd. Center Security Cameras Motion Sensors Ambient Sensors Data Input Data Input API Call API Call Microservice Microservice Microservice Data Input API Call Microservice

Interdependency edge

• Data distribution ratio: output / input.

Microservice

IoT Infrastructure in the Application’sView

q Inf-Graph: deployed microservice instances & their interactions.

14

A B C D E

0.5 0.5 1.0 1.0 1.0 1.0

App-Graph

A1 A1 B1 B1 C1 C1 D1 D1 E1 E1 D2 D2 E2 E2

δ=2.0 (2, 1) (1, 5) (1, 4) (2, 3) (2, 6) (2, 4) (2, 6)

A1 B1 C1 D1 E1 D2 E2

δ=2.0 (2, 1) (1, 5) (1, 4) (2, 3) (2, 6) (2, 4) (2, 6) Inf-Graph

Microservice Instance

• Capacity: input data volume.
• Processing delay.

Inter-Instance Communication Channel

• Transmission delay (=0 in example).

External Demand

Application Instantiation

q Real-Graph: instantiating the App-Graph in the Inf-Graph.

15

A1 A1 B1 B1 C1 C1 D1 D1 E1 E1 D2 D2 E2 E2

δ=2.0 (2, 1) (1, 5) (1, 4) (2, 3) (2, 6) (2, 4) (2, 6)

A1 B1 C1 D1 E1 D2 E2

δ=2.0 (2, 1) (1, 5) (1, 4) (2, 3) (2, 6) (2, 4) (2, 6)

A1 A1 B1 B1 C1 C1 D1 D1 E1 E1 D2 D2 E2 E2

δ=2.0 (2, 1) (1, 5) (1, 4) (2, 3) (2, 6) (2, 4) (2, 6)

A1 B1 C1 D1 E1 D2 E2

δ=2.0 (2, 1) (1, 5) (1, 4) (2, 3) (2, 6) (2, 4) (2, 6)

Inf-Graph

A1 B1 C1 D1 E1 D2 E2

(1, 1) (0.5, 6) (0.5, 5) (1, 9) (1, 13) (1, 7) (1, 13) φ=2.0 A1

B1 C1 D1 E1 D2 E2

(1, 1) (0.5, 6) (0.5, 5) (1, 9) (1, 13) (1, 7) (1, 13) φ=2.0

Real-Graph

Selected Instance

• Impact ratio: how much data this instance

gets if 1 unit is allocated to the source.

• Max cumulative delay from source.
• Both can be computed based on this graph.

Source Demand Allocation Source of Demand

• A real-graph has a single source of demand.

(There could be multiple sources in Inf-Graph.)

Problem Statement: Overview

q Inputs:

v App-Graph: microservices, interdependencies, data distribution ratios v Inf-Graph: instances, communication channels, capacities, and delays

q Outputs:

v A set of Real-Graphs. v External demand allocation for each Real-Graph.

q Constraints:

v Load balancing: total load <= instance capacity * 𝛺. v QoS awareness: maximum delay <= D.

q Objective (optimization version):

v Minimize maximum delay of all Real-Graphs.

16

Outlines

17

Background and Motivation System Modeling Algorithm Design and Analysis Performance Evaluation Discussions, Future Work and Conclusions

BLB: Basic (QoS-agnostic) Load Balancing

q Without delay constraint, the problem can be formulated as LP.

18

Variables: Per-link demand allocation function. Demand per node = external + flow-in.

find f : L 7! R∗ s.t. δn = δext

n

+ X

l∈Lin(n)

f(l), 8n 2 N; δn  Ψ · cn, 8n 2 N; r(vn,w)δn = X

l∈Lout(n,w)

f(l), 8n, w 2 Vout(vn).

Capacity (load balancing) per node. Flow conservation: sum flow towards all instances of a downstream microservice w = input data * data distribution ratio of w.

BLB: Basic (QoS-agnostic) Load Balancing

q Without delay constraint, the problem can be formulated as LP.

19

Variables: Per-link demand allocation function. Demand per node = external + flow-in.

find f : L 7! R∗ s.t. δn = δext

n

+ X

l∈Lin(n)

f(l), 8n 2 N; δn  Ψ · cn, 8n 2 N; r(vn,w)δn = X

l∈Lout(n,w)

f(l), 8n, w 2 Vout(vn).

Capacity (load balancing) per node. Flow conservation: sum flow towards all instances of a downstream microservice w = input data * data distribution ratio of w.

Theorem 1: BLB is optimally solvable in polynomial time.

Real-Graph DecompositionTheorem

q Why do we need such a theorem?

1. Transform any solution of BLB into a set of implementable real-graphs. 2. Define QoS of a load balancing plan (max delay of all real-graphs).

20

Theorem 2: Every demand allocation function f so defined can be decomposed into at most |N| + |L| real-graphs with positive source demands.

QLB: QoS-aware Load Balancing

q Fully Polynomial-Time Approximation Scheme (FPTAS) can achieve the best trade-off between time and accuracy

v Approximation ratio: (1+𝜗) – For maximization problem v Time complexity: O(poly(1/𝜗) × poly(input)) v In practice, one can arbitrarily tune 𝜗 to get best accuracy within time limit.

21

Theorem 3: QLB (optimization version) is NP-hard. Theorems 4&5: QLB (optimization version) admits an FPTAS.

A Brief Overview of Our FPTAS

q Idea:

v Pseudo-polynomial time algorithm:

Ø Expand Inf-Graph into a delay-layered graph. Ø Run BLB LP on the expanded graph.

v Discretization via approximate testing:

Ø Find delay lower & upper bounds (UB, LB) s.t. UB <= poly(input) * LB. Ø Discretize delay values based on (UB, LB). Ø Run pseudo-polynomial time algorithm. Ø Refine (UB, LB) based on output.

v Efficiency enhancement:

Ø Approximate testing to shrink initial bound s.t. UB <= constant * LB.

22

Outlines

23

Background and Motivation System Modeling Algorithm Design and Analysis Performance Evaluation Discussions, Future Work and Conclusions

Simulation Settings

q Simulated network scenarios:

v App-Graph:

Ø 5-layered applications, layer-1 being the source layer Ø 10-70 microservices: 10% in layer-1, uniformly distributed in other layers Ø 4 in-going edges per microservice in layers 2-5 Ø Data distribution ratio: uniformly generated

v Inf-Graph:

Ø 1 instance per microservice in source layer, 5-15 in others Ø Linking probability (between interdependent instances): 0.3 Ø Source demands: 100-900 units Ø Node capacities: 10-90 units Ø Node/Link delays: 0-500/1000 ms

v Load balancing goal: optimal load under BLB, or 2 x optimal load under BLB v Approximation parameter: 𝜗=0.5

q Comparisons:

v QLB v BLB v QHU: QoS-aware heuristic, solving BLB minimizing demand-weighted delay

24

Comparison Results

25

Running time Polynomially increased Max delay (QoS) QLB has improved delay

• ver the other solutions.

Outlines

26

Background and Motivation System Modeling Algorithm Design and Analysis Performance Evaluation Discussions, Future Work and Conclusions

Other Perspectives and Beyond

q So far, we’ve talked about

v Basic model: DAG-based apps, Real-Graphs v Processing capacities and delays v Network delays

q What we didn’t consider in this work

v Network topology v Network capacities & congestion v Routing v Reliability: microservice instance failures v Incentives, pricing v Payment methods

q A unified approach is still in need for high-performance IoT.

27

Computing Perspective Networking Perspective Security Perspective Economics Perspective

Our Conclusions

q Load Balancing for Interdependent IoT Microservices

v DAG model for applications:App-Graph and Inf-Graph v Application realization with Real-Graph abstraction v System-wide load balancing with QoS (delay) constraints

q Algorithmic solutions

v Optimal solution for QoS-agnostic problem v FPTAS for (NP-hard) QoS-aware problem

q Future directions

v Unified framework for IoT performance optimization

28

Thank you very much!

Q&A?

29

NP-Hardness Proof

v0 v0 u1 u1 v1 v1

1.0 1.0

u2 u2 v2 v2 uκ uκ vκ vκ

1.0 1.0 1.0

vκ-1 vκ-1

1.0

v0 u1 v1

1.0 1.0

u2 v2 uκ vκ

1.0 1.0 1.0

vκ-1

1.0

n0 n0 m1 m1 m1 m1 n1 n1 m2 m2 m2 m2 n2 n2 mκ mκ mκ mκ nκ nκ

1 1 1

δ=2.0

nκ-1 nκ-1 n0 m1 m1 n1 m2 m2 n2 mκ mκ nκ

1 1 1

δ=2.0

nκ-1

d=axκ d=axκ d=ax2 d=ax2 d=ax1 d=ax1 App-graph: Inf-graph:

30