Kairos: Preemptive Data Center Scheduling Without Runtime Estimates - - PowerPoint PPT Presentation

▶
kairos preemptive data center scheduling without runtime
SMART_READER_LITE
LIVE PREVIEW

Kairos: Preemptive Data Center Scheduling Without Runtime Estimates - - PowerPoint PPT Presentation

Mar 16, 2024 142 likes •490 views

Symposium on Cloud Computing (SoCC) Kairos: Preemptive Data Center Scheduling Without Runtime Estimates Pamela Delgado, Diego Didona, Florin Dinu and Willy Zwaenepoel October 11, 2018 1 Kairos Data center scheduling without task runtime

slide-1
SLIDE 1

Kairos: Preemptive Data Center Scheduling Without Runtime Estimates

1

Pamela Delgado, Diego Didona, Florin Dinu and Willy Zwaenepoel

October 11, 2018

Symposium on Cloud Computing (SoCC)

slide-2
SLIDE 2

Kairos

Kairos: Preemptive Data Center Scheduling Without Runtime Estimates | SoCC’18 2

Data center scheduling without task runtime estimates

slide-3
SLIDE 3

Kairos key idea

3 Kairos: Preemptive Data Center Scheduling Without Runtime Estimates | SoCC’18

  • New preemption approach

✓ No head-of-line blocking ✓ Good scheduling performance

slide-4
SLIDE 4

Data center scheduling challenge

4 Kairos: Preemptive Data Center Scheduling Without Runtime Estimates | SoCC’18

Motivation

  • Heavy-tailed workloads

… cluster

scheduler

slide-5
SLIDE 5

Problem: head-of-line blocking

5 Kairos: Preemptive Data Center Scheduling Without Runtime Estimates | SoCC’18

Motivation

  • Short waiting for long
  • High likelihood

…

slide-6
SLIDE 6

Historical use of runtime estimates

6 Kairos: Preemptive Data Center Scheduling Without Runtime Estimates | SoCC’18

Motivation

per-task estimations dual classification no estimations Eagle’16 Hawk’15 Sparrow’13 Apollo’14 Mercury*’15 Borg’15 Tetrisched’16 Firmament’16 Yaq’16 Yarn’13 Do not avoid head-of-line! Depend on runtime estimates

…

slide-7
SLIDE 7

Hard to obtain reliable estimates

7 Kairos: Preemptive Data Center Scheduling Without Runtime Estimates | SoCC’18

Motivation

  • Mis-estimations happen
  • unseen jobs, skewed input, failures/spikes
  • Consequences:
  • poor scheduling decisions*, violate SLOs^
  • complex designs to compensate

*Job-aware scheduling in Eagle: Divide and Stick to Your Probes (SoCC’16) ^ Tetrisched: global rescheduling with adaptive plan-ahead in dynamic heterogeneous clusters (Eurosys’16)

slide-8
SLIDE 8

8 Kairos: Preemptive Data Center Scheduling Without Runtime Estimates | SoCC’18

Motivation

Can we dispense with task runtime estimates altogether?

slide-9
SLIDE 9

9 Kairos: Preemptive Data Center Scheduling Without Runtime Estimates | SoCC’18

Motivation

✓ Avoid head-of-line blocking ✓ No task runtime estimates

Kairos

Can we dispense with task runtime estimates altogether?

slide-10
SLIDE 10

Kairos insight

10 Kairos: Preemptive Data Center Scheduling Without Runtime Estimates | SoCC’18

Kairos Preemption

Use preemption!!

slide-11
SLIDE 11

Preemption in Kairos

11 Kairos: Preemptive Data Center Scheduling Without Runtime Estimates | SoCC’18

Kairos Architecture

Costly resuming elsewhere: Do preemption locally!

Preempt long!

…

slide-12
SLIDE 12

Kairos architecture

12 Kairos: Preemptive Data Center Scheduling Without Runtime Estimates | SoCC’18

… Kairos centralized scheduler

Kairos node scheduler

Node j

Kairos node scheduler

Node x

Kairos node scheduler

Node y

Local preemption Load balancing

Kairos Architecture

Local preemption Local preemption

Distributed component Centralized component

slide-13
SLIDE 13

Kairos architecture

13 Kairos: Preemptive Data Center Scheduling Without Runtime Estimates | SoCC’18

… Kairos centralized scheduler

Kairos node scheduler

Node j

Kairos node scheduler

Node x

Kairos node scheduler

Node y

Local preemption Load balancing

Kairos Architecture

Local preemption Local preemption

slide-14
SLIDE 14

Kairos architecture

14 Kairos: Preemptive Data Center Scheduling Without Runtime Estimates | SoCC’18

… Kairos centralized scheduler

Kairos node scheduler

Node j

Kairos node scheduler

Node x

Kairos node scheduler

Node y

Local preemption Load balancing

Kairos Architecture

Local preemption Local preemption

slide-15
SLIDE 15

Least-Attained Service (LAS)

15 Kairos: Preemptive Data Center Scheduling Without Runtime Estimates | SoCC’18

  • Preemptive policy
  • Give resources to task that received least service

✓New task runs immediately ✓Runs as long as it is the one with least received service

slide-16
SLIDE 16

LAS rationale

16 Kairos: Preemptive Data Center Scheduling Without Runtime Estimates | SoCC’18

  • Good for heavy-tailed workloads*
  • Benefits:

1.Shorter tasks have priority (no head-of-line blocking) 2.Shorter tasks –very likely– execute until completion

*Performance modeling and design of computer systems: queueing theory in action M. Harchol-Balter 2013

slide-17
SLIDE 17

Kairos distributed scheduling

17 Kairos: Preemptive Data Center Scheduling Without Runtime Estimates | SoCC’18

Kairos Distributed scheduling

  • Node schedulers
  • LAS at the nodes

Kairos node scheduler Kairos node scheduler Kairos node scheduler

How to dispatch tasks among nodes?

… … … …

slide-18
SLIDE 18

Kairos architecture

18 Kairos: Preemptive Data Center Scheduling Without Runtime Estimates | SoCC’18

… Kairos centralized scheduler

Kairos node scheduler

Node j

Kairos node scheduler

Node x

Kairos node scheduler

Node y

Local preemption Load balancing

Kairos Architecture

Local preemption Local preemption

slide-19
SLIDE 19

Kairos centralized scheduling

19 Kairos: Preemptive Data Center Scheduling Without Runtime Estimates | SoCC’18

Kairos Centralized scheduling

… Kairos centralized scheduler

Kairos node scheduler

Node j

Kairos node scheduler

Node x

Kairos node scheduler

Node y

? 1

1st Load balancing 2nd Maximize LAS effectiveness

4 2 4

slide-20
SLIDE 20

Load balancing rationale

20 Kairos: Preemptive Data Center Scheduling Without Runtime Estimates | SoCC’18

Kairos Centralized scheduling

…

Kairos node scheduler

Node j

Kairos node scheduler

Node y

  • 1. Lowest # tasks: no idle nodes
  • Bound max # tasks

0 tasks 100 tasks

…

  • 1. Avoid!
slide-21
SLIDE 21

Load balancing rationale

21 Kairos: Preemptive Data Center Scheduling Without Runtime Estimates | SoCC’18

Kairos Centralized scheduling

Kairos node scheduler

Node j

Kairos node scheduler

Node y

*Minimizing total flow time and total completion time with immediate dispatching. Avrahami et.al. 2003 Multi-layered round robin routing for parallel servers Down et.al. 2006

  • 2. LAS-aware policy break ties:
  • Heavy-tailed for each node
  • Maximize LAS effectiveness
  • Node with lowest AS variance*
  • 2. Avoid!
  • nly short
  • nly long
slide-22
SLIDE 22

Kairos recap

22 Kairos: Preemptive Data Center Scheduling Without Runtime Estimates | SoCC’18

Kairos

1. Distributed: ✓LAS node level 2. Centralized: ✓LAS-aware load balancing technique

slide-23
SLIDE 23

Evaluation

23 Kairos: Preemptive Data Center Scheduling Without Runtime Estimates | SoCC’18

Kairos Evaluation

  • Yarn and Docker containers
  • 120 cores in 30 nodes
  • heavy-tailed workload (100 jobs)
  • Metrics: Job runtime and slowdown
  • Compare to: Big-C [ATC’17], FIFO
  • Simulation: Google trace, compare to Eagle [SoCC’16]
slide-24
SLIDE 24

What is the slowdown?

24 Kairos: Preemptive Data Center Scheduling Without Runtime Estimates | SoCC’18

Evaluation

𝑘𝑝𝑐 𝑡𝑚𝑝𝑥𝑒𝑝𝑥𝑜 = 𝑝𝑐𝑡𝑓𝑠𝑤𝑓𝑒 𝑘𝑝𝑐 𝑠𝑣𝑜𝑢𝑗𝑛𝑓 𝑣𝑜𝑑𝑝𝑜𝑢𝑓𝑜𝑒𝑓𝑒 𝑘𝑝𝑐 𝑠𝑣𝑜𝑢𝑗𝑛𝑓

Best job slowdown = 1

slide-25
SLIDE 25

Kairos vs Big-C and FIFO

25 Kairos: Preemptive Data Center Scheduling Without Runtime Estimates | SoCC’18

Kairos Evaluation

20 40 60 80 100 120 2 4 6 8 10 12 14 16 18 20

CDF Job slowdown

Kairos Big-C FIFO

Job slowdown

better

slide-26
SLIDE 26

Kairos vs Big-C and FIFO

26 Kairos: Preemptive Data Center Scheduling Without Runtime Estimates | SoCC’18

Kairos Evaluation

20 40 60 80 100 120 2 4 6 8 10 12 14 16 18 20

CDF Job runtime/expected job runtime

Kairos Big-C FIFO

Job slowdown

Slowdown in Kairos <1.8X

better

slide-27
SLIDE 27

20 40 60 80 100 120 500 1000 1500 2000 2500 3000 3500 4000

CDF Job runtime [s]

Kairos Big-C FIFO

Kairos vs Big-C and FIFO

27 Kairos: Preemptive Data Center Scheduling Without Runtime Estimates | SoCC’18

Kairos Evaluation

Job running times

2X 1.6X 2.3X

better

slide-28
SLIDE 28

20 40 60 80 100 120 500 1000 1500 2000 2500 3000 3500 4000

CDF Job runtime [s]

Kairos Big-C FIFO

Kairos vs Big-C and FIFO

28 Kairos: Preemptive Data Center Scheduling Without Runtime Estimates | SoCC’18

Kairos Evaluation

Job running times

2X 1.6X 2.3X

better

Kairos better across the board

slide-29
SLIDE 29

Kairos vs Eagle

29 Kairos: Preemptive Data Center Scheduling Without Runtime Estimates | SoCC’18

0,2 0,4 0,6 0,8 1 1,2 1,4

Kairos/Eagle Cluster load

50th 90th 99th

Lower Higher

better

  • Short jobs runtime
  • Google trace
slide-30
SLIDE 30

Kairos vs Eagle

30 Kairos: Preemptive Data Center Scheduling Without Runtime Estimates | SoCC’18

0,2 0,4 0,6 0,8 1 1,2 1,4

Kairos/Eagle Cluster load

50th 90th 99th

Lower Higher

better

  • Short jobs runtime
  • Google trace

Kairos works well at large scale

slide-31
SLIDE 31

Why are we better?

31 Kairos: Preemptive Data Center Scheduling Without Runtime Estimates | SoCC’18

Kairos Evaluation

Against FIFO ✓ FIFO does not avoid head-of-line Against Big-C ✓ We do preemption better Against Eagle ✓ Preemption

slide-32
SLIDE 32

More in the paper

32 Kairos: Preemptive Data Center Scheduling Without Runtime Estimates | SoCC’18

  • Evaluation with a uniform workload
  • Sensitivity to parameters
  • Comparison with other load balancing techniques
  • How we do preemption
  • Soon open sourced
slide-33
SLIDE 33

Kairos

33 Kairos: Preemptive Data Center Scheduling Without Runtime Estimates | SoCC’18

Kairos Summary

✓First preemptive scheduler without runtime estimates ✓Smart preemption: good job runtime and slowdown ✓LAS at node level ✓LAS-aware load balancing