Energy-aware scheduling for asymmetric distributed systems - - PowerPoint PPT Presentation

▶
energy aware scheduling for asymmetric distributed systems
SMART_READER_LITE
LIVE PREVIEW

Energy-aware scheduling for asymmetric distributed systems - - PowerPoint PPT Presentation

Aug 24, 2023 260 likes •754 views

Energy-aware scheduling for asymmetric distributed systems Non-homogeneous systems Emerging and attractive alternative to homogeneous systems o improved performance and energy efficiency benefits Different server types (large/small) are

slide-1
SLIDE 1

Energy-aware scheduling for asymmetric distributed systems

slide-2
SLIDE 2

Mosse: HetCMP+energy

Non-homogeneous systems

  • Emerging and attractive alternative to

homogeneous systems

  • improved performance and energy efficiency benefits
  • Different server types (large/small) are used to
  • run each request on a server type that is best suited for it
  • satisfy time-varying demands (e.g., compute-intensive or

memory-intensive) of a range of threads

  • Different hardware capabilities
  • Cache size
  • Frequency
  • Architecture
  • ….
slide-3
SLIDE 3

Mosse: HetCMP+energy

Challenges of Distributed Systems

  • Assignment: match threads and core/memory
  • Dynamic vs static scheduling
  • Real-time vs general purpose
  • Global vs partitioned scheduling
  • Cache partition vs cache sharing
  • Inclusive vs exclusive cache
  • Bus bandwidth partitioning vs sharing
  • Memory allocation
  • Memory bank distribution
  • …
slide-4
SLIDE 4

Mosse: HetCMP+energy

Typical datacenter workload

* Meisner et al. Power management of online data-intensive services. ISCA 2011

Load fluctuation and power consumption of Web-search running on Google servers *

(QPS = Queries Per Second)

Energy consumption is not proportional to the amount of computation!

slide-5
SLIDE 5

Mosse: HetCMP+energy

Typical server workload: Twitter

Source: ASPLOS 14, Delimitrou

slide-6
SLIDE 6

Introduction

The opportunity

10/29/18 CS3530 - Advanced Topics in Distributed and Real-time

Deadlines are pessimistic and based on worst-case execution time.

Phase 1 Phase 2 Phase 3 Deadline

Frames over time X264 Video Encoding on 4 big cores

big LITTLE big Opportunity to save energy!!!

slide-7
SLIDE 7

Mosse: HetCMP+energy

Big brawny cores achieve lower latency at all load levels

tail latency: meet QoS of 90% of requests…

Web-search running on Intel QuickIA

Performance: latency

But small wimpy cores still meet the QoS at low load using much less power!

slide-8
SLIDE 8

Mosse: HetCMP+energy

Insight: Exploit load fluctuation to improve energy efficiency and meet QoS

Scheduling HetCMP

  • Low load: Wimpy cores to reduce

power with satisfactory QoS

slide-9
SLIDE 9

Mosse: HetCMP+energy

Scheduling HetCMP

  • High load: Brawny cores

to guarantee QoS

slide-10
SLIDE 10

Introduction

The opportunity

10/29/18 CS3530 - Advanced Topics in Distributed and Real-time

Deadlines are pessimistic and based on worst-case execution time.

Phase 1 Phase 2 Phase 3 Deadline

Frames over time X264 Video Encoding on 4 big cores

big LITTLE big Opportunity to save energy!!!

slide-11
SLIDE 11

Mosse: HetCMP+energy

  • Tension between responsiveness and stability
  • Responsiveness

§ short task migration interval quickly reacts, capturing time- varying workload fluctuations

  • Stability

§ Avoid over-reactionto load fluctuations; it can cause

  • scillatory behavior

§ Consider system settling time (observe the effects of task migrations)

Challenges

slide-12
SLIDE 12

Responsiveness and stability

Slow reaction…

QoS violations!

Fast reaction!

QoS violations!

Over-reaction!!!

slide-13
SLIDE 13

Mosse: HetCMP+energy

1) PID control system

  • pros: well-known control methodology
  • cons: parameter tuning via extensive offline app profiling

2) Deadzone-based control system

  • pros: simple online scheme based on QoS thresholds
  • cons: sensitive to threshold parameter selection

Two Designs

  • Can either effectively provide high QoS while maximizing

energy efficiency?

  • Responsiveness and Stability
slide-14
SLIDE 14

Mosse: HetCMP+energy

Design 1: PID control system

monitored QoS

QoS target (e.g., 90%-tile latency)

GOAL: To keep the controlled system running as close as possible to its specified QoS target

slide-15
SLIDE 15

LUCIANO BERTINI – FeBID 2007 – Munich, Germany, May 25th, 2007

QoS Metric / Control Variable

[ ]

p x tardiness = ≤ Pr

x → p-quantile

slide-16
SLIDE 16

LUCIANO BERTINI – FeBID 2007 – Munich, Germany, May 25th, 2007

QoS Metric / Control Variable

[ ]

p x tardiness = ≤ Pr

x → p-quantile

slide-17
SLIDE 17

Mosse: HetCMP+energy

PID Control Mapping

  • Task-to-core mapping
  • Mapping from the continuous PID output to a discrete task-core mapping
  • Parameter selection/tuning
  • Classical control system method, root locus (Hellerstein et al. 2004), is

used to determine Kp, Ki, Kd parameter § Responsiveness and stability

slide-18
SLIDE 18

Mosse: HetCMP+energy

Violations

PID control: web-search

QoS Core Mapping Throughput

slide-19
SLIDE 19

Design 2: Deadzone State Machine

QoS alert: QoS variable > QoS target * UP_THR QoS safe: QoS variable < QoS target * DOWN_THR

The deadzone thresholds impact the stability of the mapping algorithm!

slide-20
SLIDE 20

Mosse: HetCMP+energy

Stability: deadzone parameters

Web-search execution with UP thr=0.8, DOWN thr=0.3

QoS Core Mapping Throughput

High QoS violations occur due to oscillatory behavior!

slide-21
SLIDE 21

Mosse: HetCMP+energy

Another challenge!

Power-efficient cores (e.g., Intel Atom) High performance core (e.g., Intel Core2 / Xeon)

Shared resource => Contention / bottleneck

slide-22
SLIDE 22

Mosse: HetCMP+energy

Benchmark thread characterization

Some observations: (1) Both MIPS and LLCM can be increased, such as milc (64M LLCM, 2K MIPS) when compared to mcf (18M LLCM, 0.4K MIPS) (2) Very similar MIPS can lead to very different LLCM, such as lbm (48M LLCM, 2.4K MIPS) and cactusADM (8M LLCM, 2.3K MIPS)

slide-23
SLIDE 23

Mosse: HetCMP+energy

Schedule!

  • Having characterized the thread…
  • SCHEDULE IT!! No, schedule THEM!!!
  • However, there is a problem…

phases….

slide-24
SLIDE 24

Mosse: HetCMP+energy

Thread performance demands

slide-25
SLIDE 25

Mosse: HetCMP+energy

Schedule!

  • NOW I understand the problem AND I have

the better characterization, therefore

  • Schedule it! Schedule them!!!
  • Bias Scheduling:
  • Use memory intensity (LLC miss rate) as a bias to

guide thread scheduling

  • highest (lowest) bias threads scheduled on small

(big) cores

slide-26
SLIDE 26

Mosse: HetCMP+energy

energy efficiency (SPEC 2006)

Performance-asymmetric multi-core processor: Quad-core x86_64 processor: big core (3.2Ghz) and small core (0.8Ghz)

  • Avg. power consumption ("Web Search Using Mobile Cores" ISCA’10):

Big core (Intel Xeon): 15.63 W Small core (Intel Atom): 1.6 W

slide-27
SLIDE 27

Mosse: HetCMP+energy

energy efficiency (SPEC 2006)

bias (LLCM) ~= 13K bias (LLCM) ~= 14K

Very similar bias measures but each thread should run energy efficiently on different core types

slide-28
SLIDE 28

Mosse: HetCMP+energy

energy efficiency (SPEC 2006)

bias (LLCM) ~= 29K

Despite being high memory-intensive (small core bias), bwaves could run on a big core type for improved energy efficiency

slide-29
SLIDE 29

Mosse: HetCMP+energy

Schedule differently!

  • NOW I understand the problem AND I have

the better characterization AND bias against memory intensity doesn’t work, therefore

  • Schedule it! Schedule them!!!
  • IPC-based Scheduling:
  • Use CPU intensity (measured IPC) to guide thread

scheduling

  • threads with highest (lowest) IPC scheduled on big

(small) cores

è Different heuristic, different day

slide-30
SLIDE 30

Mosse: HetCMP+energy

Trouble in paradise

  • single metric cannot clearly characterize

some threads and schedule them to the right core type

  • unawareness of core power usage may

allow suboptimal energy-efficient decisions

  • inherently unfair thread scheduling may

cause performance loss (big core monopoly)

slide-31
SLIDE 31

Mosse: HetCMP+energy

Return to challenges

  • Assignment: match threads and core/memory
  • How to characterize threads

§ How to choose counters § How many counters § Which counters?

  • Dynamic vs static scheduling
  • Global vs partitioned scheduling
  • Cache partition vs cache sharing
  • Inclusive vs exclusive cache
  • Bus bandwidth partitioning vs sharing
  • Memory allocation
  • Memory bank distribution
slide-32
SLIDE 32

Mosse: HetCMP+energy

Optimization+Control Approach

thread characte rization MODELING

solution Prediction !!!!

slide-33
SLIDE 33

Mosse: HetCMP+energy

Integer programming formulation

slide-34
SLIDE 34

Mosse: HetCMP+energy

Integer programming formulation

The objective function aims to minimize (in fact, maximize the inverse)

  • f the energy delay product per instruction, given by Watt / IPS^2;

that is, minimize both the energy and the amount of time required to execute thread instructions

slide-35
SLIDE 35

Mosse: HetCMP+energy

Integer programming formulation

Computational and memory capacity constraints

slide-36
SLIDE 36

Mosse: HetCMP+energy

Integer programming formulation

Each thread is assigned to a given core type

slide-37
SLIDE 37

Mosse: HetCMP+energy

Schedule differently!

  • NOW I REALLY understand the problem

AND I have the better characterization AND bias against memory intensity doesn’t work, therefore I know I have to take into account both types of counters.

slide-38
SLIDE 38

Mosse: HetCMP+energy

Application performance prediction

Oops, forgot something: the performance of a thread currently running on a given server type when assigned to run on a different server type?

  • ne approach:
  • 1. collect performance data from a representative set of workloads,

running each thread individually on each core type

  • 2. establish and solve a linear regression model

IPSbig = w1 * IPSsmall + w2 * MPSsmall + w3 IPSsmall = w4 * IPSbig + w5 * MPSbig + w6

  • ther approaches: Machine Learning, statistics, tarot…

Such a performance characterization needs to be done once at design stage.

slide-39
SLIDE 39

Prediction analysis

astar SPEC benchmark bwaves SPEC benchmark

Performance data collected from a small core to predict the performance on a big core

slide-40
SLIDE 40

Mosse: HetCMP+energy

What else????

  • Non-volatile memories (PCM? STT-RAM?)
  • Hybrid memory architecture
  • Migration of pages during runtime
  • Smart allocation of pages, cache sizes, bandwidth
  • Implementation in the OS scheduler
  • Currently we’re using affinity provided by linux
  • Modification of the lottery scheduling algorithm
  • Ticket inflation based on performance
  • Re-inforcement learning scheduler
slide-41
SLIDE 41

Mosse: HetCMP+energy

Past work: Proportional Share Scheduling

  • Adapt Lottery Scheduling
  • More tickets for more ED gains
  • Results/reality: threads can migrate too often

between cores of different types

  • threads’ cache affinity is decreased
  • excessive migrations may cause performance loss
  • Ticket inflation:
  • threads that are already running on a big core will

get additional tickets

  • help preserve cache affinity
slide-42
SLIDE 42

Adding Reinforcement Learning

Project started as a graduateclass project

  • “Leveraging reinforcement learning for energy-efficient dynamic thread

assignment in heterogeneous multi-core systems”

What was changed

  • Core assignment decided

by the Reinforcement Learning module

  • Any sequence of core

assignments can be done

Octopusman App-Monitor Application

Latency App Statistics New Core Assignment

User Hardware

Energy Deadline

RL

Delay Core Assignment Energy

slide-43
SLIDE 43

Past Work: Octopus-Man

Reinforcement Learning Module

Reward Function 𝑆 𝑒𝑓𝑚𝑏𝑧, 𝑞𝑝𝑥𝑓𝑠 = 𝑤1, 𝐷𝑏𝑡𝑓 1 𝑤2, 𝐷𝑏𝑡𝑓 2 𝑤3, 𝐷𝑏𝑡𝑓 3 𝑤4, 𝐷𝑏𝑡𝑓 4

Case 1: Delay > deadline, but using 4 big cores 𝑤1 = 1 Case 2: Delay > deadline, but reduced tardiness 𝑤2 = 𝑑𝑣𝑠𝑈𝑏𝑠𝑒𝑗𝑜𝑓𝑡𝑡 𝑞𝑠𝑓𝑤𝑈𝑏𝑠𝑒𝑗𝑜𝑓𝑡𝑡 Case 3: Delay > deadline, no “but” 𝑤3 = −𝑢𝑏𝑠𝑒𝑗𝑜𝑓𝑡𝑡 ∗ 𝑑𝑣𝑠𝑄𝑝𝑥𝑓𝑠 𝑛𝑏𝑦𝑄𝑝𝑥𝑓𝑠 Case 4: Delay < deadline 𝑤4 = 1 − 𝑑𝑣𝑠𝑄𝑝𝑥𝑓𝑠 𝑛𝑏𝑦𝑄𝑝𝑥𝑓𝑠

slide-44
SLIDE 44

Mosse: HetCMP+energy

Re-inforcement Learning Scheduler

  • Learn how to map actions to situations
  • Learning while interacting with the environment
  • Maximizing the long term cumulative reward signal
  • Appropriate for control loop
  • Take more variables/counters into account
  • Overhead, selection of counters
  • Migration Decision: migrate thread if:
  • Long-term reward is good
  • Account for response time, fairness, overhead
  • Hard to choose good reward function!
slide-45
SLIDE 45

Results

Looking at the metrics

10/29/18 CS3530 - Advanced Topics in Distributed and Real-time

blackscholes bodytrack dijkstra sha x264 Average POET 52.9562982 61.07178969 13.76518219 25.55865401 Octopus+RL 5.398457584 8.39231547 1.214574899 3.193612774 3.639792145 2 4 6 8 10 12 14 16 18 20

Violations (%)

Percentage of Violations (Baseline and Linux: 0 violations)

POET Octopus+RL

slide-46
SLIDE 46

Results

Looking at the metrics

10/29/18 CS3530 - Advanced Topics in Distributed and Real-time 0.2 0.4 0.6 0.8 1 1.2

Energy (W)

Total Energy (Normalized to 4 big cores)

POET Linux Octopus+RL

slide-47
SLIDE 47

Mosse: HetCMP+energy

Return to challenges

  • Implementation in real or emulated systems
  • Hybrid memories (DRAM+NVM) help/disturb?
  • Heuristics derived from optimizations?
  • User-level thread migration?
  • Old challenges: (1) Assignment: match threads and core/memory;

(2) How to characterize threads; (3) Dynamic vs static scheduling; (4) Global vs partitioned scheduling; (5) Cache partition vs cache sharing; (6) Inclusive vs exclusive cache; (7) Bus bandwidth partitioning vs sharing; (8) Memory allocation; (9) Memory bank distribution

slide-48
SLIDE 48

Mosse: HetCMP+energy

More challenges

  • Online thread performance prediction when

running on different core types

  • Efficient and specialized heuristics for the

thread assignment problem

  • Implementation of our scheme on Linux
  • multi-core heterogeneity emulated via frequency

scaling

  • management of thread-to-core affinity at user-level