Multi-armed Bandits for Efficient Lifetime Estimation in MPSoC - - PowerPoint PPT Presentation

▶
multi armed bandits for efficient lifetime estimation in
SMART_READER_LITE
LIVE PREVIEW

Multi-armed Bandits for Efficient Lifetime Estimation in MPSoC - - PowerPoint PPT Presentation

Jan 02, 2024 250 likes •523 views

Multi-armed Bandits for Efficient Lifetime Estimation in MPSoC Design Calvin Ma, Aditya Mahajan, and Brett H. Meyer Department of Electrical and Computer Engineering McGill University Design Differentiation in DSE Design space exploration

slide-1
SLIDE 1

Calvin Ma, Aditya Mahajan, and Brett H. Meyer Department of Electrical and Computer Engineering McGill University

Multi-armed Bandits for Efficient Lifetime Estimation in MPSoC Design

slide-2
SLIDE 2
  • Design space exploration (DSE) is often used for MPSoCs
  • Design spaces are large (on the orders of billions of alternatives)
  • Design evaluation can be complex (requiring multiple metrics)
  • Exhaustive search is usually intractable
  • Goals of DSE:

1. Differentiate poor solutions from good ones 2. Identify the Pareto-optimal set 3. Do so quickly and efficiently

Design Differentiation in DSE

29 March 2017 Brett H. Meyer / McGill University 2

slide-3
SLIDE 3
  • Semiconductor scaling has reduced integrated circuit lifetime
  • Many strategies have been developed to address failure:
  • Redundancy (at different granularities) or slack allocation
  • Thermal management and task migration
  • System-level optimization seeks to maximize mean time to

failure under other constraints (e.g., performance, power, cost)

System Lifetime Optimization for MPSoC

29 March 2017 Brett H. Meyer / McGill University 3

Electromigration Thermal Cycling Stress migration

[Source: JEDEC]

slide-4
SLIDE 4
  • Failure mechanisms are modeled mathematically
  • Historically, with the exponential distribution: easy to work with
  • Recently, with log-normal and Weibull distributions: more accurate
  • There is no straightforward closed-form solution for systems of

log-normal and Weibull distributions

  • Therefore, Monte Carlo Simulation (MCS)!
  • Use failure distributions to generate a random system instance (sample)
  • Determine when that instance fails through simulation
  • Capture statistics, and repeat!

Evaluating System Lifetime

29 March 2017 Brett H. Meyer / McGill University 4

slide-5
SLIDE 5
  • Monte Carlo Simulation is needlessly computationally expensive
  • Samples are distributed evenly to estimate lifetime
  • Poor designs are sampled as much as good designs
  • Multi-armed Bandits (MAB) are smarter
  • Samples are incrementally distributed in order to differentiate systems
  • E.g., to find the best, the best k, etc.
  • Hypothesis: MAB can achieve DSE goals with fewer evaluations

than MCS by differentiating systems, not estimating lifetime

Multi-armed Bandits for Smarter Estimation

29 March 2017 Brett H. Meyer / McGill University 5

slide-6
SLIDE 6
  • Multi-armed Bandits
  • Successive Accept Reject
  • Gap-based Exploration with Variance
  • Lifetime Differentiation Experiments and Results
  • Conclusions and Future Work

Outline

29 March 2017 Brett H. Meyer / McGill University 6

slide-7
SLIDE 7
  • Which slot machine is the best?
  • Monte Carlo Simulation is systematic
  • Try every slot machine equally
  • In the end, compare average payout
  • Multi-armed Bandits algorithms gamble

intelligently

  • Try every slot machine, but stay away from bad ones
  • Do so by managing expected payout from next trial

Multi-armed Bandits Algorithms

29 March 2017 Brett H. Meyer / McGill University 7

[CC BY-SA: Yamaguchi先生]

slide-8
SLIDE 8

Simple MAB Example

29 March 2017 Brett H. Meyer / McGill University 8

  • Assume Bernoulli-distributed

systems with different p

  • UCB1 plays (samples) the arm

(system) that maximizes

  • Explore, but favor better arms
  • Eventually, the best system is

always played

50 100 150 200 250 300 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 MAB(UCB1) on designs with survival probabilities {0.3, 0.5, 0.7, 0.8} Number of samples (n) Sample mean 0.8 0.7 0.5 0.3 MAB (UCB1)

¯ xi + r 2 ln n ni

slide-9
SLIDE 9
  • Conventional MAB formulations assume that
  • The player never stops playing
  • The reward is incrementally obtained after each arm pull
  • A single best arm is identified
  • For DSE, we relax these assumptions
  • Assume a fixed sample budget used to explore designs
  • The reward is associated with the final choice
  • Find the best m arms
  • Two MAB algorithms can be applied in this context

MAB for Lifetime Differentiation

29 March 2017 Brett H. Meyer / McGill University 9

slide-10
SLIDE 10
  • SAR divides the sample budget into n phases to compare n arms
  • Each phase, the allocated budget is divided across active arms
  • After sampling, calculate the distance from boundary between

the m good designs and n – m bad ones

  • Top m designs:
  • Bottom n – m designs:
  • Remove from consideration the design with the biggest gap

Successive Accept Reject (SAR)

29 March 2017 Brett H. Meyer / McGill University 10

∆i = ˆ µi − ˆ µi∗ ∆i = ˆ µi∗ − ˆ µi

slide-11
SLIDE 11
  • Sample all designs initially
  • Samples per design grows

as designs are removed

  • Many samples used to

differentiate mth and m+1th designs

Successive Accept Reject Example

29 March 2017 Brett H. Meyer / McGill University 11

1 2 3 4 5 6 7 8 9 10 50 100 150 200 250

Phase

Samples per design in current phase Samples per phase for 10 designs, 1000 samples

1 2 3 4 5 6 7 8 9 10 2 4 6 8 10

Number of designs remaining

slide-12
SLIDE 12

Successive Accept Reject Example

29 March 2017 Brett H. Meyer / McGill University 12

S i A t R j t (T 5 t f 10) Successive Accept Rejects (Top 5 out of 10) Successive Accept Rejects (Top 5 out of 10) p j ( p )

45 45 40 40 35 35 es 30 le 30 pl m am 25 Sa 25 f S

  • f

e o 20 ge 20 ag nt en 15 ce 15 erc 15 Pe P 10 10 5 5 0 1 0 2 0 3 0 4 0 5 0 6 0 7 0 8 0 9 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 Design Points (ranked from lowest to highest utility) Design Points (ranked from lowest to highest utility)

slide-13
SLIDE 13
  • GapE-V never removes a design from consideration
  • Instead, pick the design that minimizes the empirical gap with

the boundary, plus an exploration factor

  • Effort is focused near the boundary
  • High variance, or a limited number of samples, increase

likelihood a design is sampled

Gap-based Exploration with Variance (GapE-V)

29 March 2017 Brett H. Meyer / McGill University 13

It = −∆i + r 2aˆ σi Ti + 7ab 3(Ti − 1)

slide-14
SLIDE 14

GapE-V Example

29 March 2017 Brett H. Meyer / McGill University 14

G E (T 5 t f 10) GapE (Top 5 out of 10) GapE (Top 5 out of 10) p ( p )

25 25 20 20 es le pl m 15 am 15 Sa f S

  • f

e o ge ag 10 nt 10 en ce erc Pe P 5 0 1 0 2 0 3 0 4 0 5 0 6 0 7 0 8 0 9 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 Design Points (ranked from lowest to highest utility) Design Points (ranked from lowest to highest utility)

slide-15
SLIDE 15
  • NoC-based MPSoC lifetime optimization with slack allocation
  • Slack is spare compute and storage capacity
  • Add slack to components s.t. remapping mitigates one or more failures
  • Two applications, two architectures each
  • Component library of processors, SRAMs

Experimental Setup

29 March 2017 Brett H. Meyer / McGill University 15

ARM9 BSP- VLD

64KB VBV/ VCV1

ARM11 S/M/T M3 Rcns 64KB VCV2 M3 Pad2 96KB VCV3 M3 Pad1 256KB VMV M3 Dblk ARM11 Drng 1 2 3 4 ARM9 BSP- VLD

64KB VBV/ VCV1

ARM11 S/M/T 64KB VCV2 M3 Rcns 96KB VCV3 256KB VMV M3 Dblk ARM11 Drng 1 2 3 5 4 M3 Pad1 M3 Pad2 M3 In ARM11 NR 1MB Mem1 M3 HS 1MB Mem2 ARM11 Jug1 M3 VS M3 SE 1MB Mem3 M3 Blend M3 HVS ARM9 Jug2 1 2 3 Processor Memory Switch M3 In ARM11 NR 1MB Mem1 M3 HS 1MB Mem2 ARM11 Jug1 M3 VS M3 SE 1MB Mem3 M3 Blend M3 HVS ARM9 Jug2 2 4 3 1

MWD: 11K designs MPEG-4: 140K designs

slide-16
SLIDE 16
  • We compare SAR, GapE-V, and MCS
  • Optimal set determined with MCS using 1M samples per design
  • How likely is it that an approach picks the wrong set?
  • Compare the aggregate MTTF using policy J and the optimal set
  • is the probability of identification error, the chance a subset of

m differs significantly from the optimal set

Evaluating the Chosen m

29 March 2017 Brett H. Meyer / McGill University 16

Pr  m X

i=1

µ∗

i − EµJ(i) > ✏

  • ≤

δ

slide-17
SLIDE 17

29 March 2017 Brett H. Meyer / McGill University 17

Picking the Top 50, MWD

100 200 300 400 500 10

−2

10

−1

10 Samples Confidence parameter δ MWD3S, m=50 MCS SAR GAPE 100 200 300 400 500 10

−2

10

−1

10 Samples Confidence parameter δ MWD4S, m=50 MCS SAR GAPE

slide-18
SLIDE 18

29 March 2017 Brett H. Meyer / McGill University 18

Picking the Top 50, MPEG-4

100 200 300 400 500 10

−2

10

−1

10 Samples Confidence parameter δ MPEG4S, m=50 MCS SAR GAPE 100 200 300 400 500 10

−2

10

−1

10 Samples Confidence parameter δ MPEG5S, m=50 MCS SAR GAPE

slide-19
SLIDE 19

29 March 2017 Brett H. Meyer / McGill University 19

Comparison with MCS after 500 samples

m=20 m=30 Benchmark δ SAR GapE-V δ SAR GapE-V MWD3S 0.002 1.92x 1.72x 0.003 1.72x 1.71x MWD4S 0.071 3.33x 2.13x 0.112 2.96x 2.07x MPEG4S 0.120 3.57x 2.70x 0.101 3.52x 2.48x MPEG5S 0.052 5.26x 3.57x 0.083 4.07x 3.05x m=40 m=50 Benchmark δ SAR GapE-V δ SAR GapE-V MWD3S 0.009 1.79x 1.67x 0.021 1.49x 1.45x MWD4S 0.180 2.54x 2.01x 0.148 2.44x 1.92x MPEG4S 0.202 3.60x 2.43x 0.115 3.33x 2.27x MPEG5S 0.292 3.70x 3.07x 0.162 3.57x 2.86x

slide-20
SLIDE 20
  • No; MAB

wins!

Does Error Tolerance Matter?

29 March 2017 20

0.5 1 1.5 2 0.2 0.4 0.6 0.8 1 ε (years) Confidence parameter δ MPEG4S, 100 designs identify the top m=50, samples=50 MCS SAR GAPE 0.5 1 1.5 2 0.2 0.4 0.6 0.8 1 ε (years) Confidence parameter δ MPEG5S, 100 designs identify the top m=50, samples=50 MCS SAR GAPE 0.5 1 1.5 2 0.2 0.4 0.6 0.8 1 ε (years) Confidence parameter δ MWD3S, 100 designs identify the top m=50, samples=50 MCS SAR GAPE 0.5 1 1.5 2 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 ε (years) Confidence parameter δ MWD4S, 100 designs identify the top m=50, samples=50 MCS SAR GAPE

slide-21
SLIDE 21
  • Complexity is a function of sampling and selection
  • Sampling time ND x Tsample is fixed across approaches
  • MCS performs no selection: all designs are sampled equally
  • SAR (GapE-V) additional sorts the design list D (ND) times

What About Complexity?

29 March 2017 Brett H. Meyer / McGill University 21

Algorithm Run Time (Upper Bound) MCS ND × Tsample SAR ND × Tsample + D × Tsort(D) GapE-V ND × Tsample + ND × Tsort(D)

slide-22
SLIDE 22
  • 500 samples per design, Intel E5-2670, 96GB RAM averaged
  • ver 10 trials, or <1 ms per trial
  • When sampling complexity is low, MAB loses as the population

grows (sorting dominates)

MAB When Sampling is Expensive

29 March 2017 Brett H. Meyer / McGill University 22

Algorithm Number of Designs 50 100 200 400 MCS 4.41s 8.52s 16.86s 34.54s SAR 4.48s 10.41s 27.22s 95.26s GapE 5.33s 11.46s 34.31s 108.64s

slide-23
SLIDE 23
  • The objective of DSE is to differentiate designs
  • MCS is poorly suited for this: why evaluate bad designs?
  • MAB spends samples to efficiently separate metric estimates
  • Estimating system lifetime, MAB uses 33-81% fewer samples
  • Next step: apply in population-based design space exploration

Conclusions and Future Work

29 March 2017 Brett H. Meyer / McGill University 23

slide-24
SLIDE 24

Questions?

Thank you!

29 March 2017 Brett H. Meyer / McGill University 24

slide-25
SLIDE 25

29 March 2017 Brett H. Meyer / McGill University 25

Lifetime Distributions, MWD

10 12 14 16 500 1000 1500 MWD3S lifetime (µ=11.38 σ=0.4705) Lifetime (years) Frequency (count) 10 12 14 16 500 1000 1500 MWD4S lifetime (µ=11.88 σ=0.5982) Lifetime (years) Frequency (count)

slide-26
SLIDE 26

29 March 2017 Brett H. Meyer / McGill University 26

Lifetime Distributions, MPEG-4

10 12 14 16 2000 4000 6000 8000 10000 12000 MPEG4S lifetime (µ=12.76 σ=0.9293) Lifetime (years) Frequency (count) 10 12 14 16 2000 4000 6000 8000 10000 12000 MPEG5S lifetime (µ=13.34 σ=1.308) Lifetime (years) Frequency (count)