[PPT] - Designing Energy-Efficient Microprocessors in the Era of PowerPoint Presentation

SLIDE 1

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

Radu Teodorescu Department of Computer Science and Engineering The Ohio State University http://arch.cse.ohio-state.edu

computer architecture research lab

T

SLIDE 2

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

The case for energy efficiency

2

Mobility!
Battery life!
Energy cost !
Environment!

Energy efficiency is now crucial to all computing markets, in particular the growth areas: mobile and cloud computing.

SLIDE 3

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

Near-threshold voltage (NTV)

3

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

NT Vdd Nominal Vdd Vth Power reduction 100X Frequency cost 10X Energy reduction 10X

Voltage

Near-threshold computing, a promising energy-efficient solution.

SLIDE 4

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

Intel NTV prototype

4

SLIDE 5

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

NTV faces significant challenges

5

Reliability

1E-10 1E-08 1E-06 1E-04 1E-02 1E+00 900 825 750 675 600 525 450 375 300 Supply Voltage - millivolts Probability of SRAM Bit Failure Intel Vcc-min NTV 5% error rate

Process Variation

0.5 1 1.5 2 Frequency Distribution

nominal NTV%

Voltage Variation

Voltage Emergency

SLIDE 6

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Variation effects at NTV

6

Voltage

delay = f(Vdd - Vth) NTV Nominal Vth Vdd

Delay Nom. Delay NTV

SLIDE 7

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

Outline of our solutions

7

Reliability Process Variation Voltage Variation Parichute

[micro2010]

Data Parity Parity

Booster

[hpca2012]

VRSync

[isca2012]

Power (Watts) Cores in Barrier Time (milliseconds) Cores in Barrier Power (Watts) 10 20 30 40 50 60 70 36 38 40 42 44 46 48 50 8 16 24 32

Voltage Speculation in Itanium II [isca2013]

SLIDE 8

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

Outline of our solutions

8

Reliability Process Variation Voltage Variation Parichute

[micro2010]

Data Parity Parity

Booster

[hpca2012]

VRSync

[isca2012]

Power (Watts) Cores in Barrier Time (milliseconds) Cores in Barrier Power (Watts) 10 20 30 40 50 60 70 36 38 40 42 44 46 48 50 8 16 24 32

Voltage Speculation in Itanium II [isca2013]

SLIDE 9

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

9

1E-10 1E-08 1E-06 1E-04 1E-02 1E+00 900 825 750 675 600 525 450 375 300 Supply Voltage - millivolts Probability of Bit Cell Failure

Intel Vcc-min

350mV 5%

Parichute

SRAM failure rates

SLIDE 10

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

Turbo product codes

10

Data Parity Parity

SLIDE 11

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

Parichute ECC

11

Permutation 0 Permutation 1 Permutation 2 Permutation 3

56 100 351 511 437 351 100 437 511 56 87 204 5 2 511 56 351 499 201 1 511 73

← Permutation 0 → ← Permutation 1 →

SLIDE 12

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

Parichute cache architecture

12

Encoder

Data Block (cache line) Parity encoders Parity encoders Parity encoders PW PW PW ... PW PW PW ... PW PW PW ... ... Parity Group 0 Parity Group 1 Parity Group N Permutation Network Permutation 0 Permutation 1 Permutation N

Data In Cache Data+Parity Decoder Data Out

Line 0 Line 1 Line 2 Line 3 Line 4 Line 5 Line 6 Line 7

Data Parity +

Data bits

Redundant bits

SLIDE 13

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

Error correction example

13

a b d c e a e d c b

1-bit error ✓ 2-bit error ✗

Corrector 0 Corrector 1 Corrector 2 Corrector 3

SLIDE 14

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

Experimental setup

14

SRAM error model
SPICE model of cell
8-way 2MB caches
VARIUS
Processor model
SESC [Intel Core]
CACTI & WATTCH
Benchmarks
SPECint, SPECfp 2000
Prototype
Verilog
Synopsys Design Compiler
Nangate 45nm standard cell
Formality

Vdd Freq + Latency Nominal 0.9V 3GHz NTHigh 0.375V 460Mhz 4 NTMid 0.350V 355Mhz 4 NTLow 0.337V 300Mhz 6

Overhead used in simulations

SLIDE 15

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

Error correction strength

15

0% 25% 50% 75% 100% 5 10 15 20 25 30 35 0% 25% 50% 75% 100% 5 10 15 20 25 30 35 0% 25% 50% 75% 100% 5 10 15 20 25 30 35 SECDED

Errors in 512 data bits Percent lines correctable

OLSC 256 Parichute 252

Z. Chishti, A. R. Alameldeen, C. Wilkerson, W. Wu, and S. L. Lu,

“Improving cache lifetime reliability at ultra-low voltages,” in International Symposium on Microarchitecture, December 2009.

SLIDE 16

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

Cache capacity

16

0% 25% 50% 75% 100% 600 550 500 450 400 350 300 250 No Protection SECDED OLSC 256 Parichute 252

Remaining Cache Capacity

Parichute: 50% OLSC: 24% Parichute: 25% OLSC: 7%

SLIDE 17

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

Parichute hardware overhead

Encoder and decoder hardware
27628 standard cells
Area: 0.056mm2
Power: 11mW
Critical path: 0.95ns (1GHz)
Cache area
+ 4%

17

SLIDE 18

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

Outline of our solutions

18

Reliability Process Variation Voltage Variation Parichute

[micro2010]

Data Parity Parity

Booster

[hpca2012]

VRSync

[isca2012]

Power (Watts) Cores in Barrier Time (milliseconds) Cores in Barrier Power (Watts) 10 20 30 40 50 60 70 36 38 40 42 44 46 48 50 8 16 24 32

Voltage Speculation in Itanium II [isca2013]

SLIDE 19

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

Variation effects on frequency

19

0.5 1 1.5 2 Frequency Distribution

Vth σ/μ = 12% Vdd = 900mV Vth = 210±50mV F σ/μ = 4.4% F = 3GHz ± 260MHz

SLIDE 20

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

0.5 1 1.5 2 Frequency Distribution

Variation effects on frequency

20

Vth σ/μ = 12% Vdd = 900mV Vth = 210±50mV F σ/μ = 4.4% F = 3GHz ± 260MHz Vdd = 400mV F σ/μ = 30.6% F = 400 ± 245MHz Vth σ/μ = 12%

SLIDE 21

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

Impact of frequency variation

21

Frequency Execution progress Wasted Perf. Execution progress Frequency NTV Variation No variation Bottleneck

SLIDE 22

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

Dual-Vdd chip multiprocessor

22

Core Core 1 Core 2 Core 3 Core 4 Core 5 Core 6 Core 7 Core 8 Core 9 Core 10 Core 11 Core 12 Core 13 Core 14 Core 15 Core Core 1 Core 2 Core 3 Core 4 Core 5 Core 6 Core 7 Core 8 Core 9 Core 10 Core 11 Core 12 Core 13 Core 14 Core 15

Vdd High Vdd Low

Each core assigned two power rails:
NT

Vdd High & Low, with Fhigh and Flow

Cores can switch rapidly between the

two rails and Fhigh and Flow

SLIDE 23

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

Frequency interpolation

23

C0 C1 C2 C3 Core 775 650 575 425 Low Vdd MHz 2025 1775 1625 1375 High Vdd MHz 74% 26%

Target: 1100 MHz

60% 40% 50% 29% 50% 71% 74% 26% 60% 40% 50% 29% 50% 71%

SLIDE 24

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

Frequency interpolation - in action

24

Fastest Slowest

Slow Fast

64-core CMP

SLIDE 25

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

Two Booster algorithms: VAR & SYNC

Booster

VAR:

Eliminates heterogeneity: all cores appear to run at target F
Booster SYNC
Dynamically redistribute “boost” from blocked to active threads
Use hints from synchronization primitives
Hardware support

25

Blocked Normal Critical

SLIDE 26

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

Experimental setup

26

Processor
Modeled by SESC
Dual-issue OOO
32nm, 32 cores
3GHz at 900mV
300-2500MHz at NT
NT at 400-635mV
Benchmarks
SPLASH-2
PARSEC
Circuit modeling
SPICE
Markovic̀, et al
Variation modeling
VARIUS

SLIDE 27

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

Booster runtime Static workloads

27

0.5 0.6 0.7 0.8 0.9 1 1.1 barnes

cean

water-nsqd cholesky fft lu radix blackscholes fluidanimate swaptions dedup streamclster g.mean Normalized Execution Time Hetero Scheduling Booster VAR Booster SYNC Heterogeneous

14% 22%

SLIDE 28

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

Booster runtime Dynamic workloads

28

0.5 0.6 0.7 0.8 0.9 1 1.1 radiosity raytrace volrend bodytrack g.mean Normalized Execution Time Hetero Scheduling Booster VAR Booster SYNC Heterogeneous

9% 18%

SLIDE 29

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

Outline of our solutions

29

Reliability Process Variation Voltage Variation Parichute

[micro2010]

Data Parity Parity

Booster

[hpca2012]

VRSync

[isca2012]

Power (Watts) Cores in Barrier Time (milliseconds) Cores in Barrier Power (Watts) 10 20 30 40 50 60 70 36 38 40 42 44 46 48 50 8 16 24 32

Voltage Speculation in Itanium II [isca2013]

SLIDE 30

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

Voltage Variability

30

Voltage emergency!

0.0010 0.0015 0.3 0.4 0.5 0.6 0.7

V(out) (V) time (s) V(out) I(load)

5 10 15 20 25 30 35 40 45

10%

I(load) (A)

+10%

0.0010 0.0015 0.3 0.4 0.5 0.6 0.7

V(out) (V) time (s) V(out) I(load)

5 10 15 20 25 30 35 40 45

I(load) (A)

+10%

10%

Normal operation

Guardband

SLIDE 31

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

Synchronization-Induced Voltage Emergencies

31

Voltage Emergency

SLIDE 32

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

Power (Watts) Cores in Barrier Time (milliseconds) Cores in Barrier Power (Watts) 10 20 30 40 50 60 70 35 40 45 50 55 4 8 12 16

16 cores

32

Power (Watts) Cores in Barrier Time (milliseconds) Cores in Barrier Power (Watts) 10 20 30 40 50 60 70 30 40 50 60 70 80 1 2 3 4

4 cores

Power (Watts) Cores in Barrier Time (milliseconds) Cores in Barrier Power (Watts) 10 20 30 40 50 60 70 30 35 40 45 50 55 60 65 70 2 4 6 8

8 cores

Thread Synchronization Effects

n CMP Power Profile

Power (Watts) Cores in Barrier Time (milliseconds) Cores in Barrier Power (Watts) 10 20 30 40 50 60 70 36 38 40 42 44 46 48 50 8 16 24 32

32 cores

6X!

SLIDE 33

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

VRSync

VRSync: voltage-aware synchronization library
Reduces dI/dt caused by synchronization events
Helps reduce voltage guardband
Lower voltage guardband = Energy savings
On average

VRSync saves 33% energy

33

SLIDE 34

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

Our solution: VRSync Barriers

34

First enter Execution Blocked on barrier 1 Delay All in barrier 1 T0 T7 Linear schedule t7 T1 T2 T3 T4 T5 T6 Time Threads

Linear schedule

First enter Execution Blocked on barrier 1 All in barrier Delay All out T0 T7 Threads T1 T2 T3 T4 T5 T6 Time

Bulk schedule

50 100 150 200 250 300 400 450 500 550 600 650 700

V(out)

No. of active cores
10%

V(out) (mV) time (µs) 4 8 12 16 20 24 28 32 36

No. of active cores

50 100 150 200 250 300 400 450 500 550 600 650 700

V(out)

No. of active cores
10%

time (µs) V(out) (mV) 4 8 12 16 20 24 28 32 36

No. of active cores

SLIDE 35

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

Experimental setup

35

Processor!
Modeled by SESC!
32nm, 32 cores!
1GHz at 600mV!
Benchmarks!
SPLASH2!
PARSEC!

! ! !

Circuit modeling!
SPICE!
Markovic̀, et al.*!
Voltage Regulator!
Linear Technology’s

LTC3729L-6 polyphase!

LTspice!
Barrier!
Software Combining

Tree Barrier!

SLIDE 36

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

Eliminating Barrier-Induced Emergencies

36

Power (Watts) Cores in Barrier Time (milliseconds) Emergency Emergency Cores in Barrier Power (Watts) 10 20 30 40 50 60 70 80 90 46.98 47 47.02 47.04 47.06 8 16 24 32 Power (Watts) Cores in Barrier Time (milliseconds) Emergency Cores in Barrier Power (Watts) 10 20 30 40 50 60 70 80 90 49.44 49.47 49.5 49.53 49.56 49.59 8 16 24 32 Power (Watts) Cores in Barrier Time (milliseconds) Emergency Cores in Barrier Power (Watts) 10 20 30 40 50 60 70 80 90 49.38 49.41 49.44 49.47 49.5 49.53 8 16 24 32

Baseline Linear Bulk

fluidanimate - parsec!

SLIDE 37

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

Eliminating Barrier Emergencies

37

VRSync Bulk! Baseline!

lu – splash2!

SLIDE 38

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

Eliminating Phase Alignment Emergencies

38

Baseline! VRSync Linear!

fft – splash2!

SLIDE 39

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

Execution time overhead

39

0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 r a d i

s

i t y b a r n e s

c

e a n r a y t r a c e w a t e r

n

s q u a r e d c h

l

e s k y f f t l u r a d i x b l a c k s c h

l

e s b

d

y t r a c k f l u i d a n i m a t e s w a p t i

n

s d e d u p s t r e a m c l u s t e r g . m e a n Normalized Execution Time

2.1

Linear Bulk

11% 6%

SLIDE 40

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

VRSync: Energy Savings

40

Technique! Guardband! Runtime! Power! Energy! Baseline with High Guardband! 210mV! 1.0! 1.563! 1.563! VRSync Linear! 60mV! 1.112! 0.98! 1.086! VRSync Bulk! 60mV! 1.063! 0.99! 1.049! VRSync Bulk Fast! 160mV! 1.045! 1.361! 1.422!

31% 33%

VRSync Bulk is 33% more energy efficient than baseline with high guardband

SLIDE 41

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

Outline of our solutions

41

Reliability Process Variation Voltage Variation Parichute

[micro2010]

Data Parity Parity

Booster

[hpca2012]

VRSync

[isca2012]

Power (Watts) Cores in Barrier Time (milliseconds) Cores in Barrier Power (Watts) 10 20 30 40 50 60 70 36 38 40 42 44 46 48 50 8 16 24 32

Voltage Speculation in Itanium II [isca2013]

SLIDE 42

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

High voltage margins in modern CPUs - energy inefficient
Voltage speculation techniques exist (Razor, etc.)
Require dedicated hardware
Reliability challenges lead to heavy use of on-chip ECC
Caches, register file, TLBs, etc.
Idea: leverage on-chip ECC to dynamically lower voltage

margins (voltage speculation)

42

Voltage speculation in Itanium II

SLIDE 43

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

Voltage margin exploration

HP BL860-i4 Server (2X 9560 Itanium II 8-core CPUs)
Gradually lowered supply voltage (Vdd) for each core individually
1.1V Nominal -> 0.9, constant frequency (2.53GHz)
Logged correctable errors, power consumption
Recorded crashes, data corruption
Experiments performed with HP stress test application, SPECjbb,

SPECfp, and SPECint benchmarks

43

SLIDE 44

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

Correctable errors vs. Vdd

44

Observation: Correctable errors always triggered before uncorrectable ones, while running a stress test workload.

2 4 6 8 10 12 14 16 18 0.96 0.98 1 1.02 1.04 1.06 1.08 1.1 Error Rate (errors/minute) Supply Voltage Unsafe Vdd Itanium Core

Failure Vdd Correctable error range

SLIDE 45

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

High variation in safe Vdd

0.8$ 0.85$ 0.9$ 0.95$ 1$ 1.05$ 1.1$

Core$0$ Core$1$ Core$2$ Core$3$ Core$4$ Core$5$ Core$6$ Core$7$

Nominal$Vdd$ Safe/Min$Vdd$ Fail$Vdd$

45

Core-to-core variation in safe/min Vdd: 0.96-1V

SLIDE 46

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

ECC-based voltage speculation

Our solution: dynamically lower supply voltage
Use correctable errors as “early warning system”
Two-step approach:
Margin

Voltage - determined post-manufacturing by running stress test workload

Runtime reevaluation based on correctable error reports
Monitoring & control implemented in firmware, transparent to OS
Prototyped in HP Server with Itanium II CPUs

46

SLIDE 47

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

Margin discovery and runtime

47 Safety padding (10 mV) First Error Voltage Margin Voltage Supply Voltage Time Correctable Errors Core Vdd Core errors Discovery phase Runtime

SLIDE 48

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

Aggressive speculation

Some applications/cores more amenable to voltage speculation
Constant stream of correctable errors

48 Safety padding (10 mV) First Error Voltage Margin Voltage Supply Voltage Time Core Vdd Correctable errors Max error threshold

burst testing

Min error threshold

SLIDE 49

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

Voltage speculation in action

0.965 0.97 0.975 0.98 0.985 0.99 0.995 1 5 10 15 20 10 20 30 40 50 Supply Voltage (V) Error Rate (per minute) Time (minutes) Margin Voltage Error rate Core Voltage

49

SPECjbb

SLIDE 50

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

Power savings

0.5 0.6 0.7 0.8 0.9 1 Specjbb2005 SPECint SPECfp Relative Power Cores-only CPU Total

Cores-only: 22% SPECjbb, 23% SPECint and 18% SPECfp
Total (with uncore): 14% SPECjbb, 15% SPECint and 11% SPECfp

50

SLIDE 51

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

High variation in correctable errors

50 100 150 200 250 300 core0 core1 core2 core3 core4 core5 core6 core7 Correctable Errors Correctable Cache Errors Correctable RF Errors

(a) Processor A

50 100 150 200 250 core0 core1 core2 core3 core4 core5 core6 core7 Correctable Errors

(b) Processor B

51

SLIDE 52

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

Outline of our solutions

52

Reliability Process Variation Voltage Variation Parichute

[micro2010]

Data Parity Parity

Booster

[hpca2012]

VRSync

[isca2012]

Power (Watts) Cores in Barrier Time (milliseconds) Cores in Barrier Power (Watts) 10 20 30 40 50 60 70 36 38 40 42 44 46 48 50 8 16 24 32

Voltage Speculation in Itanium II [isca2013]

SLIDE 53

Designing Energy-Efficient Microprocessors in the Era of Unpredictable Transistors

T

computer

architecture research lab

Acknowledgments

The Research Team:
Timothy N. Miller, PhD 2012,

now Assist. Prof. @ SUNY Binghamton

Renji Thomas
Xiang Pan
Naser Sedaghati
Anys Bacha

53

The Sponsors: