Powering Compute Powering Compute Platforms in High Platforms in - - PowerPoint PPT Presentation

▶
powering compute powering compute platforms in high
SMART_READER_LITE
LIVE PREVIEW

Powering Compute Powering Compute Platforms in High Platforms in - - PowerPoint PPT Presentation

Nov 23, 2022 168 likes •527 views

Powering Compute Powering Compute Platforms in High Platforms in High Efficiency Data Efficiency Data Centers Centers Annabelle Pratt, Pavan Kumar, Annabelle Pratt, Pavan Kumar, Kevin Bross, Tomm Aldridge Kevin Bross, Tomm Aldridge

slide-1
SLIDE 1

Powering Compute Powering Compute Platforms in High Platforms in High Efficiency Data Efficiency Data Centers Centers

Annabelle Pratt, Pavan Kumar, Annabelle Pratt, Pavan Kumar, Kevin Bross, Tomm Aldridge Kevin Bross, Tomm Aldridge

slide-2
SLIDE 2

Slide 2

Agenda Agenda

  • Defining a power

Defining a power-

  • efficient data center

efficient data center

  • Improving power delivery efficiency

Improving power delivery efficiency

  • Power delivery architecture comparison

Power delivery architecture comparison

  • DC data center demonstration

DC data center demonstration

slide-3
SLIDE 3

Slide 3

Scope Scope

Die Board Server Rack Data Center

Defining Defining a power a power-

  • efficient data center

efficient data center

slide-4
SLIDE 4

Slide 4

Cool Air to Data Center Heat Rejected To Outdoors

Room Cooling System

Powering a Data Center Powering a Data Center

BLDG PWR

Server Rack

PDU PDU

PSU

VR VR

12V Loads Loads AC/DC

UPS

AC/DC DC/AC

Fans

DC/DC Defining Defining a power a power-

  • efficient data center

efficient data center

slide-5
SLIDE 5

Slide 5

Power Delivery Challenge Power Delivery Challenge

Power delivery efficiency for data center is only ~ 50%

UPS&PDU 22W Load 100W Room cooling 80W – 200W Total 285W – 405W

Get the whole picture : Consider power- efficiency at data center level

Server fans 13W PSU 48W VR 22W

Defining a power Defining a power-

  • efficient data center

efficient data center

slide-6
SLIDE 6

Slide 6

Agenda Agenda

  • Defining a power

Defining a power-

  • efficient data center

efficient data center

  • Improving power delivery efficiency

Improving power delivery efficiency

  • Power delivery architecture comparison

Power delivery architecture comparison

  • DC data center demonstration

DC data center demonstration

slide-7
SLIDE 7

Slide 7

Conventional AC architecture Conventional AC architecture

  • Prevalent architecture

Prevalent architecture

  • Multiple conversion stages all impact efficiency

Multiple conversion stages all impact efficiency

Rack

380V

480V 3φ AC 208V 1φ AC

Server

PDU PDU

PSU

AC/DC

UPS

AC/DC DC/AC

VR VR

12V Loads Loads

Fans 380V

DC/DC Improving power delivery efficiency Improving power delivery efficiency

88% x 93% x 79% x 75% = 48%

slide-8
SLIDE 8

Slide 8

Note on efficiencies Note on efficiencies

Improving power delivery efficiency Improving power delivery efficiency

50 55 60 65 70 75 80 85 90 95 100 10 20 30 40 50 60 70 80 90 100

Load [%] Efficiency [%]

5 10 15 20 25 30 35 40 45 50

Input power savings [%]

Baseline efficiency Improved efficiency Input power savings

  • Use heavy load efficiencies for comparison

Use heavy load efficiencies for comparison

slide-9
SLIDE 9

Slide 9

Power Train: UPS Power Train: UPS

Rack

380V

480V 3φ AC 208V 1φ AC

Server

PDU PDU

PSU

AC/DC

UPS

AC/DC DC/AC

VR VR

12V Loads Loads

Fans 380V

DC/DC Improving power delivery efficiency Improving power delivery efficiency

Double Conversion UPS

– –Most commonly used in data centers today Most commonly used in data centers today – –Typical Typical 88% 88% efficient efficient – –As high as As high as 94% 94%

88% x 93% x 79% x 75% = 48%

slide-10
SLIDE 10

Slide 10

88% x 93% x 79% x 75% = 48%

Power Train: PDU Power Train: PDU

Rack

380V

480V 3φ AC 208V 1φ AC

Server

PDU PDU

PSU

AC/DC

UPS

AC/DC DC/AC

VR VR

12V Loads Loads

Fans 380V

DC/DC Improving power delivery efficiency Improving power delivery efficiency

Power Distribution Unit

– –Transformer steps down 480V AC to 208V AC Transformer steps down 480V AC to 208V AC – –Provides branch protection Provides branch protection – –Typically Typically 97 97 -

  • 99%

99% efficient efficient – –Include cable losses here Include cable losses here

slide-11
SLIDE 11

Slide 11

Power Train: PSU Power Train: PSU

Server Rack

PDU PDU

PSU

AC/DC

UPS

AC/DC DC/AC DC/DC

VR VR

12V Loads Loads

Fans

PSU

– – Typical PSU efficiency Typical PSU efficiency 75% 75% – – EPA Energy Star EPA Energy Star*

*, 80PLUS, SSI promote

, 80PLUS, SSI promote 80% 80% – –Technology exists for Technology exists for ~90% ~90%

– – Initial cost remains a challenge Initial cost remains a challenge

* * Draft

Draft

Improving power delivery efficiency Improving power delivery efficiency

88% x 93% x 79% x 75% = 48%

slide-12
SLIDE 12

Slide 12

Power Train: VR Power Train: VR

Server Rack

PDU PDU

PSU

AC/DC

UPS

AC/DC DC/AC DC/DC

VR VR

12V Loads Loads

Fans

VR

–Typical efficiency 75% (system aggregate) –Continually increasing, approaching 80%

* * Draft

Draft

Improving power delivery efficiency Improving power delivery efficiency

88% x 93% x 79% x 75% = 48%

slide-13
SLIDE 13

Slide 13

88% x 93% x 79% x 75% = 48%

Conventional AC architecture Conventional AC architecture

  • High efficiency components can reduce input

High efficiency components can reduce input power by 30% power by 30%

Rack

380V

480V 3φ AC 208V 1φ AC

Server

PDU PDU

PSU

AC/DC

UPS

AC/DC DC/AC

VR VR

12V Loads Loads

Fans 380V

DC/DC

Use high efficiency components to reduce energy consumption

94% x 94% x 89% x 86% = 68%

Improving power delivery efficiency Improving power delivery efficiency

slide-14
SLIDE 14

Slide 14

Agenda Agenda

  • Defining a power

Defining a power-

  • efficient data center

efficient data center

  • Improving power delivery efficiency

Improving power delivery efficiency

  • Power delivery architecture comparison

Power delivery architecture comparison

  • DC data center demonstration

DC data center demonstration

slide-15
SLIDE 15

Slide 15

Best Best-

  • in

in-

  • Class AC Architecture

Class AC Architecture

Rack

380V

208V 1φ AC

Server

PDU PDU

PSU

AC/DC

VR VR

12V Loads Loads

Fans 380V

DC/DC

98% x 94% x 89% x 86% = 71%

Power delivery architecture comparison Power delivery architecture comparison

  • Avoid double conversion in UPS

Avoid double conversion in UPS

  • Use line

Use line-

  • interactive or Delta Conversion UPS

interactive or Delta Conversion UPS

– –Highly efficient ~ Highly efficient ~98% 98% – –Not in wide Not in wide-

  • spread use today

spread use today

UPS

DC/AC AC/DC

Line-interactive

slide-16
SLIDE 16

Slide 16

Best Best-

  • in

in-

  • Class AC Architecture

Class AC Architecture

Rack

380V

208V 1φ AC

Server

PDU PDU

PSU

AC/DC

VR VR

12V Loads Loads

Fans 380V

DC/DC

480V 3φ AC UPS

DC/AC AC/DC

98% x 94% x 89% x 86% = 71%

Power delivery architecture comparison Power delivery architecture comparison

  • Avoid double conversion in UPS

Avoid double conversion in UPS

  • Use line

Use line-

  • interactive or Delta Conversion UPS

interactive or Delta Conversion UPS

– –Highly efficient ~ Highly efficient ~98% 98% – –Not in wide Not in wide-

  • spread use today

spread use today

Delta Conversion

slide-17
SLIDE 17

Slide 17

400V 3φ AC

400V AC Architecture 400V AC Architecture

  • If UPS output 400V, do not need transformer in PDU

If UPS output 400V, do not need transformer in PDU

Rack

380V 480V 3φ AC 230V 1φ AC

Server

PDU PDU

PSU

AC/DC

UPS

AC/DC DC/AC

VR VR

12V Loads Loads

Fans 380V

DC/DC

94% x 97% x 89% x 86% = 70%

Power delivery architecture comparison Power delivery architecture comparison

slide-18
SLIDE 18

Slide 18

Rack Level Rack Level -

  • 48V DC Architecture

48V DC Architecture

  • Reduce heat load in individual server

Reduce heat load in individual server

  • Reduce PSU volume

Reduce PSU volume

  • Rack level AC/DC redundancy

Rack level AC/DC redundancy

Rack

380V

208V Server

PDU PDU

PSU

VR VR

12V Loads Loads

Fans

  • 48V

DC/DC AC/DC

480V 3φ AC

AC/DC DC/AC

94% x 94% x 92% x 93% x 86% = 65%

UPS

Power delivery architecture comparison Power delivery architecture comparison

slide-19
SLIDE 19

Slide 19

  • 48V DC Architecture

48V DC Architecture

  • Remove conversion stages

Remove conversion stages

  • Used in telecommunications industry

Used in telecommunications industry

  • ~ 100 x copper UPS to PDU

~ 100 x copper UPS to PDU

  • ~ 20 x copper PDU to rack

~ 20 x copper PDU to rack

– – Addressed with distributed UPS Addressed with distributed UPS

Rack

380V

Server

PDU PDU

PSU

VR VR

12V Loads Loads

Fans

  • 48V DC/DC

480V 3φ AC

AC / DC

  • 48V

DC

93% x 97% x 93% x 86% = 72%

UPS

Power delivery architecture comparison Power delivery architecture comparison

slide-20
SLIDE 20

Slide 20

Rack Level 380V DC Architecture Rack Level 380V DC Architecture

  • Similar to

Similar to -

  • 48V rack level DC architecture

48V rack level DC architecture

  • Use 380V DC link

Use 380V DC link

  • Available in some high end servers

Available in some high end servers

Rack

380V

208V AC Server

PDU PDU

PSU

VR VR

12V Loads Loads

Fans 380V DC

DC/DC AC/DC

480V 3φ AC

AC/DC DC/AC

94% x 94% x 96% x 93% x 86% = 68%

Power delivery architecture comparison Power delivery architecture comparison

slide-21
SLIDE 21

Slide 21

380V DC

380V DC Architecture 380V DC Architecture

  • Eliminate intermediate conversion stages

Eliminate intermediate conversion stages

  • No harmonics or phasing requirements

No harmonics or phasing requirements

  • Fewer components

Fewer components → → higher reliability higher reliability

Rack

380V

Server

PDU PDU

PSU

VR VR

12V Loads Loads

Fans 380V DC/DC

480V 3φ AC

AC/DC

97% x 97% x 93% x 86% = 76%

Power delivery architecture comparison Power delivery architecture comparison

slide-22
SLIDE 22

Slide 22

Comparing Efficiencies Comparing Efficiencies

40 45 50 55 60 65 70 75 80 Baseline AC Rack level 48V DC High efficiency AC Rack level 380V DC 400V AC Best-in-class AC 48V DC 380V DC Power delivery efficiency [%] Power delivery architecture comparison Power delivery architecture comparison

+5.5%

slide-23
SLIDE 23

Slide 23

0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0 Baseline AC Rack level 48V DC High efficiency AC Rack level 380V DC 400V AC Best-in-class AC 48V DC 380V DC Total building input power [MW]

Comparing input power Comparing input power

For 10,000 servers with 300 W load power each and cooling COP = 2.6

Power delivery architecture comparison Power delivery architecture comparison

$370,000 savings per year at 0.1$/kWh

slide-24
SLIDE 24

Slide 24

5000 10000 15000 20000 25000 Baseline AC Rack level 48V DC High efficiency AC Rack level 380V DC 400V AC Best-in-class AC 48V DC 380V DC Number of servers for 10MW power budget

# servers / # servers / 10MW 10MW power budget power budget

Power delivery architecture comparison Power delivery architecture comparison

7.5% increase in number of servers (~1,300) Servers with 300 W load power each and cooling COP = 2.6

slide-25
SLIDE 25

Slide 25

Architecture observations Architecture observations

  • High efficiency architecture and components

High efficiency architecture and components

– – Significant power savings Significant power savings – – Increased server density Increased server density

  • AC and

AC and -

  • 48V DC distribution used today

48V DC distribution used today

  • Facility level 380V DC delivers highest efficiency

Facility level 380V DC delivers highest efficiency – – Not used in industry today Not used in industry today – – Need to demonstrate feasibility Need to demonstrate feasibility

Power delivery architecture choice can reduce energy consumption & TCO, increase density

Power delivery architecture comparison Power delivery architecture comparison

slide-26
SLIDE 26

Slide 26

Agenda Agenda

  • Defining a power

Defining a power-

  • efficient data center

efficient data center

  • Improving power delivery efficiency

Improving power delivery efficiency

  • Power delivery architecture comparison

Power delivery architecture comparison

  • DC data center demonstration

DC data center demonstration

slide-27
SLIDE 27

Slide 27

DC Data Center Demonstration DC Data Center Demonstration

  • Compares AC architecture to 380V DC

Compares AC architecture to 380V DC

  • Demonstrated operational servers with 380V DC

Demonstrated operational servers with 380V DC

AC DC 380V DC Lights Dist. 48V

Summer 2006 : Summer 2006 : Intel collaboration with Lawrence Berkeley Intel collaboration with Lawrence Berkeley National Lab (LBNL) and multiple industry partners National Lab (LBNL) and multiple industry partners

DC data center demonstration DC data center demonstration

slide-28
SLIDE 28

Slide 28

Results from DC Demo Results from DC Demo

DC data center demonstration DC data center demonstration

  • Input power savings of ~ 7%

Input power savings of ~ 7%

  • Confirms theoretical evaluation

Confirms theoretical evaluation

  • *Ratio of DC Load power to AC load power = 0.974

*Ratio of DC Load power to AC load power = 0.974

– – Reduced due to elimination of AC/DC stage in server PSU Reduced due to elimination of AC/DC stage in server PSU

24.10 24.10 22.70* 22.70* 25.91 25.91 23.29 23.29

kW kW kW kW kW kW kW kW

Input Input Load Load Input Input Load Load 380V DC System 380V DC System AC System AC System

slide-29
SLIDE 29

Slide 29

380V DC Data Centers 380V DC Data Centers

  • DC data center

DC data center – –Equipment development Equipment development

– –DC Circuit Breakers DC Circuit Breakers – –Certified power supplies Certified power supplies

– –Building codes may need to be expanded Building codes may need to be expanded – –Infrastructure development, maintenance Infrastructure development, maintenance – –General acceptance by customers to drive General acceptance by customers to drive cost, volume cost, volume

Demonstration of 380V DC data center confirmed its viability and efficiency benefits

DC data center demonstration DC data center demonstration

slide-30
SLIDE 30

Slide 30

Summary Summary

  • Get the whole picture : Consider power

Get the whole picture : Consider power-

  • efficiency

efficiency at data center level at data center level

  • Use high efficiency components to reduce energy

Use high efficiency components to reduce energy consumption consumption

  • Power delivery architecture choice can reduce

Power delivery architecture choice can reduce energy consumption & TCO, increase density energy consumption & TCO, increase density

  • Demonstration of 380V DC data center confirmed

Demonstration of 380V DC data center confirmed its viability and efficiency benefits its viability and efficiency benefits

slide-31
SLIDE 31

Slide 31

References References

–For energy efficiency specifications, see http://www.80plus.org/ and http://www.energystar.gov/ –For information on the DC demo, see http://hightech.lbl.gov/dc-powering/ –Marquet, D., et al., “New flexible powering architecture for integrated service operators”, IEEE Intelec Conference, 2005

– This paper provides many other good references

–Jill Jones, Empires of Light: Edison, Tesla, Westinghouse, and the Race to Electrify the World, Random House, 2003

slide-32
SLIDE 32
slide-33
SLIDE 33

Slide 33

Back Back-

  • up

up

slide-34
SLIDE 34

Slide 34

η2 η1 η4 η3

Pload Pin

A

η1 η4

Pload Pin

B

4 3 2 1

η η η η

load A in

P P =

4 1η

η

load B in

P P =

3 2 4 3 2 1 4 1

η η η η η η η η = × =

load load A in B in

P P P P

What about light load? What about light load?

  • For the same load power

For the same load power, lower efficiencies , lower efficiencies ( (η η) result in higher power savings ) result in higher power savings

  • Case for power savings stronger for

Case for power savings stronger for conversion stages with lower efficiencies conversion stages with lower efficiencies

3 2

1 1 η η − = − = −

A in B in A in B in A in

P P P P P

10 20 30 40 50 60 70 80 90 100

Efficiency [%] Power savings [%]

(η2 = η3)

slide-35
SLIDE 35

Slide 35

Copper comparison AC Copper comparison AC vs vs DC DC

Rac Rac Rac N Vac Vac/√3 Rac Rac N Vdc Vdc/2

ac ac ac

  • I

V P 3 =

Iac Idc

dc dc dc

  • I

V P =

ac ac ac loss

R I P

2

3 =

dc dc ac loss

R I P

2

2 =

For the same output power :

ac dc ac dc dc dc dc

  • ac

ac ac

  • I

V V I I V P I V P 3 3 = ∴ = = =

And for the same cable losses : Output power Cable losses

ac ac dc ac dc ac dc dc dc dc loss ac ac ac loss

R V V R I I R R I P R I P

2 2 2 2 2

3 2 3 2 3 2 3         = = ∴ = = =

Copper

ac ac

R Cu 3 ∝

dc dc

R Cu 2 ∝

ac dc ac dc dc ac ac dc ac dc

Cu V V Cu V V R R Cu Cu

2 2

3 4 3 3 2 3 2 3 2         × =         × × = × =

For Vdc = 380V, compared to: Vac = 480V, need 2.1 x more copper Vac = 208V, need 2.5 x less copper