Green computing in IEEE 802.3az enabled clusters Dimitar Pavlov - - PowerPoint PPT Presentation

Green computing in IEEE 802.3az enabled clusters

Dimitar Pavlov Joris Soeurt

SNE

July 5, 2012

Dimitar Pavlov, Joris Soeurt (SNE) Green computing July 5, 2012 1 / 26

Green Computing

”Data centres emit over 150 metric tons of CO2 per year, and the volume is increasing.”1 ”Carbon dioxide emissions account for 80% of the contribution to global warming...”2 Different strategies towards environmentally sustainable IT

◮ Computational efficiency (e.g. optimizing of algorithms) ◮ Consolidation (e.g. virtualization) ◮ Reducing / recycling of e-waste ◮ Resource allocation (e.g. route data to most green datacenter) ◮ Green networking 1Baroudi et al. (2009) 2Lashof et al. (1990) Dimitar Pavlov, Joris Soeurt (SNE) Green computing July 5, 2012 2 / 26

Approaches to Green Networking

”Recent studies have shown that network devices comprise more than 15%

• f the total energy consumption of a datacenter.”3

Adjust transmission power based on cable length

◮ Cables of 5m do not need same transmission power as 100m cables

Power down circuitry when the line protocol is down

◮ If the line protocol is down, why keep the hardware active?

Use signalling to put circuitry in lower power mode when idle

◮ Done by IEEE 802.3az ◮ Signalling protocol to put circuitry (of both sides of the connection) in

sleep mode when the transmit buffer is empty

◮ State transitions operates on the microsecond level, and is therefore

invisible to higher layers

◮ Both sides should announce 802.3az support during autonegotiation 3Barroso et al. 2007 Dimitar Pavlov, Joris Soeurt (SNE) Green computing July 5, 2012 3 / 26

Research motivation

Research motivation

How can an application optimize its energy effiency using the 802.3az protocol in cluster environments? How does the protocol achieve its energy savings? How to model the energy characteristics of 802.3az compliant devices? How to apply the energy model in cluster computing?

Dimitar Pavlov, Joris Soeurt (SNE) Green computing July 5, 2012 4 / 26

Cluster Overview

W W W W W W W W W W W W

Figure: Simplified model of cluster environment

Dimitar Pavlov, Joris Soeurt (SNE) Green computing July 5, 2012 5 / 26

Theoretical Study on 802.3az

What is the background of this protocol? How does the protocol achieve its energy savings? What earlier research has been done on this protocol?

Dimitar Pavlov, Joris Soeurt (SNE) Green computing July 5, 2012 6 / 26

Timeline of 802.3az

12-2006 Call for interest 8-2007 Project authorization request 8-2008 Task Force Review 9-2008 Draft 1.0 7-2009 Draft 2.0 3-2010 Draft 3.0 11-2010 IEEE standard

Researched mostly theoretically before, hardware implementations of final standard only now arriving on market 802.3az has not been researched in the context of cluster environments

Dimitar Pavlov, Joris Soeurt (SNE) Green computing July 5, 2012 7 / 26

802.3az link states

WAKE REFRESH ACTIVE SLEEP ACTIVE

Td Ts Tq Tr Tw

Refresh interval power saving occurs

182μs 20 000μs 198μs 16.5μs

Ts = time to sleep

◮ Send LPI

Tq = time quiescent Tr = time to refresh

◮ Detect link failure

Tw = time to wake

◮ Equals time needed for

sending max size frame

Td = decision time

Dimitar Pavlov, Joris Soeurt (SNE) Green computing July 5, 2012 8 / 26

802.3az projected energy savings

10 20 30 40 50 60 70 80 90 100 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Energy consumption (% of peak) Load No EEE EEE (simulation) Ideal

Figure: Simulated energy consumption [Reviriego 2009]

Dimitar Pavlov, Joris Soeurt (SNE) Green computing July 5, 2012 9 / 26

Experimental Phase

Observe the energy behaviour of 802.3az in different situations

◮ (devices, linkspeeds, throughput and protocols)

Construct energy profiles for different devices

Dimitar Pavlov, Joris Soeurt (SNE) Green computing July 5, 2012 10 / 26

Equipment Overview

(a) Cisco SG300-28 (b) Huawei S1728GWR-4P (c) Extreme x440-24p (d) Racktivity PDU (e) Schleifenbauer PDU (f) Realtek 8111E (g) Intel I350-T2 NICs (h) Servers

Dimitar Pavlov, Joris Soeurt (SNE) Green computing July 5, 2012 11 / 26

Test setup

V V V

VLAN

2

VLAN

1

VLAN

3

VLAN

5

VLAN

7

VLAN

9

VLAN

11

VLAN

4

VLAN

6

VLAN

8

VLAN

10 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 Controller Source PDU

1: 18W 2: 0W 3: 0W 4: 0W 5: 0W 6:0W 7: 0W 8: 0W

ethernet 25 27 26 28 power Switch Destination

Dimitar Pavlov, Joris Soeurt (SNE) Green computing July 5, 2012 12 / 26

Test setup

Figure: What it actually looked like...

Dimitar Pavlov, Joris Soeurt (SNE) Green computing July 5, 2012 13 / 26

Experiments – Maximum energy savings

Goal: determine maximum energy savings with 802.3az

◮ Fully utilize the switch to measure maximum consumption ◮ Measure the minimal switch consumption when no traffic is present ◮ Compare both measurements to determine maximum energy savings Dimitar Pavlov, Joris Soeurt (SNE) Green computing July 5, 2012 14 / 26

Experiments – Maximum energy savings (cont’d)

8 9 10 11 12 13 14 15 16 17 18 19 20 21

10 20 30 40 50 60 70 80 90 100 110 120 130 140 150 160

Energy consumption (W) Time (Sec) No EEE EEE

Figure: Cisco SG300-28 using Iperf – TCP/UDP traffic at 1Gbps

Dimitar Pavlov, Joris Soeurt (SNE) Green computing July 5, 2012 15 / 26

Experiments – Throughput vs energy consumption

Goal: determine the relationship between throughput and energy usage

◮ Generation of traffic is done with Iperf ◮ The transmission rate is set with tc per test run ◮ Energy usage of the switch is measured per test run ◮ Traffic is generated for 5 minutes then the measurements are averaged

per run

Dimitar Pavlov, Joris Soeurt (SNE) Green computing July 5, 2012 16 / 26

Experiments – Throughput vs energy consumption (cont’d)

Results from TCP tests only. UDP shows unexpected results.

9 10 11 12 13 14 15 16 17 18 19 20 21

50 100 150 200 250 300 350 400 450 500 550 600 650 700 750 800 850 900 950 1000

Energy consumption (W) Throughput (MBit) No EEE EEE

Figure: Cisco SG300-28, Es =

• 10.9 + Thrghpt ∗ 18.9−10.9

400

, if Thrghpt < 400 18.9, if Es ≥ 400

Dimitar Pavlov, Joris Soeurt (SNE) Green computing July 5, 2012 17 / 26

Experiments Summary

Experiment shows that 802.3az has the potential to save power Vendor claims of 30% savings are generally true Odd power usage distribution – does not conform to previous research4 Constructed power profiles, which were used as input to final phase

4Riveriego, 2009 Dimitar Pavlov, Joris Soeurt (SNE) Green computing July 5, 2012 18 / 26

Applying 802.3az in applications

Distributed computing & Cluster computing Model for optimizing energy usage

◮ Define a way to determine energy usage with 802.3az ◮ Estimate switches energy consumption based on number of active ports ◮ Estimate time distribution for particular tasks with a focus on parallel

computing

◮ Determine best transmission rate for a fixed quantity of data

Combine output of all phases and create a prototype power calculator

Dimitar Pavlov, Joris Soeurt (SNE) Green computing July 5, 2012 19 / 26

Estimating time distribution in parallel computing

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32

Time Needed for T ask (minutes) Number of nodes Time

Figure: Time Needed for Task, Tt = Ts

n + (n − 1) ∗ C

Dimitar Pavlov, Joris Soeurt (SNE) Green computing July 5, 2012 20 / 26

Total power consumption for a parallelized task

0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2 0.22

2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32

T

• tal Energy Consumed for T

ask (Switch) (Wh) Number of nodes Swith(es) Energy Consumption

Figure: Huawei S1728GWR-4P, Etask = (Pb ∗ Ns + Ppp ∗ Nn) ∗ T

Dimitar Pavlov, Joris Soeurt (SNE) Green computing July 5, 2012 21 / 26

Estimating optimal bandwidth for a fixed-datasize task

0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2

150 200 250 300 350 400 450 500 550 600 650 700 750 800 850 900 950 1000

T

• tal Switch Energy Consumed for T

ask (Wh) Speed (mbps) T

• tal Switch Energy Consumed for T

ask T

• tal Switch Energy Consumed

Figure: Huawei S1728GWR-4P, Etask = Ptotal ∗ (Ad∗8)

St

Dimitar Pavlov, Joris Soeurt (SNE) Green computing July 5, 2012 22 / 26

Prototype power calculator operation

Number of nodes Switch power profile Ports per switch Total time for task Communication overhead Speed Data size Available time

Calculator

Power as function of number of nodes Power as function of transmission rate

input

• utput

Dimitar Pavlov, Joris Soeurt (SNE) Green computing July 5, 2012 23 / 26

Conclusions

The 802.3az protocol can potentially optimize the energy efficiency of networked environments The technology, when applied within a distributed computing environment, contributes to green IT efforts 802.3az can save energy with the vast majority of traffic patterns To achieve optimal energy savings, one needs to perform low-level software changes

Dimitar Pavlov, Joris Soeurt (SNE) Green computing July 5, 2012 24 / 26

Future work

Extending the created energy model and applying it to other distributed computing environments Analyze actual cluster traffic patterns to optimize model and recommendations Work towards integrating computational performance into energy model Research applicability with multi-core architectures and incorporate into energy model Further investigate 802.3az in the context of UDP and 100BASE-TX Investigate 802.3az in the context of other transport protocols

Dimitar Pavlov, Joris Soeurt (SNE) Green computing July 5, 2012 25 / 26

Questions

Dimitar Pavlov - dimitar.pavlov@os3.nl Joris Soeurt - joris.soeurt@os3.nl

Dimitar Pavlov, Joris Soeurt (SNE) Green computing July 5, 2012 26 / 26