SLIDE 1 Increasing Large-Scale Data Center Capacity by Statistical Power Control
Guosai Wang, Shuhao Wang, Bing Luo, Weisong Shi, Yinghang Zhu, Wenjun Yang, Dianming Hu, Longbo Huang, Xin Jin, Wei Xu
SLIDE 2 Data Centers
Expensive to build and operate
Building cost (large DCs): $9,000–$13,000/KW* High power consumption: 10–20 MW
Goal: Fully utilize the capacity of data centers to reduce the TCO. Our Result:
- +17% servers → +15% throughput
- Power violations effectively avoided.
- No performance disturbance to existing jobs.
[*LA Barroso, etc. The datacenter as a computer: An introduction to the design of warehouse-scale machines. 2013]
SLIDE 3
Observation: Avg power utilization < 72% at DC level Reason: Conservative power provisioning
Provision according with rated power Running power < Rated power
Underutilized Capacity in DCs
SLIDE 4
Observation: Avg power utilization < 72% at DC level Reason: Conservative power provisioning
Provision according with rated power Running power < Rated power
Over-provisioning of the facility power?
Increase the number of servers on each rack.
Underutilized Capacity in DCs
SLIDE 5 Why People Under-provision?
[Fan X, etc. Power provisioning for a warehouse-sized computer. ISCA 2007]
Row Power Time Power limit Servers on the row level
SLIDE 6 Why People Under-provision?
[Fan X, etc. Power provisioning for a warehouse-sized computer. ISCA 2007]
Row Power Time Power limit Servers on the row level
Under-utilized capacity
SLIDE 7 Why People Under-provision?
[Fan X, etc. Power provisioning for a warehouse-sized computer. ISCA 2007]
Servers on the row level Over-provisioning Row Power Time Power limit
SLIDE 8 Why People Under-provision?
[Fan X, etc. Power provisioning for a warehouse-sized computer. ISCA 2007]
Servers on the row level Over-provisioning Row Power Time Power limit
Power violation!
SLIDE 9 Traditional approach: Power capping
Dynamic Voltage and Frequent Scaling (DVFS) Power ≈ C·V²·F
Degrade the performance of running jobs!
Violate the SLA of the latency-sensitive jobs.
Row Power Time
Power Capping Degrades Performance
SLIDE 10
Traditional approach: Power capping
Dynamic Voltage and Frequent Scaling (DVFS) Power ≈ C·V²·F
Degrade the performance of running jobs!
Violate the SLA of the latency-sensitive jobs.
Power Capping Degrades Performance
SLIDE 11
Can we control the power without affecting the performance of existing jobs?
Power Control Method
SLIDE 12 Large variations on power utilization at row level
Temporal (over time) and spatial (across different rows).
Idea: Dynamically move workload out of the heavily used rows.
Key Observation
Time/hour Normalized Row Power Time/hour Row
SLIDE 13 Large variations on power utilization at row level
Temporal (over time) and spatial (across different rows).
Idea: Dynamically move workload out of the heavily used rows.
Key Observation
Time/hour Normalized Row Power Time/hour Row
SLIDE 14 Large variations on power utilization at row level
Temporal (over time) and spatial (across different rows).
Idea: Dynamically move workload out of the heavily used rows.
Key Observation
Time/hour Normalized Row Power Time/hour Row
SLIDE 15
the scheduler.
new job placement.
Our Solution: Statistical Power Control
Two simple APIs: Freeze/unfreeze. Decoupled with the over- complicated scheduler. Indirect workload balancing. Running jobs unaffected. Does not necessarily work perfectly. Tolerate noises. System identification in a production environment.
SLIDE 16 Light workload
No control action.
Example: Statistical Power Control
Scheduler
Running Jobs
Power Controller
Aggregated real-time power
SLIDE 17 Light workload
No control action.
Example: Statistical Power Control
Scheduler
Running Jobs
Power Controller
Aggregated real-time power New jobs
SLIDE 18 Light workload
No control action.
Scheduler
Running Jobs
Power Controller
New jobs Aggregated real-time power
Example: Statistical Power Control
SLIDE 19 Light workload
No control action.
Scheduler
Running Jobs
Power Controller
New jobs Aggregated real-time power
Example: Statistical Power Control
SLIDE 20 Light workload
No control action.
Scheduler
Running Jobs
Power Controller
New jobs Aggregated real-time power
Example: Statistical Power Control
SLIDE 21 Light workload
No control action.
Scheduler
Running Jobs
Power Controller
New jobs Aggregated real-time power
Example: Statistical Power Control
SLIDE 22 Heavy workload.
High row power.
Scheduler
Running Jobs
Power Controller
Aggregated real-time power
Example: Statistical Power Control
SLIDE 23 Heavy workload.
High row power.
Scheduler
Running Jobs
Power Controller
Aggregated real-time power
Freeze
Example: Statistical Power Control
SLIDE 24 Heavy workload.
High row power.
Scheduler
Running Jobs
Power Controller
New jobs Aggregated real-time power
Freeze
Example: Statistical Power Control
SLIDE 25 Heavy workload.
High row power.
Scheduler
Running Jobs
Power Controller
New jobs Aggregated real-time power
Freeze
Example: Statistical Power Control
SLIDE 26 Heavy workload.
High row power.
Scheduler
Running Jobs
Power Controller
New jobs Aggregated real-time power
Freeze
Example: Statistical Power Control
SLIDE 27 Heavy workload.
High row power.
Scheduler
Running Jobs
Power Controller
New jobs Aggregated real-time power
Freeze
Example: Statistical Power Control
SLIDE 28 Heavy workload.
High row power.
Scheduler
Running Jobs
Power Controller
New jobs Aggregated real-time power
Freeze
Example: Statistical Power Control
SLIDE 29 Heavy workload.
High row power.
Scheduler
Running Jobs
Power Controller
New jobs Aggregated real-time power
Freeze
Unused power Jobs
Example: Statistical Power Control
SLIDE 30 Some jobs finished.
Scheduler
Running Jobs
Power Controller
Aggregated real-time power
Freeze
Example: Statistical Power Control
SLIDE 31 Some jobs finished.
Scheduler
Running Jobs
Power Controller
Aggregated real-time power
Unfreeze
Example: Statistical Power Control
SLIDE 32 Some jobs finished.
Scheduler
Running Jobs
Power Controller
Aggregated real-time power
Unfreeze
Example: Statistical Power Control
SLIDE 33 Power Control Model Blueprint
- Dynamic control at each minute.
- No control needed when the power is low.
- Freeze more/fewer servers when power is high/low.
SLIDE 34 Power Control Model Blueprint
- Dynamic control at each minute.
- No control needed when the power is low.
- Freeze more/fewer servers when power is high/low.
?
SLIDE 35 Power Control Model Blueprint
- Dynamic control at each minute.
- No control needed when the power is low.
- Freeze more/fewer servers when power is high/low.
? ?
SLIDE 36 10 20 30 40 50 0.68 0.7 0.72 0.74 0.76 0.78 0.8 0.82 0.84
Time/min Normalized Server Power
Two effects jointly impact on the row-level power.
- Existing jobs will finish
- Statistically fewer jobs scheduled to the row
Effect of Freezing Servers
Fig: Average normalized power of about 80 servers after they are frozen.
SLIDE 37 Two effects jointly impact on the row-level power.
- Existing jobs will finish
- Statistically fewer jobs scheduled to the row
How to quantify these effects?
System identification in a production environment? Designed a controlled experiment.
Effect of Freezing Servers
SLIDE 38 Controlled experiment in production environment. Idea: A/B testing
Controlled Experiment Design
Row 1 Row 2 Row n
SLIDE 39 Controlled experiment in production environment. Idea: A/B testing
Controlled Experiment Design
Row 1 Row 2 Row n
SLIDE 40 Controlled experiment in production environment. Idea: A/B testing
Controlled Experiment Design
Row 1 Row 2 Row n
SLIDE 41 Controlled experiment in production environment. Idea: A/B testing
Controlled Experiment Design
Row 1 Row 2 Row n
Experiment Group Control Group
Correlation coefficient of the group power is 0.946
Power Controller
Control Actions
SLIDE 42
Dynamic Control Model
How many servers do we need to freeze in a row?
Freeze too few: Risk of Power violations! Freeze too many: Reduce the throughput!
Optimization problem:
Maximize: TPW (Throughput per Provisioned Watt) s.t. No power violation
Key idea:
Use simple system model and tolerate inaccuracy with dynamic control.
SLIDE 43 Use heuristics to derive a simple control model.
Take control actions at each minute. Details in the paper.
Dynamic Control Model
Realtime Row Power Freezing Ratio
SLIDE 44 Use heuristics to derive a simple control model.
Take control actions at each minute. Details in the paper.
Dynamic Control Model
Realtime Row Power Freezing Ratio
SLIDE 45 Use heuristics to derive a simple control model.
Take control actions at each minute. Details in the paper.
Dynamic Control Model
Realtime Row Power Freezing Ratio
SLIDE 46 Use heuristics to derive a simple control model.
Take control actions at each minute. Details in the paper.
Dynamic Control Model
Realtime Row Power Freezing Ratio
SLIDE 47 Use heuristics to derive a simple control model.
Take control actions at each minute. Details in the paper.
Dynamic Control Model
Realtime Row Power Freezing Ratio
SLIDE 48 Use heuristics to derive a simple control model.
Take control actions at each minute. Details in the paper.
Dynamic Control Model
Realtime Row Power Freezing Ratio
SLIDE 49
- Safety: Unacceptable to truly trigger power violations in
production environment.
- Flexibility: How to test various over-provisioning ratio?
Solution: Emulating power violations by virtually scaling down the power budget of the row.
How to Emulate Over-provisioning?
SLIDE 50
- Safety: Unacceptable to truly trigger power violations in
production environment.
- Flexibility: How to test various over-provisioning ratio?
Solution: Emulating power violations by virtually scaling down the power budget of the row.
How to Emulate Over-provisioning?
Actual row power budget: P Assumed row power budget: P’
Over-provisioning ratio: (P-P’)/P’
P
SLIDE 51
- Safety: Unacceptable to truly trigger power violations in
production environment.
- Flexibility: How to test various over-provisioning ratio?
Solution: Emulating power violations by virtually scaling down the power budget of the row.
How to Emulate Over-provisioning?
Actual row power budget: P Assumed row power budget: P’
Over-provisioning ratio: (P-P’)/P’
P P’
SLIDE 52
Controlled experiments on production environment. Over-provisioning ratio = 0.25
Effectiveness
SLIDE 53
Controlled experiments on production environment. Over-provisioning ratio = 0.25
Effectiveness
SLIDE 54
Controlled experiments on production environment. Over-provisioning ratio = 0.25
Effectiveness
SLIDE 55
Controlled experiments on production environment. Over-provisioning ratio = 0.25
Effectiveness
SLIDE 56
Controlled experiments on production environment. Over-provisioning ratio = 0.25
Effectiveness
SLIDE 57
Controlled experiments on production environment. Over-provisioning ratio = 0.25
Effectiveness
SLIDE 58 Throughput per Provisioned Watt (TPW): Gain in TPW:
How to Decide Over-provisioning Ratio?
TPW = Throughput during time interval T P⋅T
GTPW = r
T ⋅(1+r O)−1
P Provisioned power r
T Throughput ratio (≤1)
r
O Over-provisioning ratio (≥1)
SLIDE 59 Throughput per Provisioned Watt (TPW): Gain in TPW:
How to Decide Over-provisioning Ratio?
TPW = Throughput during time interval T P⋅T
GTPW = r
T ⋅(1+r O)−1
P Provisioned power r
T Throughput ratio (≤1)
r
O Over-provisioning ratio (≥1)
r
T <1
SLIDE 60 Throughput per Provisioned Watt (TPW): Gain in TPW:
By emulations we found when .
TPW = Throughput during time interval T P⋅T
GTPW = r
T ⋅(1+r O)−1
P Provisioned power r
T Throughput ratio (≤1)
r
O Over-provisioning ratio (≥1)
GTPW = 0.149 r
O = 0.17
How to Decide Over-provisioning Ratio?
SLIDE 61
statistically influencing new job placement
unfreeze)
control
Conclusion
Avoid performance degradation. Decouple the power control module and the complicated scheduler. Tolerate inaccuracy. Build and evaluate system model in production environment without disturbing it too much.
SLIDE 62
statistically influencing new job placement
unfreeze)
control
Conclusion
Avoid performance degradation. Decouple the power control module and the complicated scheduler. Tolerate inaccuracy. Build and evaluate system model in production environment without disturbing it too much.
SLIDE 63
Outline:
Power over-provisioning motivation Ideas of statistical power control Dynamic Control model Controlled experiment design Effectiveness Deciding over-provisioning ratio Conclusion
Q&A
SLIDE 64
Outline:
Power over-provisioning motivation Ideas of statistical power control Dynamic Control model Controlled experiment design Effectiveness Deciding over-provisioning ratio Conclusion
Q&A
SLIDE 65
Outline:
Power over-provisioning motivation Ideas of statistical power control Dynamic Control model Controlled experiment design Effectiveness Deciding over-provisioning ratio Conclusion
Q&A
SLIDE 66
Outline:
Power over-provisioning motivation Ideas of statistical power control Dynamic Control model Controlled experiment design Effectiveness Deciding over-provisioning ratio Conclusion
Q&A
SLIDE 67
Outline:
Power over-provisioning motivation Ideas of statistical power control Dynamic Control model Controlled experiment design Effectiveness Deciding over-provisioning ratio Conclusion
Q&A
SLIDE 68
SLIDE 69
Backup Slides
SLIDE 70 Ampere Architecture
Row #1 Row #2 Row #n
Scheduler Power Monitor Controller
... ...
freeze/unfreeze scheduling actions aggregated power
SLIDE 71
Job Durations
SLIDE 72 under Different
GTPW
r
O
Fix r
O,P mean ↗⇒ umean ↗⇒ r T ↘⇒ GTPW ↘
r
O ↗⇒ umean ↗
GTPW < r
O
SLIDE 73
The effects of freezing ratio u on the power change f(u).
Quantify the Effect of Freezing Ratio
SLIDE 74
What if the workload increases in the future? What if the jobs are locality-aware scheduled? What if the amount of jobs is small and they are long- lived? How to jointly optimize the control among all rows? Experiments needed before deployment?
Limitations and Discussion
