for Smart Power and Energy Systems 20.Nov.2019 Yi Wang, - - PowerPoint PPT Presentation
Data Analytics, Forecasting, and Optimization for Smart Power and Energy Systems 20.Nov.2019 Yi Wang, yiwang@eeh.ee.ethz.ch | 1 Appointment 2019.2- Postdoc, ETH Zurich (Prof. Gabriela Hug) Education 2010.9-2014.6 Bachelor, Huazhong
|
1
| 2
Education
2010.9-2014.6 Bachelor, Huazhong University of Science and Technology 2014.9-2019.1 Ph.D., Tsinghua University (Prof. Chongqing Kang) 2017.3-2018.4 Visiting Student, University of Washington (Prof. Daniel Kirschen)
Appointment
2019.2- Postdoc, ETH Zurich (Prof. Gabriela Hug)
Research Interests
Data Analytics for Smart Grid Cyber-Physical Power and Energy Systems Multi-Energy Systems Integration
| 3
| 4
| 5
220kV 380kV
| 6
Picture: M. Ruh
220kV – 380kV 36kV – 150kV 1kV – 36kV 0.4kV – 1kV
| 7
Picture: www.stromonline.ch
| 8
Picture: www.stromonline.ch
| 9
Picture: www.stromonline.ch
| 10
Installed Capacity/ 100GW 年份
Current (2018) WP: 210GW PV: 180GW 13th five year plan WP: 250 GW PV : 150 GW Basic Scenario： WP: 400 GW PV : 400 GW High Scenario WP: 1200 GW PV : 1000 GW High Scenario WP: 2400 GW PV : 2700 GW Basic Scenario WP: 1000 GW PV : 1000 GW Year
| 11
Wind Curtailment Rates of Different Provinces in 2016
Year China US 2012 2277 2347 2013 2323 2744 2014 2057 2759 2015 2042 2721 2016 2221 2756
Utilization Hour of Wind Power(China VS US)
Promoting the accommodation of renewable energy is an effective approach to the low-carbon energy systems !!!
| 12
Daily wind power Daily PV power
In traditional power system, the variation of load are balanced by controllable thermal generations and hydropower; the integration of high penetrated renewable energy present great uncertainties to the power systems.
| 13
The power generation and consumption should be balanced in real-time.
Real-time Balance in Power Systems Flexibility: Better ways of matching supply and demand over multiple time and spatio-scales. Supply Demand
Traditional Generation Renewable Energy Energy Demand
| 14
Flexibility
Quantify the requirement for flexibility: (1) Probabilistic Load Forecasting Explore flexibility
(2) Electricity Consumer Behavior Modeling Explore flexibility beyond the power systems: (3) Multi-Energy Systems Integration
| 15
Picture: Zongxiang Lu
Picture: Ning Zhang
| 16
Explore the value behind massive smart meter data, renewable energy data, economic data, etc. It helps to:
| 17
Picture: Ning Zhang
| 18
|
19
What is Probabilistic Load Forecasting? PLFs can be in the form of quantiles, intervals, or density functions.
Picture: Tao Hong
19
|
20
From point load forecasting to probabilistic forecasting?
Modeling Inputs Outputs
Residual Modeling & Outputs Ensemble Scenario Generation Probabilistic Models
Two-stage Bootstrap Sampling Temperature Scenario Generation Probabilistic Net Load Forecasting Pinball Loss Guided LSTM Combining Probabilistic Forecasts Conditional Residual Modeling
Predictions are ineluctably vitiated by errors, originating from noise in the explanatory variables (e.g. due to the chaotic nature of weather conditions) as well as model misspecifications.
20
| 21
| 22
In statistics and machine learning, ensemble methods use multiple learning algorithms to
learning algorithms alone [1]. Western Phrase: Two heads are better than one; Chinese Saying: Three vice-generals are equal to one Zhuge Liang
[1] https://en.wikipedia.org/wiki/Ensemble_learning
Which method is the best? Is it possible to combine these methods?
22
| 23
Various ensemble methods have been studied to combine multiple point forecasts. However, combining probabilistic load forecasts is a rarely touched area.
Combine point forecasts Combine probabilistic forecasts One dimension High dimension RMSE, MAPE Reliability, sharpness, calibration Analytical solution ???
Contributions of our works: New problem: Extend the ensemble method to the PLF area; Elegant formulation: Formulate the combining problem into an LP or QP model.
23
| 24
1) How to generate multiple PLF models? 2) Among the N forecasting models, how many and which methods should be selected for the final ensemble formation process? 3) How much weight should be given to each method for the optimal combination?
24
| 25
, 1 1 1
n
T N n n t t t n N n n n
Loss function Combined forecast Real load Summation and non-negative constraints Determine the weights A deep investigation of the loss function is the key to formulate and solve the optimization problem.
25
| 26
Pinball loss Continuous ranked probability score (CRPS)
2
CRPS , = 1
t t t t
F y F z z y dz
Pinball loss and CRPS assess the calibration and sharpness simultaneously, thus balancing the statistical consistency between the distributional forecasts and the
, , , , ,
ˆ ˆ ˆ PL , ˆ ˆ 1
t t q t q t t q t t q t t q t
y y q y y y y y y q y y
26
| 27
, , , , ,
ˆ ˆ ˆ PL , ˆ ˆ 1
t t q t q t t q t t q t t q t
y y q y y y y y y q y y
, 1 1 1
n
T N n n t t t n N n n n
, ,
ˆn q t y
, n q
,
, , , 1 1 , , 1
n q
T N n q n q t t t n N n q n q n
, , , , 1
N q t n q n q t n
27
| 28
, , , , , , , , , ,
ˆ ˆ arg min , ˆ ˆ =arg min max , 1 ˆ ˆ s.t. , 1, .
q q
q t q t q t t T t t q t q t t T q t n q n t q n q n q n N n N
L y y q y y q y y y y n
, , , , , , , , , , ,
ˆ arg min ˆ ˆ s.t. , 1, . ˆ ˆ , 1
q
q t q t T q t n q n t q n q n q n N n N t q t t q t q t q t
v y y n v q y y v q y y
, , ,
ˆ ˆ max , 1
t q t t q t q t
v q y y q y y Auxiliary decision variables
Constrained quantile regression averaging
28
| 29
We try to combine 13 individual methods and test the combined forecasts in ISO-NE dataset.
Pinball losses of different combining methods. Relative improvements compared with the best individual.
29
| 30
Models that are selected for different quantiles for total load (SYS).
30
Models that are selected for the 90-th quantile for different zones.
| 31
2
t t t t
The applications of the CRPS have been hampered by a lack of readily computable solutions to the integral:
1 CRPS , = 2
t t
F y E Y y E Y Y
This drawback is overcome by [1]:
[1] L. Baringhaus and C. Franz, “On a new multivariate two-sample test,” Journal of Multivariate Analysis, vol. 88, no. 1, pp. 190–206, 2004.
Let’s consider a simple case: Gaussian Mixture Distribution
31
| 32
Lemma 1: The expectation of an absolute value of a finite mixture distribution is the weighted sum of the corresponding expectations of absolute values of the components of the finite mixture distribution. If are the N components of the finite mixture distribution X with weights , then
1 N n n n
E X E X
1 2
, ,
N
X X X
1 2
, ,
N
t t
Two lemmas for Gaussian model:
32
Lemma 2: If X and Y are independent random variables that are finite mixtures of normal distributions, then their sum is also a finite mixture of normal
1 1 1 1
( ) ( | , ), ( ) ( | , ) 0, 0, 1, 1
L M X l l l Y m m m l m L M l m l m l m
f x x f y y
where is the PDF of normal distribution , then the PDF of Z=X+Y is: ( | , )
( , ) N
2 2 1 1
( ) ( | , )
L M Z l m l m l m l m
f z x
| 33
If individual density forecasts are Gaussian distributed , the combined forecast follows Gaussian mixture distribution:
( )
n
f x
1
( ) ( )
N X n n n
f x f x
Then, the CRPS can be calculated as:
1 1 1
1 CRPS( , ) 2
N N N n n i j i j n i j
F y E Y y E Y Y
The expectation of the absolute value of a normal distribution can be calculated as follows:
( , ) N
2 2
2
( ) ( ) ( ) 2 [2 ( ) 1] E X x f x dx xf x dx x f x dx e
33
| 34
Thus, we have:
, 1 1 1
CRPS( , )
N N N i j i j n n i j n
F y
2 2 2 , 2 2 2 2
( ) ( ) 1 exp( ) [2 ( ) 1] 2( ) 2 2
i j i j i j i j i j i j i j
where
2 2
( ) ( ) 2 exp( ) ( )[2 ( ) 1] 2
n n n n n n n
y y y
Finally, we have:
T T T
QP problem! Is this convex?
34
| 35
PDFs of predictions of four typical days given by the base models and their combination CRPS of the best individual model and combined models Performances of combined models
35
| 36
Relative CRPS improvements of the three combination methods Relative MAPE improvements of the three combination methods
36
| 37
Weights of the base models in the MAPE-guided model Weights of the base models in the CRPS-guided model
37
| 38
| 39
No. System/ Data Data Source Data Type Frequency Data Structure 1 Economic Information Statistic Bureau GDP、CPI、PMI（Purchasing Managers Index）、Sales Value、 Prosperity Index Per Month Non structural 2 Energy Consumption Data Energy Efficiency Platform Electrical Load、Output、Power Quality、Temperature 15Min Non structural /Structural 3 Meteorological Data Meteorological Bureau Temperature、Humidity、 Rainfall Per Day Structural 4 EV Charging Data Charging-Pile RTU Current、Voltage、Charging Rate、State of Charge 15Min Structural 5 Customer Service Voice Data Customer Service System Customer Voice Data Real Time Non structural
10 million Smart Meters, 15min 60GB per day, 21TB per year.
| 40
Participators and their businesses on the demand side
| 41
Data Analytics is commonly dissected into three stages: descriptive analytics (what do the data look like), predictive analytics (what is going to happen with the data), and prescriptive analytics (what decisions can be made from the data).
| 42
attitudes of customers under a certain environment to maximize the overall utility.
behavior utility, behavior results.
Subject Utility Results Means Environment Temporal Extension Forecasting Aggregation Spatial Extension
| 43
| 44
Retailers attempt to analyze customers’ electricity consumption behaviors, so that they can provide diversified and personalized services. Can we identify the social-demographic information of the consumers?
Challenges: 1)Problem formulation; 2)High-dimensional load data; 3)High time-shift invariance;
| 45
| 46
| 47
retired or not), #4 (have children or not), and #8 (cooking facility type) are higher than 75%;
proportion) are lower than 60%;
| 48
Characteristics of Individual Smart Meter Data
Sparsity: only a small fraction of the time has higher electricity consumption while the rest approximates to zero. Diversity: load profiles are various with different customers and in different days, but it can be decomposed into different parts.
| 49
1 K i ik k k
Idea of Sparse Coding
| 50
2 , ,
F i i k k n
1) Search a redundant dictionary D that captures the features or PUPs of load profiles as well as possible 2) Optimize the coefficient vector A of each load profile to guarantee its sparsity and an acceptable reconstruction error.
Non-Negative Sparse Coding
| 51
| 52
Linear SVM
| 53
Ten most relevant PUPs for SMEs and residential customers
Shape Duration Peak times SME Vaulted Long Dawn, working hours Resident Sharp peak Short Morning, night
| 54
Parameter RMSE MAE Accuracy F1 K-SVD 5, 80 0.099 0.060 0.874 0.793 k-means 80 0.120 0.180 0.786 0.752 PCA 5 0.111 0.167 0.771 0.764 DWT 5 0.141 0.327 0.667 0.688 PAA 6 0.112 0.181 0.706 0.725 Original 48 / / 0.735 0.724
Comparison with Different Techniques
TP TN Accuracy TP TN FP FN
2 1 precision recall F precision recall
1 1
1
K M i i i i i
MAE x a d K
2 1 1
1 ( )
K M i i i i i
RMSE x a d K
Data Compression-Based Classification-Based
| 55
How to provide diversified services for different consumers to enhance the competitiveness of the retailers?
55
| 56
Main Idea
Data-driven price design. Smart meter data contains great value which may help retailing price design. Respect self-selection. Consumers’ willingness and rights to choose must be respected.
Challenges Data-driven price design Compatible incentive design
56
| 57
Leader——Retailer
Follower——Consumers A Stackelberg game
57
| 58
Consumer Utility
Original electricity consumption is the realization of Utility Maximization! Consumer Strategy
Utility Maximization How can smart meter data be useful?
58
| 59
Compatible incentive Individual rationality If the retailer wants consumer k to choose pricing scheme r, the retailer must guarantee choosing r is consumer k’s dominant strategy If the retailer wants consumer k to choose new pricing scheme r, the retailer must guarantee choosing r is at least as good as previous situation
59
| 60
Forward contracts Day-ahead market Real-time market Where the retailer purchases electricity Balance predictable load Price uncertainty Balance unpredictable load Price and load uncertainty Risk loss measure——CVaR Purchasing strategy
Which is considered more important? Risk Weighting factor
60
| 61
Why clustering? Different Clustering Methods
Clustering evaluation
Davies Bouldin Index With-cluster compactness Between-cluster seperation One method may not fit all data sets
Centroid as representative
61
| 62
Optimization framework – an MINLP model
Lower Risk Less changes
62
| 63
Optimization framework – an MINLP model
* nonlinear terms are marked in red
Consumer payment Forward contracts DA Risk Loss in DA & RT Consumer load Forward contracts DA DA=Day-ahead market RT= Real-time market
63
Choosing pr is consumer k’s dominant strategy, k likes pr than any other pricing schemes Choosing pr is consumer k’s rational choice, k likes pr than the old pricing schemes
| 64
Optimization framework – an MINLP model
* nonlinear terms are marked in red
Reactions Utility
64
Loss in DA Loss in RT Price category：CPP RTP ToU Lower Risk Less changes m block ToU
| 65
Nonlinear model Linear model
Take as a whole
| 66
Nonlinear model Linear model
Conversed to linear equations
* new variables are marked in blue
| 67
Linear segment approximation(12 segments)
| 68
DB index result Clustering result
| 69
Personalized price Consumer response
| 70
ToU under different elasticity Total load under different elasticity
Elasticity Original
Retailing Profit($) 752 833 977 1186 1385
Retailing profit under different elasticity
Elasticity Willingness to change
| 71
How CVaR, the quantity of power bought from day-ahead market and through forward contracts changes with the change of risk weighting factor?
risk weighting factor rises attach more importance to risk minimize CVaR buy less from day-ahead market buy more through forward contracts
| 72
RP SW AP F/SC Original 752.03 0.2
HIA-COMP 1186.01 339.72 0.1947 65%/89% HIA-WARD 1188.70 10.01 0.1971 33%/59% KM-PLUS 1145.68 7.01 0.1973 9%/20% KM-SAMPLE 1137.61 4.50 0.1975 22%/48% KM-UNIFORM 1142.61 15.76 0.1973 11%/31% FCM(m=1.1) 1150.43 9.43 0.1970 30%/47% FCM(m=1.2) 1176.08 18.64 0.1968 19%/35% FCM(m=1.3) 1208.06 0.64 0.1970 8%/20% GMEM-PLUS 1145.82 36.01 0.1965 13%/28% GMEM-RAND 1144.85 46.60 0.1967 10%/24% How much profit does the retailer get? RP=Retaling Profit($) How much welfare do the consumers get? SW=Social Welfare AP=Average Price($/kWh) How well does clustering perform? F/SC=First/Second Choice
for consumers
| 73
| 74
Energy Hub
Heat Gas Electricity
…
Heat ,1 in
v
, in m
v
Cooling Electricity ,1
v
…
,2 in
v
,2
v
,
v
Focus more on energy delivery Focus more on energy conversion
2003 Vision of Future Energy Networks
| 75
Large potential in supplementary for renewable energy accommodation Power system is the core of the multiple energy systems
Electricity Generation Transmission Consumption
electricity and heat
Heat Gas
depending on the distribution of sources.
uneconomical storage
Energy Internet
intelligence
energy
|
Large-sized Combined Heat and Power (CHP) units have been installed The output power of CHP is determined by heat demand, which makes the CHP units less flexible This leads to huge wind power curtailment Why is the electric-heat coordination important?
76
|
77
|
Key problem 1 Unit/system modelling Key problem 2 Coupling theory
Modelling technique of MES A Planning theory of MES Operation theory of MES D B C
Simulation Platform
78
|
W
Q
R
QCHP
v
WCHP
v
,Q
v
,W
v
QWARG
v
,R
v
RWARG
v
FCHP
v
in
v
Heat Gas Electricity
Heat ,1 in
, in m
Cooling Electricity ,1
,2 in
,2
,
multiple inputs and multiple outputs.
,F ,R ,Q ,W
in in T
Q R R Q Q W
T
79
|
80
|
Node Branch Input Output
1
v
2
v
3
v
4
v
5
v
6
v
7
v
8
l
, in e
v
, in g
v
,
v
,
v
Port
Branch, describes the energy flow. Node, is the abstract of the energy converter, but also the abstraction of branch endpoints. Port, is defined as the interface of a node that exchange energy with others.
A MES consists of two basic elements: energy conversion devices and their connection relationship.
81
|
The port-branch incidence matrix is defined to describe the connection relation between the ports of a node and the branches. 1 branch is connected to input port of node 1 branch is connected to output port of node 0 branch is not connected to any port of node
b
b k g m b k g b g
W
Q
R
QCHP
v
WCHP
v
,Q
v
,W
v
QWARG
v
,R
v
RWARG
v
FCHP
v
in
v
Port-Branch Incidence Vectors and Matrices
FCHP QWARG QCHP WCHP RWARG
T
1
,1 ,2 ,
T g g g g k
82
|
W
Q
R
QCHP
v
WCHP
v
,Q
v
,W
v
QWARG
v
,R
v
RWARG
v
FCHP
v
in
v
Converter Characteristic Matrices
The nodal converter characteristic matrix Hg of node g defines the energy conversion characteristics of the node. Type 1: single input port and single output port Type 2: single input port and multiple output ports
2 R
,
f is the number of the input port f is the number of the output port i 1 i
ise
i i k
k k h i
Q 1 W
1 = 1 H
1,
1 if is the number of input port 1 if is the number of output port
k i
k h k i
1 Q W
= 1 1 1 H
83
|
Energy Conversion Matrices Given the port-branch incidence matrix and the converter characteristic matrix, we can calculate the branch energy conversion matrix for node g:
W
Q
R
QCHP
v
WCHP
v
,Q
v
,W
v
QWARG
v
,R
v
RWARG
v
FCHP
v
in
v
g g g
Q 1 W
The system energy conversion matrix Z is the combination of the nodal energy conversion matrix of all nodes in EH:
T T T 1 2
T N
2 R
84
|
The system energy conversion matrix Z is the combination of the nodal energy conversion matrix of all nodes in EH:
T T T 1 2
T N
Q W R
For the MES, the system energy conversion matrix Z is:
W
Q
R
QCHP
v
WCHP
v
,Q
v
,W
v
QWARG
v
,R
v
RWARG
v
FCHP
v
in
v
Then, we can obtain the energy conversion equation of the EH:
85
|
W
Q
CHP AB
AB
CERG
C
WARG
R
,W1 in
v
,W1
v
WQ
WQ
TD
TC
C
v
TS
1
v
2
v
3
v
4
v
6
v
5
v
7
v
8
v
10
v
9
v
11
v
12
v
,W2
v
,R1
v
,R2
v
,Q1
v
,Q2
v
,Q3
v
,Q4
v
1 D
v
,W2 in
v
,F1 in
v
,F2 in
v
W
v
2 D
v
Q
v
Case Studies
87
| 88
|
Problem Statement
89
|
1,1 1,2 1,3 1,4 1, 2 1, 1 1, 2,1 2,2 2,3 2,4 2, 2 2, 1 2, 3,1 3,2 3,3 3,4 3, 2 3, 1 3, 4,1 4,2 4,3 4,4 4, 2 4, 1 3, 2,1 2,2 2,3 2,4 2, 2 2, 1 2, 1,1 1,2 n n n n n n n n n n n n m m m m m n m n m n m m
x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x
1,3 1,4 1, 2 1, 1 1, ,1 ,2 ,3 ,4 , 2 , 1 , m m m n m n m n m m m m m n m n m n
x x x x x x x x x x x
Input #1 #2 #N-1 #N #1 #2 #3 #N Output … …
Input and Output Ports Incidence Matrix
Output port Input port Branch
1 2 3 4 1 2 3 ① ② ③ ④ ⑤
EH input EH output
90
|
ij
in
I O
1
K G I g g K g
, , , , 1 1 1 S T M in s m t s m t s s t m
C f V
O
2
l g l g
, , 1
l t s l
91
|
Investment Decisions Connection Relationship Operation Scenario 1 Operation Scenario 2 …… Operation Scenario S Investment Constraints Operation Constraints Connection Relationship Constraints
92
|
93
The Beijing government is planning to build a subsidiary administrative center in the undeveloped district of Tongzhou in the southeast of Beijing, containing Beijing municipal government and consist of offices, commercial buildings and residential buildings.
Total area:
– 155 square kilometers
Core district area:
– 6 square kilometers
Planned building area
– 3.8 million square meters.
|
10 20 30 40 50 60 70 80 1 192 383 574 765 956 1147 1338 1529 1720 1911 2102 2293 2484 2675 2866 3057 3248 3439 3630 3821 4012 4203 4394 4585 4776 4967 5158 5349 5540 5731 5922 6113 6304 6495 6686 6877 7068 7259 7450 7641 7832 8023 8214 8405 8596
Load/MW Time/Hour
Heating Load Cooling Load Electricity Load
94
|
200 400 600 800 1000 1200 1400 1600 20 40 60 80 100 120 140 160 180 200 Price/RMB/MWh Demand/MW
Electricity Cooling Heat Electricity Price
Scenario #1 Scenario #2 Scenario #3 Scenario #4 Scenario #5
95
|
96
|
97
CERG: compression electric refrigerator group WARG: water absorption refrigerator group EHP: electric heat pump HS: heat storage CS: cooling storage
|
98
Case 1: Planning results from the model Case 2: Wind power heating case Case 3 Import city heat network plan Case 4 Combine cooling and heating plan Case 5 Gas boiler plan Potential planning for Tongzhou
|
99
Comparison of the costs and emissions Plan 1 2 3 4 5 Total operation cost (104 ￥) 85049 88673 88622 87677 126582 Total emission (104 kg CO2) 220.97 238.92 241.82 231.22 241.27
| 100
| 101
Analytics and Optimization of Local Power and Energy Systems
| 102
Yi Wang | yiwang@eeh.ee.ethz.ch | www.eeyiwang.com