1)Why is Forecasting Important? The stability of the power grid is - - PDF document

▶

Sep 11, 2023 183 likes •362 views

Has Wind Energy Forecasting Solved the Challenge Posed by Intermittency? Evidence from the United Kingdom Kevin F. Forbes Ernest M. Zampelli School of Business and Economics The Catholic University of America Forbes@CUA.edu 9 th Annual

SLIDE 1

Has Wind Energy Forecasting Solved

the Challenge Posed by Intermittency? Evidence from the United Kingdom

Kevin F. Forbes Ernest M. Zampelli School of Business and Economics The Catholic University of America Forbes@CUA.edu

9th Annual Trans-Atlantic Infraday Federal Energy Regulatory Commission Washington, DC 30 October 2015

The Organization of this Talk

1)Why is Forecasting Important? 2) The Literature on Wind Energy Forecast Accuracy 3) What is the level of forecast skill ? Specifically, what does the Mean Squared Error Skill Score (MSESS) indicate about the solar and wind energy forecasts? How does this level of accuracy compare to the accuracy

f the load forecasts?

4)From the point of view of a system operator, how does wind energy compare with conventional forms of generation? 5)What are the prospects for improving the accuracy of the wind, solar, and load forecasts?

SLIDE 2

1)Why is Forecasting Important?

The stability of the power grid is enhanced when forecasts are more
accurate. This is important because blackouts have very high societal

costs

Some forms of balancing technologies such as open-cycle gas

turbines have above average emissions factors.

The market price of upward balancing power can be very costly.

Errors in the Day-Ahead Load Forecast for New York City and the Differential between the Real-Time and Day-Ahead Prices in New York City, 6 August 2009 – 30 June 2013.

Note: Excludes the period of time when operations were affected by Superstorm Sandy in late October 2012

SLIDE 3

2) The Literature on Forecast Accuracy

Some researchers calculate a root-mean-squared error of the forecasts and then weight it by the capacity of the equipment used to produce the energy. The reported capacity weighted root mean squared errors (CWRMSE) are usually less than 10

percent. Adherents of this approach include Lange, et al. (2006, 2007), Cali et al. (2006), Krauss, et al. (2006), Holttinen, et
al. (2006), Kariniotakis, et al. (2006), and even NERC (2010, p. 9).

In a publication entitled, “Wind Power Myths Debunked,” Milligan, et al. (2009) draw on research from Germany to argue that it is a fiction that wind energy is difficult to forecast. In their words: “In other research conducted in Germany, typical wind forecast errors for a single wind project are 10% to 15% root mean-squared error (RMSE) of installed wind capacity (emphasis added) but drop to 5% to 7% for all of Germany.” (Milligan, et al. 2009, p. 93) The UK’s Royal Academy of Engineering (2014, p. 33) has noted that wind energy’s capacity weighted forecast error of about five percent is evidence that that the wind energy forecasts are highly accurate. A report by the IPCC ( 2012 p, 623) on renewable energy indicates that wind energy is moderately predictable as evidenced by a capacity weighted RMS forecast error that is less than 10%. Solar energy is reported to be even more accurate.

The Literature on Forecast Accuracy (Continued)

NREL (2013) implicitly endorses capacity weighted RMSEs for wind energy but makes use of energy weighted RMSEs when discussing the accuracy of load forecasts. In contrast, Forbes et. al. (2012) calculate a root-mean-squared forecast error for wind energy in nine electricity control areas. The RMSEs are normalized by the mean level of wind energy that is

produced. The reported energy weighted root mean squared errors

(EWRMSE) are in excess of 20 %.

SLIDE 4

3) Using The Mean-Squared-Error Skill Score (MSESS) to Assess Forecast Accuracy

A useful alternative to both the energy weighted and capacity weighted RMSE is the mean-squared-error skill score (MSESS). With this metric, one can evaluate the skill of a forecast relative to a persistence forecast, a persistence forecast being a period-ahead forecast that assumes that the outcome in period t equals the outcome in period t-1. The MSESS with the persistence forecast as a reference is calculated as follows: Where is the mean squared error of the forecast that is being evaluated and is the mean squared error a persistence forecast. A perfect forecast would have a MSESS equal to one. A MSESS equal to zero indicates that the forecast skill is equal to that

f a persistence forecast. A negative MSESS indicates that the forecast under

evaluation is inferior to a persistence forecast.

How accurate are the forecasts?

MSESS were computed for the following zones and/or control areas:
Bonneville Power Administration
CAISO: SP15 and NP15
MISO
PJM
50Hertz in Germany
Amprion in Germany
Elia in Belgium
RTE in France
National Grid in Great Britain
Finland
Sweden
Norway
Eastern Denmark
Western Denmark
When possible the MSESS are reported for Wind, Solar, and Load

SLIDE 5

Mean Squared Error Skill Scores (MSESS) with a Persistence Forecast as Reference

Control Area/Zone Forecast Type Sample Period Observations Granularity MSESS 50Hertz (Germany) Day-Ahead Load 1Jan2011 – 31Dec2013 104,590 Quarter-Hour

62.7486

Day-Ahead Wind 1Jan2011 – 31Dec2013 104,590 Quarter-Hour

31.3501

Day-Ahead Solar 1Jan2011 – 31Dec2013 54,545 Quarter-Hour

5.26831

Amprion (Germany) Day-Ahead Load 1Jan2011 – 31Dec2013 103,326 Quarter-Hour

12.3308

Day-Ahead Wind 1Jan2011 – 31Dec2013 103,326 Quarter-Hour

14.5887

Day-Ahead Solar 1Jan2011 – 31Dec2013 55,498 Quarter-Hour

11.20691

Mean Squared Error Skill Scores (MSESS) with a Persistence Forecast as Reference (Continued)

Control Area/Zone Forecast Type Sample Period Observations Granularity MSESS

California ISO Day-Ahead Load 1Jan2013 – 31Dec2013 8,760 Hourly 0.6026 NP15 Day-Ahead Wind 1Jan2013 – 31Dec2013 8,704 Hourly

6.1401

NP15 Hour-Ahead Wind 1Jan2013 – 31Dec2013 8,704 Hourly

2.3605

NP15 Day-Ahead Solar 1Jan2013 – 31Dec2013 8,666 Hourly

3.2002

NP15 Hour-Ahead Solar 1Jan2013 – 31Dec2013 8,666 Hourly

2.4846

SP15 Day-Ahead Wind 1Jan2013 – 31Dec2013 8,752 Hourly

4.8210

SP15 Hour-Ahead Wind 1Jan2013 – 31Dec2013 8,752 Hourly

2.1894

SP15 Day-Ahead Solar 1Jan2013 – 31Dec2013 8,752 Hourly 0.7050 SP15 Hour-Ahead Solar 1Jan2013 – 31Dec2013 8,752 Hourly 0.7972

SLIDE 6

Mean Squared Error Skill Scores (MSESS) with a Persistence Forecast as Reference (Continued)

Control Area/Zone Forecast Type Sample Period Observations Granularity MSESS Belgium Day-Ahead Solar 1Jan2013 – 31Dec2013 17,921 Quarter-Hour

12.2621

Intra-Day Solar 1Jan2013 – 31Dec2013 11,278 Quarter-Hour

9.7931

France Day-Ahead Load 1Jan2012 – 31Dec2013 35,088 Half-Hourly 0.3842 Day-Ahead Wind 1Jan2012 – 31Dec2013 17,349 Hourly

5.7375

Hour 1 Same Day, Wind 1Jan2012 – 31Dec2013 15,109 Hourly

5.2889

Norway Day-Ahead Load 1Jan2011 – 31Dec2013 26,160 Hourly 0.1870 Sweden Day-Ahead Load 1Jan2011 – 31Dec2013 26,160 Hourly 0.2008 Finland Day-Ahead Load 1Jan2011 – 31Dec2013 26,159 Hourly 0.0486 Eastern Denmark Day-Ahead Load 1Jan2011 – 31Dec2013 26,160 Hourly 0.3953 Day-Ahead Wind 1Jan2011 – 31Dec2013 26,107 Hourly

2.7507

Western Denmark Day-Ahead Load 1Jan2011 – 31Dec2013 26,160 Hourly 0.6560 Day-Ahead Wind 1Jan2011 – 31Dec2013 26,105 Hourly

3.6749

Mean Squared Error Skill Scores (MSESS) with a Persistence Forecast as Reference (Continued)

Control Area/Zone Forecast Type Sample Period Observations Granularity MSESS

MISO Day-Ahead Wind Energy 1Jan2011 – 31Dec2013 26,303 Hourly

4.3873

PJM Day-Ahead Load 1Jan2011 – 31Dec2013 26,160 Hourly 0.4727 New York City Day-Ahead Load 1Jan2011 – 31Dec2013 25,675 Hourly 0.1703 Bonneville Power Five Minute-Ahead Wind 1Jan2012 – 31Dec2013 206,477 Five minutes

36.25762

Hour-Ahead Wind 1Jan2012 – 31Dec2013 16,847 Hourly

63.57602

Great Britain Day-Ahead Load 1Jan2012 – 31Dec2013 30,477 Half-Hourly 0.62 Day-Ahead Wind 1Jan2012 – 31Dec2013 30,477 Half-Hourly

19.032

1 Daylight portion of the sample period 2MSESS calculation excludes periods in which wind energy production was curtailed by the

system operator.

SLIDE 7

4)From the point of view of a system operator, how does wind energy compare with conventional forms of generation? Evidence from Great Britain

In Great Britain, each generating station informs the system operator of its

intended level of generation one hour prior to real-time. This value is known as the final physical notification (FPN).

Generators also submit bids (a proposal to reduce generation) and offers (a

proposal in increase generation) to provide balancing services

During real-time, the system operator accepts the bids and offers based on system

conditions.

In short, the revised generation schedule equals the FPN plus the level of

balancing services volume requested by the system operator.

Failure to follow the revised generation schedule gives rise to an electricity market

imbalance that needs to be resolved by other generators.

The Revised Generation Schedules vs Actual Generation: The Case of Coal in Great Britain

2000 4000 6000 8000 10000 12000

Metered Generation (MWh)

2000 4000 6000 8000 10000 12000

Scheduled Generation including Balancing Actions (MWh)

EWRMSE = 2.5 %

SLIDE 8

The Revised Generation Schedules vs Actual Generation: The Case of Combined Cycle Gas Turbines in Great Britain

2500 5000 7500 10000 12500

Metered Generation (MWh)

2500 5000 7500 10000 12500

Scheduled Generation including Balancing Actions (MWh)

EWRMSE = 5.6%

Actual vs. Scheduled Generation: The Case of Nuclear Energy in Great Britain

1000 2000 3000 4000 5000

Metered Generation (MWh)

1000 2000 3000 4000 5000

Scheduled Generation (MWh)

EWRMSE = 7.4 %

SLIDE 9

The Revised Generation Schedules vs Actual Generation: The Case of Wind Energy in Great Britain, 1 Jan 2012 – 31 Dec 2013

500 1000 1500 2000 2500 3000

Metered Generation (MWh)

500 1000 1500 2000 2500 3000

Scheduled Generation including Balancing Actions (MWh)

EWRMSE= 18 %

Average Imbalances by Fuel in Great Britain, 1 Jan 2012- 31 December 2013

SLIDE 10

A Closer look at the Wind Energy Imbalances, 1 Jan 2012 – 30 June 2014

5) The Prospects for Improving the Forecasts

Significant improvements in day-ahead forecasts will probably

require major advances in meteorological research. One obvious place to begin is to note that the heat trapping properties of Greenhouse gases most likely have implications for wind speeds.

Significant improvements in very short run forecasts (e.g. one or two

hours ahead) are possible by exploiting the systematic nature of the existing forecast errors.

SLIDE 11

The Systematic Nature of the Existing Day-Ahead Forecast Errors for Wind Energy: Evidence from Great Britain

Out of Sample Results for Wind Energy in Great Britain, 1 Jan 2014 – 30 June 2014

Forecast Type Number of Observations MSESS EWRMSE Day-Ahead Wind Forecast 8,571

35.05

31.9 % Forecast equal to the levels of generation declared by operators

ne hour prior to real-

time 8,571

19.71

24.2 % Modified Forecast: available to system

perator 30 min prior to

real-time 8,571

1.95

9.1 %

SLIDE 12

Further Analysis

What about other Power Systems ?
What about Solar Energy ?

Hour-Ahead Forecasted and Actual Wind Energy in SP15 in CAISO, 1 January – 30 September 2015

EWRMSE = 37.1 % MSESS = -2.62

SLIDE 13

Actual Wind Energy and a Revised Hour-Ahead Wind Energy Forecast for SP15 in CAISO), 1 January – 30 September 2015

EWRMSE = 15.8 % MSESS = 0.34

Actual Solar Energy in 50Hertz in Germany and an Out-of-Sample Econometrically Modified Solar Energy Forecast, 1 July 2013 – 3 March 2014

For the daylight period: EWRMSE = 4.8 % MSESS = 0.768

SLIDE 14

Summary and Conclusions

With few exceptions, the load forecasts examined in this study have positive skill scores

relative to a persistence load forecast.

With few exceptions, the solar and wind forecasts examined in this study have negative skill

scores relative to the corresponding persistence forecasts.

Evidence has been presented that the forecast errors have a systematic component
Evidence has also been presented that econometric modelling of this systematic

component can yield short-run solar and wind energy forecasts that are significantly more

accurate. This does not resolve the challenge of intermittency but may mitigate matters.
The modelling approach can also be applied to improve load forecasts. See

http://dialogue.usaee.org/index.php/day-ahead-market-prices-of-electricity-and-economic- fundamentals-preliminary-evidence-from-new-york-city

References

Godfrey Boyle, 2010. Renewable energy technologies for electricity generation, in Harnessing Renewable Energy in Electric Power Systems, Boaz Moselle, Jorge Padilla, and Richard Schmalenese (eds.), RFF Press, Washington, DC, 2010, at 7-29. California Independent System Operator, ISO New England, Midwest Independent Transmission System Operator, New York Independent System Operator , PJM Interconnection, and Southwest Power Pool, 2010. 2010 ISO/RTO Metrics Report. At http://www.isorto.org/atf/cf/%7B5B4E85C6-7EAC-40A0-8DC3- 003829518EBD%7D/2010%20ISO-RTO%20Metrics%20Report.pdf <last accessed 15 feb 2012> Ümit Cali, Bernhard Lange, Rene Jursa, Kai Biermann, 2006. Short-term prediction of distributed generation – Recent advances and future challenges, Elftes Kasseler Symposium Energie-Systemtechnik. At http://www.iset.uni-kassel.de/public/kss2006/KSES_2006.pdf <last accessed 15 feb 2012> Mark A. Delucchi and Mark Z. Jacobson, 2011. Providing all global energy with wind, water, and solar power, Part II: Reliability, system and transmission costs, and policies. Energy Policy, 39, at 1170-1190. European Wind Energy Association, 2007. Debunking the Myths. At http://www.ewea.org/fileadmin/ewea_documents/documents/publications/wind_benefits/Windpower_is_unreliable.pdf <last accessed 15 feb 2012> Kevin Forbes, Marco Stampini, and Ernest M. Zampelli, 2012a. Are Policies to Encourage Wind Energy Predicated on a Misleading Statistic?, The Electricity Journal, Volume 25, Issue 3, pp. 42-54 Kevin Forbes, Marco Stampini, and Ernest M. Zampelli, 2012b. Do Policies to Encourage Wind Energy Inadvertently Pose Challenges to Electric Power Reliability? Evidence from the 50Hertz Control Area in Germany, The Electricity Journal, November 2012, Volume 25, Issue 9, pp. 37-42 GE Energy, 2010. Western Wind and Solar Integration Study, NREL/SR-550-47434, National Renewable Energy Laboratory, Golden, Colorado, May. At http://www.nrel.gov/wind/systemsintegration/pdfs/2010/wwsis_final_report.pdf <last accessed 15 feb 2012> Gregor Giebel, Richard Brownsword, George Kariniotakis, Michael Denhard, and Caroline Draxl, 2011. The State-Of-The-Art in Short-Term Prediction of Wind Power A Literature Overview, 2nd Edition. Project report for the Anemos.plus and SafeWind projects. 109 pp. Risø, Roskilde, Denmark. Available at http://130.226.56.153/zephyr/publ/GGiebelEtAl-StateOfTheArtInShortTermPrediction_ANEMOSplus_2011.pdf <last accessed 15 feb 2012> Hannale Holttinen, Peter Meibom, Antje Orths, Frans van Hulle, Bernhard Lange, Mark O’Malley, Jan Pierik, Bart Ummels, John Olav Tande,Ana Estanqueiro, Manuel Matos, Emilio Gomez, Lennart Söder, Goran Strbac, Anser Shakoor, Joao Ricardo, J. Charles Smith, Michael Milligan, and Erik Ela, 2009. IEA WIND Task 25: Design and operation of power systems with large amounts of wind power. At http://www.vtt.fi/inf/pdf/tiedotteet/2009/T2493.pdf <last accessed 15 feb 2012>

SLIDE 15

References (Continued)

Hannale Holttinen, Pirkko Saarikivi, Sami Repo, Jussi Ikäheimo, Goran Koreneff, 2006. Prediction Errors and Balancing Costs for Wind Power Production in Finland. Global Wind Power Conference, Adelaide Intergovernmental Panel on Climate Change, 2012, Renewable Energy Sources and Climate Change Mitigation Special Report of the Intergovernmental Panel on Climate Change. At http://srren.ipcc-wg3.de/report/IPCC_SRREN_Full_Report.pdf George Kariniotakis, 2006. State of the art in wind power forecasting, 2nd International Conference on Integration of Renewable Energies and Distributed Energy Resources, Napa, California/USA, 4-8 December. Mattias Lange and Ulrich Focken, 2005. State-of-the-Art in Wind Power Prediction in Germany and International Developments. Prediction of Wind Power and Reducing the Uncertainty for Grid Operators, Second Workshop of International Feed-In Cooperation, Berlin (DE) http://www.energymeteo.de/media/fic_eeg_article.pdf <last accessed 15 feb 2012> Bernhard Lange, Kurt Rohrig, Bernhard Ernst, Florian Schlögl, Umit Cali, Rene Jursa, and Javad Moradi, 2006. Wind power prediction in Germany – Recent advances and future challenges. European Wind Energy Conference and Exhibition, Athens (GR). Bernhard Lange, Kurt Rohrig, Florian Schlögl, Umit Cali, and Rene Jursa,2006. Wind Power Forecasting. in: Boyle, G.(Ed.), Renewable Electricity and the Grid. Earthscan, London, England, at 95-120. Bernhard Lange, Arne Wessel, Jan Dobschinski, and Kurt Rohrig, 2009. Role of Wind Power Forecasts in Grid Integration Kasseler Symposium Energie-Systemtechnik, at 118-130 http://www.iset.uni-kassel.de/public/kss2009/2009_KSES_Tagungsband.pdf <last accessed 15 feb 2012> Bernhard Lange, Kurt Rohrig, Bernhard Ernst, Florian Schlögl, Umit Cali, Rene Jursa, and Javad Moradi, 2006. Wind power prediction in Germany – Recent advances and future challenges, Zeitschrift für Energiewirtschaft, vol. 30, no 2, at115-120. At http://www.iset.uni- kassel.de/abt/FB-I/publication/Lange-et-al_2006_EWEC_paper.pdf <last accessed 15 feb 2012>

References (Continued)

Bernhard Lange, Kurt Rohrig, Florian Schlögl, Umit Cali,and Rene Jursa, 2007. Wind Power Forecasting, in Renewable Electricity and the Grid, Godfrey Boyle, Ed. Sterling,VA: Earthscan, London, at 95-120. David Milborrow, 2007. Wind Power on the Grid, in Renewable Electricity and the Grid, Godfrey Boyle, Ed. Sterling,VA: Earthscan, London, at 31-54 Michael Milligan,Kevin Porter, Edgar DeMeo, Paul Denholm, Hannele Holttinen, Brendan Kirby, Nicholas Miller, Andrew Mills, Mark O’Malley, Matthew Schuerger, and Lennart Soder , 2009. Wind Power Myths Debunked, IEEE Power and Energy, November/December vol 7 no 6, at 89-99. National Grid, 2009. Operating the Electricity Transmission Networks in 2020: Initial Consultation. At http://www.nationalgrid.com/NR/rdonlyres/32879A26-D6F2-4D82-9441- 40FB2B0E2E0C/39517/Operatingin2020Consulation1.pdf <last accessed 15 feb 2012> North American Electric Reliability Corporation, 2009b. Accommodating High Levels of Variable Generation, April. At http://www.nerc.com/files/IVGTF_Report_041609.pdf <last accessed 15 feb 2012> NERC, 2010. IVGTF Task 2.1 Report: Variable Generation Power Forecasting for Operations. At http://www.nerc.com/files/Varialbe%20Generationn%20Power%20Forecasting%20for%20Operations.pdf <last accessed 15 feb 2012> Jennifer Rodgers and Kevin Porter, 2009. Central Wind Power Forecasting Programs in North America by Regional Transmission Organizations and Electric Utilities, NREL/SR-550-46763. Available at http://www.nrel.gov/docs/fy10osti/46763.pdf <last accessed 15 feb 2012>

SLIDE 16

References (Continued)

National Grid, 2009. Operating the Electricity Transmission Networks in 2020: Initial Consultation. At http://www.nationalgrid.com/NR/rdonlyres/32879A26-D6F2-4D82-9441-40FB2B0E2E0C/39517/Operatingin2020Consulation1.pdf <last accessed 15 feb 2012> North American Electric Reliability Corporation, 2009b. Accommodating High Levels of Variable Generation, April. At http://www.nerc.com/files/IVGTF_Report_041609.pdf <last accessed 15 feb 2012> NERC, 2010. IVGTF Task 2.1 Report: Variable Generation Power Forecasting for Operations. At http://www.nerc.com/files/Varialbe%20Generationn%20Power%20Forecasting%20for%20Operations.pdf <last accessed 15 feb 2012> Jennifer Rodgers and Kevin Porter, 2009. Central Wind Power Forecasting Programs in North America by Regional Transmission Organizations and Electric Utilities. Royal Academy of Engineering, 2014, Wind Energy : Implications of Large-Scale Deployment on the GB Electricity System http://www.raeng.org.uk/publications/reports/wind-energy-implications-of-large-scale-deployment