Short-Range Forecasting for Energy
Short-range forecasts for periods on the order of hours to days and up to two weeks ahead are necessary to smoothly run transmission and distribution systems, plan maintenance, protect infrastructure and allocate units. In particular, forecasting the renewable energy resources on a day-to-day basis enables integration of increasing capacities of these variable resources. This chapter describes the basics of this short-range forecasting, beginning with the observation-based “nowcasting” of the first 15 minutes and ranging up to two weeks using numerical weather prediction. We discuss how blending multiple forecasts can increase accuracy and how probabilistic forecasts are being used to quantify the forecast uncertainty.
KeywordsWind power forecasts Solar power forecasts Renewable energy Nowcasting Numerical weather prediction DICast Forecast blending Analog ensemble Probabilistic forecasts
The Need for Short-Range Forecasts
Utilities and Independent Transmission Operators (ITOs) depend on accurate forecasts for the next hours to several weeks. They often need to plan operation and maintenance outages weeks to months ahead. Weather events may make it difficult to maintain power lines, wind turbines and other infrastructure. If they can plan around events such as lightning storms and high wind events, the safety of the workers and efficiency of the maintenance can be greatly improved.
Weather also impacts day ahead planning of how to commit units. Although weather impacts all types of power production, it actually also drives the renewable units. It is important to be able to forecast the wind, solar and hydro power available the next day, or often, over the next several days too. The marginal cost to run these renewable resources is quite low and it is economically advantageous to allocate as much power from those units as possible. But overallocation of those units when the wind, irradiance or water power is not available could lead to using much more expensive reserve units in real time. Thus it is critical to produce high quality wind and solar power forecasts.
For timescales less than a day, it is important to know the most recent update to the forecast in order to balance the load in close to real time. Sharp up or down ramps in renewable energy must be balanced with the reserve units if the grid operators were not expecting them.
Overview of Scales
The shortest time frames of the first several hours benefit from forecasting based on data sensors that determine the current situation of the variable of interest. For instance, having in situ measurements of wind speed or solar radiation in the field allow us to train statistical learning or artificial intelligence models to recognize current conditions and likely changes. Over the first 15–45 minutes, it is often difficult to beat a persistence forecast. If we know the wind now, the best forecast in the first few minutes is that there is no change. For solar radiation, we often use a “smart persistence” where we expect the cloud cover to stay the same, but recognize that the solar angle will change. For solar energy, sky imaging traces cloud conditions in real time, which allows us to use motion vectors derived from image processing succeeding images, or if there is a co-located wind profiler, one can project the cloud motion for the next few steps (Kleissl 2010; Chu et al. 2013; Nguyen and Kleissl 2014; Peng et al. 2015). Cloud imager-based forecasts are effective for about 15–30 minutes.
Statistical learning methods have been shown to be effective from about 30 minutes to around three hours. Such methods may consist of using techniques such as artificial neural networks (ANN—Mellit 2008; Wang et al. 2012), autoregressive models (Hassanzadeh et al. 2010; Yang et al. 2012), Markov process models (Morf 2014) or support vector machines (Sharma et al. 2011; Bouzerdoum et al. 2013) to recognize patterns in the changes of wind speed or solar radiation. Blended techniques, such as that of Pedro and Coimbra (2012), that use a genetic algorithm to optimize an ANN have also been effective. In addition, new methods blend weather observations with irradiance observations and use clustering techniques to identify regimes and then train ANN for the individual regimes (Kazor and Hering 2015), which was shown to improve upon non-regime-dependent ANN and upon smart persistence (McCandless et al. 2016a). Recent work has added satellite data to this type of forecasting (McCandless et al. 2016b). Some methods also predict the variability of the resource (McCandless et al. 2015).
In the time range from about an hour out to six hours, cloud motion vectors derived from satellite data are often used to forecast solar irradiance (Miller et al. 2013). When a satellite observes the cloud cover and the forecast system then advects it using motion vectors derived from successive images or uses observed or modeled data to advect those clouds, the derived liquid water path can be used to provide good estimates of irradiance attenuation. Satellite-based methods depend on the satellite data being received and processed before being used in the motion vector models, which take on the order of 30 minutes after observation. They also do not account for cloud development or dissipation, so the forecasts of individual clouds are only accurate for a limited period of time, beginning to degrade after the first couple hours.
Wind energy forecasting at these scales of the first few hours can be improved by other methods of remote sensing. Mahoney et al. (2012) describe the Variational Doppler Radar Analysis System (VDRAS) that assimilates radar data into a cloud-resolving model to better predict winds. Because that model does not include the full physics, it can be updated at frequencies as high as every 15 minutes. That work showed that in case studies, the winds could be well predicted for the first two hours and could identify weather ramps as they approach. This method, however, relies on having radars sited in close proximity to the wind farms.
Numerical Weather Prediction
Beyond about three to six hours, the workhorse of forecasting is NWP. NWP consists of the integration of the nonlinear partial differential equations governing atmospheric flow and includes appropriate models for the physics of clouds, radiation, turbulence, land surface conditions and more (Warner 2011). As computer power has advanced, so has our capability to provide higher resolution simulations in closer to real time. The national centers now run very short-range simulations at about three kilometers horizontal resolution over limited regions (such as over the USA) as often as hourly. Global models have necessarily coarser resolution and run less frequently. As of mid-2017, the US Global Forecast System is run every six hours at 13 kilometers resolution with hourly output for the first five days and at 70 kilometers out to 16 days. The European Center for Medium Forecast is run twice a day at nine kilometers resolution for 16 days with output at three hours temporal resolution. The national centers are continually updating their model resolutions, lengths of simulations and frequency of the runs as computer power is upgraded. In addition, they are including ensemble runs, which provide probabilistic information as well as improving upon the deterministic forecast. As mentioned in the introduction and elaborated later, the sensitivity to initial conditions is what partly limits predictability, which necessitates running ensembles of models.
It is important to assimilate observations of weather data from the global networks to provide the best possible initial condition to the runs. To improve forecasts at specific points, such as at a wind farm, it is advantageous to also assimilate specialized data (such as wind speed measurements) at that farm. Mahoney et al. (2012) and Wilczak et al. (2015) provide evidence that assimilating local wind farm data can improve the NWP forecasts. In a case study, Cheng et al. (2017) show that real-time four-dimensional data assimilation can reduce the mean absolute error in the forecast by 30–40% in the first three hours. Kosovic et al. (2017) indicate that such local data assimilation can significantly improve forecasts over the national models for the first 15 hours. Versions of the models are now being built to specifically improve upon predicting variables of particular importance to energy, such as wind (Wilczak et al. 2015), solar (Jimenez et al. 2016) and hydro (Gochis et al. 2014), although the use of national hydrological models is still in its infancy.
Blending the Forecasts and Predicting Power
Probabilistic Forecasts and the Analog Ensemble
Utilities and ITOs are also requesting probabilistic information to plan reserves, alleviating transmission bottlenecks, and better planning for renewable operations. To this end, the meteorology sector has been providing probabilistic forecasts. All major national forecasting centers now run ensemble forecasts. The rate of spread of the members of the ensemble quantifies the uncertainty. Various postprocessing methods are used to remove bias, sharpen the probability density function and calibrate the spread, or reliability, of the forecast.
Another interesting postprocessing method is the analog ensemble (AnEn) technique. In this case, however, rather than running multiple models, the historical output from a single, often high resolution, model is used to generate the ensemble. A search of the historical forecast records is made to identify the forecasts that are most similar to the current forecast. We then compare the forecasts to the corresponding observations. Those observations then become the analog ensemble. This method is effective at both improving on the deterministic forecast and using the multiple analogs to form an ensemble that can be used to quantify the uncertainty of the forecast (Delle Monache et al. 2013; Haupt and Delle Monache 2014; Alessandrini et al. 2015).
Accuracy of forecasts has been improving steadily with some areas now seeing single digit errors in terms of percentage of capacity at a wind farm (Orwig et al. 2014; Haupt and Mahoney 2015). These improvements have stemmed from including observations in the immediate vicinity of the resource, both in the nowcasting and assimilated into the NWP models, as well as better methods of blending multiple models for the appropriate timescales. Solar power predictions have not been a focus for very long, but rapid improvement is also happening here (Lorenz et al. 2009; 2014; Tuohy et al. 2015; Haupt et al. 2017).
- Cheng, W. Y. Y., Liu, Y., Bourgeois, A., Wu, Y., & Haupt, S. E. (2017). Short-term wind forecast of a data assimilation/weather forecasting system with wind turbine anemometer measurement assimilation. Renewable Energy, 107, 340–351. https://doi.org/10.1016/j.renene.2017.02.014.CrossRefGoogle Scholar
- Giebel, G., & Kariniotakis, G. (2007). Best practice in short-term forecasting – A users guide. European wind energy conference and exhibition. Milan (IT), 7–10 May 2007. Retrieved from http://powwow.risoe.dk/publ.htm
- Gochis, D. J., Yu, W., & Yates, D. N. (2014). The WRF-Hydro model technical description and user’s guide, version 2.0. NCAR Technical Document. 120 pages. Retrieved from http://www.ral.ucar.edu/projects/wrf_hydro/
- Hassanzadeh, M., Etezadi-Amoli, M., & Fadali, M. S. (2010). Practical approach for sub-hourly and hourly prediction of PV power output. North American Power Symposium (NAPS), 1–5, September 26–28.Google Scholar
- Haupt, S. E., & Delle Monache, L. (2014). Understanding ensemble prediction: How probabilistic wind power prediction can help in optimizing operations. WindTech, 10(6), 27–29.Google Scholar
- Haupt, S. E., & Mahoney, W. P. (2015). Wind power forecasting. IEEE Spectrum, 2015, 46–52.Google Scholar
- Haupt, S. E., Kosović, B., Jensen, T., Lazo, J. K., Lee, J. A., Jiménez, P. A., et al. (2017). Building the Sun4Cast system: Improvements in solar power forecasting. Bulletin of the American Meteorological Society. https://doi.org/10.1175/BAMS-D-16-0221.1.
- Jimenez, P. A., Hacker, J. P., Dudhia, J., Haupt, S. E., Ruiz-Arias, J. A., Gueymard, C. A., et al. (2016). WRF-solar: An augmented NWP model for solar power prediction. Bulletin of the American Meteorological Society, 1249–1264. https://doi.org/10.1175/BAMS-D-14-00279.1.
- Kleissl, J. (2010). Current state of the art in solar forecasting. Final Report California Renewable Energy Forecasting, Resource Data and Mapping Appendix A. California Renewable Energy Collaborative.Google Scholar
- Kosovic, B., Haupt, S. E., Adriaansen, D., Alessandrini, S., Jensen, T., Wiener, G., et al. (2017). Scientific advances in wind power forecasting. Submitted to Wind Energy.Google Scholar
- Lorenz, E., Kihnert, I., & Heinemann, D. (2014). Overview of irradiance and photovoltaic power prediction. In A. Troccoli, L. Dubus, & S. E. Haupt (Eds.), Weather matters for energy (pp. 420–454). New York: Springer.Google Scholar
- Mahoney, W. P., Parks, K., Wiener, G., Liu, Y., Myers, B., Sun, J., et al. (2012). A wind power forecasting system to optimize grid integration. Special issue of IEEE Transactions on Sustainable Energy on Applications of Wind Energy to Power Systems, 3 (4), 670–682.Google Scholar
- McCandless, T. C., Haupt, S. E., & Young, G. S. (2015). A model tree approach to forecasting solar irradiance variability. Solar Energy, 120, 514–524. https://doi.org/10.1016/j.solener.2015.07.0200038-092X. CrossRefGoogle Scholar
- McCandless, T. C., Haupt, S. E., & Young, G. S. (2016a). A regime-dependent artificial neural network technique for short-range solar irradiance forecasting. Applied Energy, 89, 351–359.Google Scholar
- Miller, S. D., Heidinger, A. K., & Sengupta, M. (2013). Chapter 3: Physically based satellite methods. In J. Kleissl (Ed.), Solar energy forecasting (504 pp.). Oxford: Elsevier. ISBN:9780123971777.Google Scholar
- Monteiro, C., Bessa, R., Miranda, V., Botterud, A., Wang, J., & Conzelmann, G.. (2009). Wind power forecasting: State-of-the-Art 2009. Argonne National Laboratory, Argonne, IL, ANL/DIS-10-1, November 2009.Google Scholar
- Myers, W., & Linden, S. (2011). A turbine hub height wind speed consensus forecasting system. AMS Ninth Conference on Artificial Intelligence and its Applications to the Environmental Sciences, Seattle, WA, January 23–27, 2011.Google Scholar
- Myers, W., Wiener, G., Linden, S., & Haupt, S. E. (2011). A consensus forecasting approach for improved turbine hub height wind speed predictions. Proceedings of WindPower 2011.Google Scholar
- Orwig, K. D., Ahlstrom, M., Banunarayanan, V., Sharp, J., Wilczak, J. M., Freedman, J., et al. (2014). Recent trends in variable generation forecasting and its value to the power system. IEEE Transactions on Renewable Energy, 6(3), 924–933.Google Scholar
- Pelland, S., Remund, J., Kleissl, J., Oozeki, T., & De Brabandere, K.. (2013). Photovoltaic and solar forecasting: State of the art. IEA PVPS Task 14, Subtask 3.1 Report IEA-PVPS T14-01: 2013.Google Scholar
- Sharma, N., Sharma, P., Irwin, D., & Shenoy, P. (2011). Predicting solar generation from weather forecasts using machine learning. Proceedings of the 2nd IEEE International Conference on Smart Grid Communications, Brussels, 17–20 October, pp. 32–37.Google Scholar
- Warner, T. T. (2011). Numerical weather and climate prediction. Cambridge: Cambridge University Press. 526 pp.Google Scholar
Open Access This chapter is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, duplication, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, a link is provided to the Creative Commons license and any changes made are indicated.
The images or other third party material in this chapter are included in the work’s Creative Commons license, unless indicated otherwise in the credit line; if such material is not included in the work’s Creative Commons license and the respective action is not permitted by statutory regulation, users will need to obtain permission from the license holder to duplicate, adapt or reproduce the material.