Short-Range Forecasting for Energy

  • Sue Ellen Haupt
Open Access


Short-range forecasts for periods on the order of hours to days and up to two weeks ahead are necessary to smoothly run transmission and distribution systems, plan maintenance, protect infrastructure and allocate units. In particular, forecasting the renewable energy resources on a day-to-day basis enables integration of increasing capacities of these variable resources. This chapter describes the basics of this short-range forecasting, beginning with the observation-based “nowcasting” of the first 15 minutes and ranging up to two weeks using numerical weather prediction. We discuss how blending multiple forecasts can increase accuracy and how probabilistic forecasts are being used to quantify the forecast uncertainty.


Wind power forecasts Solar power forecasts Renewable energy Nowcasting Numerical weather prediction DICast Forecast blending Analog ensemble Probabilistic forecasts 

The Need for Short-Range Forecasts

Utilities and Independent Transmission Operators (ITOs) depend on accurate forecasts for the next hours to several weeks. They often need to plan operation and maintenance outages weeks to months ahead. Weather events may make it difficult to maintain power lines, wind turbines and other infrastructure. If they can plan around events such as lightning storms and high wind events, the safety of the workers and efficiency of the maintenance can be greatly improved.

Weather also impacts day ahead planning of how to commit units. Although weather impacts all types of power production, it actually also drives the renewable units. It is important to be able to forecast the wind, solar and hydro power available the next day, or often, over the next several days too. The marginal cost to run these renewable resources is quite low and it is economically advantageous to allocate as much power from those units as possible. But overallocation of those units when the wind, irradiance or water power is not available could lead to using much more expensive reserve units in real time. Thus it is critical to produce high quality wind and solar power forecasts.

For timescales less than a day, it is important to know the most recent update to the forecast in order to balance the load in close to real time. Sharp up or down ramps in renewable energy must be balanced with the reserve units if the grid operators were not expecting them.

Overview of Scales

To forecast across these scales from minutes to a few weeks, meteorologists must combine forecast methods that are most appropriate to each scale. Figure 7.1 illustrates the big picture of the types of systems that may be useful for such forecasts. Figure 7.1 focuses on the short time range where the stakeholders require optimal accuracy. That figure demonstrates that observation-based nowcasting, which refers to the first few hours of the forecast, provides a much more accurate forecast in the short range, but its skill drops off rapidly. Numerical weather prediction (NWP) becomes more important at about three hours and provides value out to about two weeks. Because NWP run at high resolution over a sizable domain requires on the order of hours to run on supercomputers and often requires spin-up time, it is not available for real-time use in the shortest ranges. Modern methods of forecasting renewable energy output employing postprocessing methods to blend disparate models or ensembles are shown to greatly improve the forecast skill (Giebel and Kariniotakis 2007; Monteiro et al. 2009; Mahoney et al. 2012; Ahlstrom et al. 2013; Orwig et al. 2014; Tuohy et al. 2015). Figure 7.1 indicates that this blending can provide value beyond that of the input models.
Fig. 7.1

Blending of NWP models with observation-based nowcasting enables optimization of the short-range forecast


The shortest time frames of the first several hours benefit from forecasting based on data sensors that determine the current situation of the variable of interest. For instance, having in situ measurements of wind speed or solar radiation in the field allow us to train statistical learning or artificial intelligence models to recognize current conditions and likely changes. Over the first 15–45 minutes, it is often difficult to beat a persistence forecast. If we know the wind now, the best forecast in the first few minutes is that there is no change. For solar radiation, we often use a “smart persistence” where we expect the cloud cover to stay the same, but recognize that the solar angle will change. For solar energy, sky imaging traces cloud conditions in real time, which allows us to use motion vectors derived from image processing succeeding images, or if there is a co-located wind profiler, one can project the cloud motion for the next few steps (Kleissl 2010; Chu et al. 2013; Nguyen and Kleissl 2014; Peng et al. 2015). Cloud imager-based forecasts are effective for about 15–30 minutes.

Statistical learning methods have been shown to be effective from about 30 minutes to around three hours. Such methods may consist of using techniques such as artificial neural networks (ANN—Mellit 2008; Wang et al. 2012), autoregressive models (Hassanzadeh et al. 2010; Yang et al. 2012), Markov process models (Morf 2014) or support vector machines (Sharma et al. 2011; Bouzerdoum et al. 2013) to recognize patterns in the changes of wind speed or solar radiation. Blended techniques, such as that of Pedro and Coimbra (2012), that use a genetic algorithm to optimize an ANN have also been effective. In addition, new methods blend weather observations with irradiance observations and use clustering techniques to identify regimes and then train ANN for the individual regimes (Kazor and Hering 2015), which was shown to improve upon non-regime-dependent ANN and upon smart persistence (McCandless et al. 2016a). Recent work has added satellite data to this type of forecasting (McCandless et al. 2016b). Some methods also predict the variability of the resource (McCandless et al. 2015).

In the time range from about an hour out to six hours, cloud motion vectors derived from satellite data are often used to forecast solar irradiance (Miller et al. 2013). When a satellite observes the cloud cover and the forecast system then advects it using motion vectors derived from successive images or uses observed or modeled data to advect those clouds, the derived liquid water path can be used to provide good estimates of irradiance attenuation. Satellite-based methods depend on the satellite data being received and processed before being used in the motion vector models, which take on the order of 30 minutes after observation. They also do not account for cloud development or dissipation, so the forecasts of individual clouds are only accurate for a limited period of time, beginning to degrade after the first couple hours.

Wind energy forecasting at these scales of the first few hours can be improved by other methods of remote sensing. Mahoney et al. (2012) describe the Variational Doppler Radar Analysis System (VDRAS) that assimilates radar data into a cloud-resolving model to better predict winds. Because that model does not include the full physics, it can be updated at frequencies as high as every 15 minutes. That work showed that in case studies, the winds could be well predicted for the first two hours and could identify weather ramps as they approach. This method, however, relies on having radars sited in close proximity to the wind farms.

Numerical Weather Prediction

Beyond about three to six hours, the workhorse of forecasting is NWP. NWP consists of the integration of the nonlinear partial differential equations governing atmospheric flow and includes appropriate models for the physics of clouds, radiation, turbulence, land surface conditions and more (Warner 2011). As computer power has advanced, so has our capability to provide higher resolution simulations in closer to real time. The national centers now run very short-range simulations at about three kilometers horizontal resolution over limited regions (such as over the USA) as often as hourly. Global models have necessarily coarser resolution and run less frequently. As of mid-2017, the US Global Forecast System is run every six hours at 13 kilometers resolution with hourly output for the first five days and at 70 kilometers out to 16 days. The European Center for Medium Forecast is run twice a day at nine kilometers resolution for 16 days with output at three hours temporal resolution. The national centers are continually updating their model resolutions, lengths of simulations and frequency of the runs as computer power is upgraded. In addition, they are including ensemble runs, which provide probabilistic information as well as improving upon the deterministic forecast. As mentioned in the introduction and elaborated later, the sensitivity to initial conditions is what partly limits predictability, which necessitates running ensembles of models.

It is important to assimilate observations of weather data from the global networks to provide the best possible initial condition to the runs. To improve forecasts at specific points, such as at a wind farm, it is advantageous to also assimilate specialized data (such as wind speed measurements) at that farm. Mahoney et al. (2012) and Wilczak et al. (2015) provide evidence that assimilating local wind farm data can improve the NWP forecasts. In a case study, Cheng et al. (2017) show that real-time four-dimensional data assimilation can reduce the mean absolute error in the forecast by 30–40% in the first three hours. Kosovic et al. (2017) indicate that such local data assimilation can significantly improve forecasts over the national models for the first 15 hours. Versions of the models are now being built to specifically improve upon predicting variables of particular importance to energy, such as wind (Wilczak et al. 2015), solar (Jimenez et al. 2016) and hydro (Gochis et al. 2014), although the use of national hydrological models is still in its infancy.

Blending the Forecasts and Predicting Power

Modern forecasting includes postprocessing the NWP output and forecast blending to improve upon the results. At a basic level, a multivariate statistical regression model known as model output statistics (MOS) is applied to remove biases (Glahn and Lowry 1972). More complex methods (such as ANN, autoregressive models and others) are also used to provide nonlinear corrections to models (Myers et al. 2011; Myers and Linden 2011; Giebel and Kariniotakis 2007; Pelland et al. 2013). Ensemble MOS (Wilks and Hamill 2007) not only corrects the individual models but also optimizes weights for blending forecasts from multiple models. It is typical to blend multiple models together using some statistical learning technique to produce a better forecast than any single model could produce consistently, often with a 10–15% improvement over the best model forecast (Mahoney et al. 2012; Myers et al. 2011). Figure 7.2 displays an example of wind speed forecasts using National Center for Atmospheric Research (NCAR’s) Dynamic Integrated foreCast (DICast®) system to blend multiple models, with the blended forecast showing a lower average root mean square error than any input model.
Fig. 7.2

Mean root mean square error for wind speed forecasts at METAR sites over the contiguous USA from multiple models and the DICast forecast (red) for the month from 5 October to 5 November, 2015

Probabilistic Forecasts and the Analog Ensemble

Utilities and ITOs are also requesting probabilistic information to plan reserves, alleviating transmission bottlenecks, and better planning for renewable operations. To this end, the meteorology sector has been providing probabilistic forecasts. All major national forecasting centers now run ensemble forecasts. The rate of spread of the members of the ensemble quantifies the uncertainty. Various postprocessing methods are used to remove bias, sharpen the probability density function and calibrate the spread, or reliability, of the forecast.

Another interesting postprocessing method is the analog ensemble (AnEn) technique. In this case, however, rather than running multiple models, the historical output from a single, often high resolution, model is used to generate the ensemble. A search of the historical forecast records is made to identify the forecasts that are most similar to the current forecast. We then compare the forecasts to the corresponding observations. Those observations then become the analog ensemble. This method is effective at both improving on the deterministic forecast and using the multiple analogs to form an ensemble that can be used to quantify the uncertainty of the forecast (Delle Monache et al. 2013; Haupt and Delle Monache 2014; Alessandrini et al. 2015).

Accuracy of forecasts has been improving steadily with some areas now seeing single digit errors in terms of percentage of capacity at a wind farm (Orwig et al. 2014; Haupt and Mahoney 2015). These improvements have stemmed from including observations in the immediate vicinity of the resource, both in the nowcasting and assimilated into the NWP models, as well as better methods of blending multiple models for the appropriate timescales. Solar power predictions have not been a focus for very long, but rapid improvement is also happening here (Lorenz et al. 2009; 2014; Tuohy et al. 2015; Haupt et al. 2017).


  1. Ahlstrom, M., Bartlett, D., Collier, C., Duchesne, J., Edelson, D., Gesino, A., et al. (2013). Knowledge is power: Efficiently integrating wind energy and wind forecasts. IEEE Power and Energy Magazine, 11(6), 45–52.CrossRefGoogle Scholar
  2. Alessandrini, S., Delle Monache, L., Sperati, S., & Cervone, G. (2015). An analog ensemble for short-term probabilistic solar power forecast. Applied Energy, 157, 95–110.CrossRefGoogle Scholar
  3. Bouzerdoum, M., Mellit, A., & Pavan, A. (2013). A hybrid model (SARIMA-SVM) for short-term power forecasting of a small-scale grid-connected photovoltaic plant. Solar Energy, 98, 226–235.CrossRefGoogle Scholar
  4. Cheng, W. Y. Y., Liu, Y., Bourgeois, A., Wu, Y., & Haupt, S. E. (2017). Short-term wind forecast of a data assimilation/weather forecasting system with wind turbine anemometer measurement assimilation. Renewable Energy, 107, 340–351. Scholar
  5. Chu, Y., Pedro, H., & Coimbra, C. (2013). Hybrid intra-hour DNI forecasts with sky image processing enhanced by stochastic learning. Solar Energy, 98, 592–603.CrossRefGoogle Scholar
  6. Delle Monache, L., Eckel, F. A., Rife, D. L., Nagarajan, B., & Searight, K. (2013). Probabilistic weather prediction with an Analog Ensemble. Monthly Weather Review, 141, 3498–3516.CrossRefGoogle Scholar
  7. Giebel, G., & Kariniotakis, G. (2007). Best practice in short-term forecasting – A users guide. European wind energy conference and exhibition. Milan (IT), 7–10 May 2007. Retrieved from
  8. Glahn, H. R., & Lowry, D. A. (1972). The use of model output statistics (MOS) in objective weather forecasting. Journal of Applied Meteorology, 11, 1203–1211.CrossRefGoogle Scholar
  9. Gochis, D. J., Yu, W., & Yates, D. N. (2014). The WRF-Hydro model technical description and user’s guide, version 2.0. NCAR Technical Document. 120 pages. Retrieved from
  10. Hassanzadeh, M., Etezadi-Amoli, M., & Fadali, M. S. (2010). Practical approach for sub-hourly and hourly prediction of PV power output. North American Power Symposium (NAPS), 1–5, September 26–28.Google Scholar
  11. Haupt, S. E., & Delle Monache, L. (2014). Understanding ensemble prediction: How probabilistic wind power prediction can help in optimizing operations. WindTech, 10(6), 27–29.Google Scholar
  12. Haupt, S. E., & Mahoney, W. P. (2015). Wind power forecasting. IEEE Spectrum, 2015, 46–52.Google Scholar
  13. Haupt, S. E., Kosović, B., Jensen, T., Lazo, J. K., Lee, J. A., Jiménez, P. A., et al. (2017). Building the Sun4Cast system: Improvements in solar power forecasting. Bulletin of the American Meteorological Society.
  14. Jimenez, P. A., Hacker, J. P., Dudhia, J., Haupt, S. E., Ruiz-Arias, J. A., Gueymard, C. A., et al. (2016). WRF-solar: An augmented NWP model for solar power prediction. Bulletin of the American Meteorological Society, 1249–1264.
  15. Kazor, K., & Hering, A. S. (2015). Assessing the performance of model-based clustering methods in multivariate time series with application to identifying regional wind regimes. Journal of Agricultural, Biological, and Environmental Statistics, 20, 192–217.CrossRefGoogle Scholar
  16. Kleissl, J. (2010). Current state of the art in solar forecasting. Final Report California Renewable Energy Forecasting, Resource Data and Mapping Appendix A. California Renewable Energy Collaborative.Google Scholar
  17. Kosovic, B., Haupt, S. E., Adriaansen, D., Alessandrini, S., Jensen, T., Wiener, G., et al. (2017). Scientific advances in wind power forecasting. Submitted to Wind Energy.Google Scholar
  18. Lorenz, E., Hurka, J., Heinemann, D., & Beyer, H. (2009). Irradiance forecasting for the power prediction of grid-connected photovoltaic systems. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2(1), 2–10.CrossRefGoogle Scholar
  19. Lorenz, E., Kihnert, I., & Heinemann, D. (2014). Overview of irradiance and photovoltaic power prediction. In A. Troccoli, L. Dubus, & S. E. Haupt (Eds.), Weather matters for energy (pp. 420–454). New York: Springer.Google Scholar
  20. Mahoney, W. P., Parks, K., Wiener, G., Liu, Y., Myers, B., Sun, J., et al. (2012). A wind power forecasting system to optimize grid integration. Special issue of IEEE Transactions on Sustainable Energy on Applications of Wind Energy to Power Systems, 3 (4), 670–682.Google Scholar
  21. McCandless, T. C., Haupt, S. E., & Young, G. S. (2015). A model tree approach to forecasting solar irradiance variability. Solar Energy, 120, 514–524. CrossRefGoogle Scholar
  22. McCandless, T. C., Haupt, S. E., & Young, G. S. (2016a). A regime-dependent artificial neural network technique for short-range solar irradiance forecasting. Applied Energy, 89, 351–359.Google Scholar
  23. McCandless, T. C., Young, G. S., Haupt, S. E., & Hinkelman, L. M. (2016b). Regime-Dependent short-range solar irradiance forecasting. Journal of Applied Meteorology and Climatology, 55, 1599–1613.CrossRefGoogle Scholar
  24. Mellit, A. (2008). Artificial intelligence technique for modeling and forecasting of solar radiation data: A review. Int. Journal Artificial Intelligence and Soft Computing, 1(1), 52–76.CrossRefGoogle Scholar
  25. Miller, S. D., Heidinger, A. K., & Sengupta, M. (2013). Chapter 3: Physically based satellite methods. In J. Kleissl (Ed.), Solar energy forecasting (504 pp.). Oxford: Elsevier. ISBN:9780123971777.Google Scholar
  26. Monteiro, C., Bessa, R., Miranda, V., Botterud, A., Wang, J., & Conzelmann, G.. (2009). Wind power forecasting: State-of-the-Art 2009. Argonne National Laboratory, Argonne, IL, ANL/DIS-10-1, November 2009.Google Scholar
  27. Morf, H. (2014). Sunshine and cloud cover prediction based on Markov processes. Solar Energy, 110, 615–626.CrossRefGoogle Scholar
  28. Myers, W., & Linden, S. (2011). A turbine hub height wind speed consensus forecasting system. AMS Ninth Conference on Artificial Intelligence and its Applications to the Environmental Sciences, Seattle, WA, January 23–27, 2011.Google Scholar
  29. Myers, W., Wiener, G., Linden, S., & Haupt, S. E. (2011). A consensus forecasting approach for improved turbine hub height wind speed predictions. Proceedings of WindPower 2011.Google Scholar
  30. Nguyen, D., & Kleissl, J. (2014). Stereographic methods for cloud base height determination using two sky imagers. Solar Energy, 107, 495–509.CrossRefGoogle Scholar
  31. Orwig, K. D., Ahlstrom, M., Banunarayanan, V., Sharp, J., Wilczak, J. M., Freedman, J., et al. (2014). Recent trends in variable generation forecasting and its value to the power system. IEEE Transactions on Renewable Energy, 6(3), 924–933.Google Scholar
  32. Pedro, H. T., & Coimbra, C. F. M. (2012). Assessment of forecasting techniques for solar power production with no exogenous inputs. Solar Energy, 86(7), 2017–2028.CrossRefGoogle Scholar
  33. Pelland, S., Remund, J., Kleissl, J., Oozeki, T., & De Brabandere, K.. (2013). Photovoltaic and solar forecasting: State of the art. IEA PVPS Task 14, Subtask 3.1 Report IEA-PVPS T14-01: 2013.Google Scholar
  34. Peng, Z., Yu, D., Huang, D., Heiser, J., Yoo, S., & Kalb, P. (2015). 3D cloud detecting and tracking system for solar forecast using multiple sky imagers. Solar Energy, 118, 496–519.CrossRefGoogle Scholar
  35. Sharma, N., Sharma, P., Irwin, D., & Shenoy, P. (2011). Predicting solar generation from weather forecasts using machine learning. Proceedings of the 2nd IEEE International Conference on Smart Grid Communications, Brussels, 17–20 October, pp. 32–37.Google Scholar
  36. Tuohy, A., Zack, J., Haupt, S., Sharp, J., Ahlstrom, M., Dise, S., et al. (2015). Solar forecasting – Method, challenges, and performance. IEEE Power and Energy Magazine, 13(6), 50–59. Scholar
  37. Wang, F., Mi, Z., Su, S., & Zhao, H. (2012). Short-term solar irradiance forecasting model based on artificial neural network using statistical feature parameters. Energies, 5, 1355–1370.CrossRefGoogle Scholar
  38. Warner, T. T. (2011). Numerical weather and climate prediction. Cambridge: Cambridge University Press. 526 pp.Google Scholar
  39. Wilczak, J., et al. (2015). The Wind Forecast Improvement Project (WFIP): A Public-Private Partnership addressing wind energy forecast needs. Bulletin of the American Meteorological Society, 96, 1699–1718.CrossRefGoogle Scholar
  40. Wilks, D. S., & Hamill, T. M. (2007). Comparison of ensemble-MOS methods using GFS reforecasts. Monthly Weather Review, 135, 2379–2390.CrossRefGoogle Scholar
  41. Yang, D., Jirutitijaroen, P., & Walsh, W. M. (2012). Hourly solar irradiance time series forecasting using cloud cover index. Solar Energy, 86, 3531–3543.CrossRefGoogle Scholar

Copyright information

© The Author(s) 2018

Open Access This chapter is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits use, duplication, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, a link is provided to the Creative Commons license and any changes made are indicated.

The images or other third party material in this chapter are included in the work’s Creative Commons license, unless indicated otherwise in the credit line; if such material is not included in the work’s Creative Commons license and the respective action is not permitted by statutory regulation, users will need to obtain permission from the license holder to duplicate, adapt or reproduce the material.

Authors and Affiliations

  • Sue Ellen Haupt
    • 1
  1. 1.National Center for Atmospheric Research (NCAR)BoulderUSA

Personalised recommendations