Observational data-based quality assessment of scenario generation for stochastic programs

  • Didem Sarı Ay
  • Sarah M. RyanEmail author
Original Paper


In minimization problems with uncertain parameters, cost savings can be achieved by solving stochastic programming (SP) formulations instead of using expected parameter values in a deterministic formulation. To obtain such savings, it is crucial to employ scenarios of high quality. An appealing way to assess the quality of scenarios produced by a given method is to conduct a re-enactment of historical instances in which the scenarios produced are used when solving the SP problem and the costs are assessed under the observed values of the uncertain parameters. Such studies are computationally very demanding. We propose two approaches for assessment of scenario generation methods using past instances that do not require solving SP instances. Instead of comparing scenarios to observations directly, these approaches consider the impact of each scenario in the SP problem. The methods are tested in simulation studies of server location and unit commitment, and then demonstrated in a case study of unit commitment with uncertain variable renewable energy generation.


Stochastic programming Scenario generation method assessment Scenario quality 



  1. Ahmed S, Garcia R, Kong N, Ntaimo L, Parija G, Qiu F, Sen S (2015) SIPLIB: a stochastic integer programming test problem library. Accessed 10 Jan 2019
  2. Bakirtzis EA, Biskas PN, Labridis DP, Bakirtzis AG (2014) Multiple time resolution unit commitment for short-term operations scheduling under high renewable penetration. IEEE Trans Power Syst 29:149–159CrossRefGoogle Scholar
  3. Bayraksan G, Morton DP (2006) Assessing solution quality in stochastic programs. Math Program 108:495–514. CrossRefGoogle Scholar
  4. Birge JR, Louveaux F (1997) Introduction to stochastic programming. Springer series in operations research. Springer, New YorkGoogle Scholar
  5. Bruninx K, Delarue E, D‘haeseleer W (2014) A practical approach on scenario generation and reduction algorithms for wind power forecast error scenarios. Accessed 8 Mar 2019
  6. Bruninx K, Bergh KVd, Delarue E, D‘haeseleer W (2016a) Optimization and allocation of spinning reserves in a low-carbon framework. IEEE Trans Power Syst 31:872–882. CrossRefGoogle Scholar
  7. Bruninx K, Dvorkin Y, Delarue E, Pandžić H, D’haeseleer W, Kirschen DS (2016b) Coupling pumped hydro energy storage with unit commitment. IEEE Trans Sustain Energy 7:786–796. CrossRefGoogle Scholar
  8. Du E, Zhang N, Kang C, Xia Q (2018) Scenario map-based stochastic unit commitment. IEEE Trans Power Syst PP:1. Google Scholar
  9. Dupacova J, Gröwe-Kuska N, Römisch W (2003) Scenario reduction in stochastic programming: an approach using probability metrics. Math Program 95:493–511. CrossRefGoogle Scholar
  10. Feng Y, Ryan SM (2016) Solution sensitivity-based scenario reduction for stochastic unit commitment. CMS 13:29–62. CrossRefGoogle Scholar
  11. Feng YH, Rios I, Ryan SM, Spurkel K, Watson JP, Wets RJB, Woodruff DL (2015) Toward scalable stochastic unit commitment. Part 1: load scenario generation. Energy Syst 6:309–329. CrossRefGoogle Scholar
  12. Heitsch H, Römisch W (2003) Scenario reduction algorithms in stochastic programming. Comput Optim Appl 24:187–206. CrossRefGoogle Scholar
  13. Heitsch H, Römisch W (2007) A note on scenario reduction for two-stage stochastic programs. Oper Res Lett 35:731–738. CrossRefGoogle Scholar
  14. Kaut M, Wallace SW (2007) Evaluation of scenario-generation methods for stochastic programming. Pac J Optim 3:257–271Google Scholar
  15. Lunn AD, Davies SJ (1998) A note on generating correlated binary variables. Biometrika 85:487–490. CrossRefGoogle Scholar
  16. Morales JM, Pineda S, Conejo AJ, Carrion M (2009) Scenario reduction for futures market trading in electricity markets. IEEE Trans Power Syst 24:878–888. CrossRefGoogle Scholar
  17. Ntaimo L, Sen S (2005) The million-variable “march” for stochastic combinatorial optimization. J Glob Optim 32:385–400. CrossRefGoogle Scholar
  18. Papavasiliou A, Oren SS (2013) Multiarea stochastic unit commitment for high wind penetration in a transmission constrained network. Oper Res 61:578–592CrossRefGoogle Scholar
  19. Pinson P, Girard R (2012) Evaluating the quality of scenarios of short-term wind power generation. Appl Energy 96:12–20. CrossRefGoogle Scholar
  20. Pinson P, Madsen H, Nielsen HA, Papaefthymiou G, Klockl B (2009) From probabilistic forecasts to statistical scenarios of short-term wind power production. Wind Energy 12:51–62CrossRefGoogle Scholar
  21. Rachev ST (1991) Probability metrics and the stability of stochastic models. Wiley series in probability and mathematical statistics. Applied probability and statistics. Wiley, ChichesterGoogle Scholar
  22. Rios I, Wets RJB, Woodruff DL (2015) Multi-period forecasting and scenario generation with limited data. CMS 12:267–295. CrossRefGoogle Scholar
  23. Sarı D, Ryan S (2016) MTDrh: mass transportation distance rank histogram. Accessed 8 Mar 2019
  24. Sarı D, Ryan S (2017) Statistical reliability of wind power scenarios and stochastic unit commitment cost. Energy Syst. Google Scholar
  25. Sarı D, Lee Y, Ryan S, Woodruff D (2016) Statistical metrics for assessing the quality of wind power scenarios for stochastic unit commitment. Wind Energy 19:873–893CrossRefGoogle Scholar
  26. Staid A, Watson J-P, Wets RJB, Woodruff DL (2017) Generating short-term probabilistic wind power scenarios via nonparametric forecast error density estimators. Wind Energy 20:1911–1925. CrossRefGoogle Scholar
  27. Takriti S, Birge JR, Long E (1996) A stochastic model for the unit commitment problem. IEEE Trans Power Syst 11:1497–1508. CrossRefGoogle Scholar
  28. Wu H, Shahidehpour M (2014) Stochastic SCUC solution with variable wind energy using constrained ordinal optimization. IEEE Trans Sustain Energy 5:379–388. CrossRefGoogle Scholar
  29. Zheng QPP, Wang JH, Liu AL (2015) Stochastic optimization for unit commitment—a review. IEEE Trans Power Syst 30:1913–1924CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Industrial EngineeringAlanya Alaaddin Keykubat UniversityAntalyaTurkey
  2. 2.Industrial and Manufacturing Systems EngineeringIowa State UniversityAmesUSA

Personalised recommendations