Mathematical Programming

, Volume 152, Issue 1–2, pp 275–300 | Cite as

Evaluating policies in risk-averse multi-stage stochastic programming

  • Václav Kozmík
  • David P. Morton
Full Length Paper Series A


We consider a risk-averse multi-stage stochastic program using conditional value at risk as the risk measure. The underlying random process is assumed to be stage-wise independent, and a stochastic dual dynamic programming (SDDP) algorithm is applied. We discuss the poor performance of the standard upper bound estimator in the risk-averse setting and propose a new approach based on importance sampling, which yields improved upper bound estimators. Modest additional computational effort is required to use our new estimators. Our procedures allow for significant improvement in terms of controlling solution quality in SDDP-style algorithms in the risk-averse setting. We give computational results for multi-stage asset allocation using a log-normal distribution for the asset returns.


Multi-stage stochastic programming Stochastic dual dynamic programming Importance sampling Risk-averse optimization 

Mathematics Subject Classification (2010)

90C15 49M27 



We thank two anonymous referees and an Associate Editor for comments that improved the paper. The research was partly supported by the Czech Science Foundation through grant 402/12/G097, by the Defense Threat Reduction Agency through Grant HDTRA1-08-1-0029, and by the National Science Foundation through Grant ECCS-1162328.


  1. 1.
    Armadillo C++ linear algebra library.
  2. 2.
    Artzner, P., Delbaen, F., Eber, J.-M., Heath, D.: Coherent measures of risk. Math. Financ. 9, 203–228 (1999)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Bayraksan, G., Morton, D.P.: Assessing solution quality in stochastic programs via sampling. In: Oskoorouchi, M., Gray, P., Greenberg, H. (eds.) Tutorials in Operations Research. pp. 102–122. INFORMS, Hanover (2009). ISBN: 978-1-877640-24-7Google Scholar
  4. 4.
    Bayraksan, G., Morton, D.P.: A sequential sampling procedure for stochastic programming. Oper. Res. 59, 898–913 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Birge, J.R.: Decomposition and partitioning methods for multistage stochastic linear programs. Oper. Res. 33, 989–1007 (1985)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Blum, M., Floyd, R. W., Pratt, V., Rivest, R. L., Tarjan, R. E.: Linear time bounds for median computations. In: Proceedings of the Fourth Annual ACM Symposium on Theory of Computing, pp 119–124 (1972)Google Scholar
  7. 7.
    Chen, Z.L., Powell, W.B.: Convergent cutting-plane and partial-sampling algorithm for multistage stochastic linear programs with recourse. J. Optim. Theory Appl. 102, 497–524 (1999)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Chiralaksanakul, A., Morton, D.P.: Assessing policy quality in multi-stage stochastic programming. Stochastic Programming E-Print Series, vol. 12 (2004). Available at:
  9. 9.
    de Queiroz, A.R., Morton, D.P.: Sharing cuts under aggregated forecasts when decomposing multi-stage stochastic programs. Oper. Res. Lett. 41, 311–316 (2013)Google Scholar
  10. 10.
    Donohue, C.J., Birge, J.R.: The abridged nested decomposition method for multistage stochastic linear programs with relatively complete recourse. Algorithm. Oper. Res. 1, 20–30 (2006)MathSciNetzbMATHGoogle Scholar
  11. 11.
    Eichhorn, A., Römisch, W.: Polyhedral risk measures in stochastic programming. SIAM J. Optim. 16, 69–95 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Goor, Q., Kelman, R., Tilmant, A.: Optimal multipurpose-multireservoir operation model with variable productivity of hydropower plants. J. Water Resour. Plan. Manag. 137, 258–267 (2011)CrossRefzbMATHGoogle Scholar
  13. 13.
    Guigues, V.: SDDP for some interstage dependent risk averse problems and application to hydro-thermal planning. Comput. Optim. Appl. 57, 167–203 (2014)Google Scholar
  14. 14.
    Guigues, V., Römisch, W.: Sampling-based decomposition methods for multistage stochastic programs based on extended polyhedral risk measures. SIAM J. Optim. 22, 286–312 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Hesterberg, T.C.: Weighted average importance sampling and defensive mixture distributions. Technometrics 37, 185–194 (1995)Google Scholar
  16. 16.
    Higle, J.L., Rayco, B., Sen, S.: Stochastic scenario decomposition for multistage stochastic programs. IMA J. Manag. Math 21, 39–66 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Homem-de-Mello, T., de Matos, V.L., Finardi, E.C.: Sampling strategies and stopping criteria for stochastic dual dynamic programming: a case study in long-term hydrothermal scheduling. Energy Syst. 2, 1–31 (2011)CrossRefGoogle Scholar
  18. 18.
  19. 19.
    Infanger, G., Morton, D.P.: Cut sharing for multistage stochastic linear programs with interstage dependency. Math. Prog. 75, 241–256 (1996)MathSciNetzbMATHGoogle Scholar
  20. 20.
    Knopp, R.: Remark on algorithm 334 [G5]: normal random deviates. Commun. ACM 12, 281 (1966)CrossRefGoogle Scholar
  21. 21.
    Krokhmal, P., Zabarankin, M., Uryasev, S.: Modeling and optimization of risk. Surv. Oper. Res. Manag. Sci. 16, 49–66 (2011)Google Scholar
  22. 22.
    L’Ecuyer random streams generator.
  23. 23.
    Linowsky, K., Philpott, A.B.: On the convergence of sampling-based decomposition algorithms for multi-stage stochastic programs. J. Optim. Theory Appl. 125, 349–366 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Löhndorf, N., Wozabal, D., Minner, S.: Optimizing trading decisions for hydro storage systems using approximate dual dynamic programming. Oper. Res. 61, 810–823 (2013)Google Scholar
  25. 25.
    Mo, B., Gjelsvik, A., Grundt, A.: Integrated risk management of hydro power scheduling and contract management. IEEE Trans. Power Syst. 16, 216–221 (2001)CrossRefGoogle Scholar
  26. 26.
    Pereira, M.V.F., Pinto, L.M.V.G.: Multi-stage stochastic optimization applied to energy planning. Math. Prog. 52, 359–375 (1991)MathSciNetCrossRefGoogle Scholar
  27. 27.
    Pflug, GCh., Römisch, W.: Modeling, Measuring and Managing Risk. World Scientific Publishing, Singapore (2007)CrossRefGoogle Scholar
  28. 28.
    Philpott, A.B., Guan, Z.: On the convergence of sampling-based methods for multi-stage stochastic linear programs. Oper. Res. Lett. 36, 450–455 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Philpott, A.B., de Matos, V.L.: Dynamic sampling algorithms for multi-stage stochastic programs with risk aversion. Eur. J. Oper. Res. 218, 470–483 (2012)CrossRefzbMATHGoogle Scholar
  30. 30.
    Philpott, A.B., de Matos, V.L., Finardi, E.C.: On solving multistage stochastic programs with coherent risk measures. Oper. Res. 61, 957–970 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  31. 31.
    Philpott, A.B., Dallagi, A., Gallet, E.: On cutting plane algorithms and dynamic programming for hydroelectricity generation. In: Pflug, GCh., Kovacevic, R.M., Vespucci, M.T. (eds.) Handbook of Risk Management in Energy Production and Trading. Springer, UK (2013)Google Scholar
  32. 32.
    Rebennack, S., Flach, B., Pereira, M.V.F., Pardalos, P.M.: Stochastic hydro-thermal scheduling under \(\text{ CO }_{2}\) emissions constraints. IEEE Trans. Power Syst. 27, 58–68 (2012)CrossRefGoogle Scholar
  33. 33.
    Rockafellar, R.T., Wets, R.J.-B.: Scenarios and policy aggregation in optimization under uncertainty. Math. Oper. Res. 16, 119–147 (1991)MathSciNetCrossRefzbMATHGoogle Scholar
  34. 34.
    Rudloff, B, Street, A., Valladao, D.: Time consistency and risk averse dynamic decision models: definition, interpretation and practical consequences. Available at:
  35. 35.
    Ruszczynski, A., Shapiro, A.: Conditional risk mappings. Math. Oper. Res. 31, 544–561 (2006)MathSciNetCrossRefGoogle Scholar
  36. 36.
    Ruszczynski, A.: Risk-averse dynamic programming for Markov decision processes. Math. Prog. 125, 235–261 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  37. 37.
    Sen, S., Zhou, Z.: Multistage stochastic decomposition: a bridge between stochastic programming and approximate dynamic programming. SIAM J. Optim. 24, 127–153 (2014)Google Scholar
  38. 38.
    Shapiro, A., Dentcheva, D., Ruszczynski, A.: Lectures on Stochastic Programming: Modeling and Theory. SIAM-Society for Industrial and Applied Mathematics (2009)Google Scholar
  39. 39.
    Shapiro, A.: Inference of statistical bounds for multistage stochastic programming problems. Math. Methods Oper. Res. 58, 57–68 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  40. 40.
    Shapiro, A.: On a time consistency concept in risk averse multistage stochastic programming. Oper. Res. Lett. 37, 143–147 (2009)MathSciNetCrossRefGoogle Scholar
  41. 41.
    Shapiro, A.: Analysis of stochastic dual dynamic programming method. Eur. J. Oper. Res. 209, 63–72 (2011)CrossRefzbMATHGoogle Scholar
  42. 42.
    Shapiro, A., Tekaya, W., da Costa, J.P., Soares, M.P.: Risk neutral and risk averse stochastic dual dynamic programming method. Eur. J. Oper. Res. 224, 375–391 (2013)CrossRefzbMATHGoogle Scholar
  43. 43.
    West, M., Harrison, J.: Bayesian Forecasting and Dynamic Models, 2nd edn. Springer, New York (1997)zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg and Mathematical Optimization Society 2014

Authors and Affiliations

  1. 1.Department of Probability and Mathematical StatisticsCharles University in Prague, Faculty of Mathematics and PhysicsPragueCzech Republic
  2. 2.Graduate Program in Operations Research and Industrial EngineeringThe University of Texas at AustinAustinUSA

Personalised recommendations