Skip to main content
SpringerLink
Log in

Scenario generation for stochastic optimization problems via the sparse grid method

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

We study the use of sparse grids in the scenario generation (or discretization) problem in stochastic programming problems where the uncertainty is modeled using a continuous multivariate distribution. We show that, under a regularity assumption on the random function involved, the sequence of optimal objective function values of the sparse grid approximations converges to the true optimal objective function values as the number of scenarios increases. The rate of convergence is also established. We treat separately the special case when the underlying distribution is an affine transform of a product of univariate distributions, and show how the sparse grid method can be adapted to the distribution by the use of quadrature formulas tailored to the distribution. We numerically compare the performance of the sparse grid method using different quadrature rules with classic quasi-Monte Carlo (QMC) methods, optimal rank-one lattice rules, and Monte Carlo (MC) scenario generation, using a series of utility maximization problems with up to 160 random variables. The results show that the sparse grid method is very efficient, especially if the integrand is sufficiently smooth. In such problems the sparse grid scenario generation method is found to need several orders of magnitude fewer scenarios than MC and QMC scenario generation to achieve the same accuracy. It is indicated that the method scales well with the dimension of the distribution—especially when the underlying distribution is an affine transform of a product of univariate distributions, in which case the method appears scalable to thousands of random variables.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Bungartz, H.J., Griebel, M.: Sparse grids. Acta Numer. 13, 147–269 (2004). doi:10.1017/S0962492904000182

    Article  MathSciNet  Google Scholar 

  2. Casey, M., Sen, S.: The scenario generation algorithm for multistage stochastic linear programming. Math. Oper. Res. 30, 615–631 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  3. Chen, M., Mehrotra, S.: Epi-convergent scenario generation method for stochastic problems via sparse grid. Tech. rep., Northwestern University (2007). http://edoc.hu-berlin.de/series/speps/2008-7/PDF/7.pdf

  4. Consigli, G., Dupačová, J., Wallace, S.: Generating scenarios for multistage stochastic programs. Ann. Oper. Res. 100, 25–53 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  5. Davis, P.J., Rabinowitz, P.: Methods of Numerical Integration. Academic Press, San Diego (1975)

    MATH  Google Scholar 

  6. Dempster, M.A.H., Thompson, R.T.: EVPI-based importance sampling solution procedure for multistage stochastic linear programming on parallel MIMD architectures. Ann. Oper. Res. 90, 161–184 (1999)

    Article  MATH  MathSciNet  Google Scholar 

  7. Dick, J., Pillichshammer, F.: Digital Nets and Sequences. Cambridge University Press, Cambridge (2010)

    Book  MATH  Google Scholar 

  8. Donohue, C.: Stochastic network programming and the dynamic vehicle allocation problem. Ph.D. thesis, The University of Michigan, Ann Arbor (1996)

  9. Donohue, C.J., Birge, J.R.: The abridged nested decomposition method for multistage stochastic linear programs with relatively complete recourse. Algorithmic Oper.Res. 1, 20–30 (2006)

    MATH  MathSciNet  Google Scholar 

  10. Dupačová, J., Gröwe-Kuska, N., Römisch, W.: Scenario reduction in stochastic programming: an approach using probability metrics. Math. Program. 95, 493–511 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  11. Evans, L.C.: Partial Differential Equations. American Mathematical Society, Providence (1998)

    MATH  Google Scholar 

  12. Folland, G.B.: Real Analysis: Modern Techniques and Their Applications, 2nd edn. Wiley, Hoboken (1999)

    MATH  Google Scholar 

  13. Genz, A., Keister, B.: Fully symmetric interpolatory rules for multiple integrals over infinite regions with gaussian weight. J. Comput. Appl. Math. 71(2), 299–309 (1996). doi:10.1016/0377-0427(95)00232-4

    Article  MATH  MathSciNet  Google Scholar 

  14. Gerstner, T., Griebel, M.: Numerical integration using sparse grids. Numer. Algorithms 18, 209–232 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  15. Heiss, F., Winschel, V.: Likelihood approximation by numerical integration on sparse grids. J. Econom. 144(1), 62–80 (2008). doi:10.1016/j.jeconom.2007.12.004

    Article  MathSciNet  Google Scholar 

  16. Heitsch, H., Römisch, W.: Scenario tree modeling for multistage stochastic programs. Math. Program. 118(2), 371–406 (2007). doi:10.1007/s10107-007-0197-2

    Article  Google Scholar 

  17. Hinrichs, A., Novak, E., Ullrich, M., Wozniakowski, H.: The curse of dimensionality for numerical integration of smooth functions. Math. Comput. 83, 2853–2863 (2014)

    Article  MATH  MathSciNet  Google Scholar 

  18. Kaut, M., Wallace, S.W.: Evaluation of scenario-generation methods for stochastic programming. Pac. J. Optim. 2, 257–271 (2007)

    MathSciNet  Google Scholar 

  19. King, A.J., Wets, R.J.B.: Epi-consistency of convex stochastic programs. Stoch. Stoch. Rep. 34, 83–92 (1991)

    Article  MATH  MathSciNet  Google Scholar 

  20. Krylov, V.I.: Approximate Calculation of Integrals. Dover, Mineola (2005)

    MATH  Google Scholar 

  21. Kuo, F.: Component-by-component constructions achieve the optimal rate of convergence for multivariate integration in weighted Korobov and Sobolev spaces. J. Complex. 19(3), 301–320 (2003). doi:10.1016/S0885-064X(03)00006-2. http://www.sciencedirect.com/science/article/pii/S0885064X03000062

  22. L’Ecuyer, P., Munger, D.: LatticeBuilder: A general software tool for constructing rank-1 lattice rules. Submitted (2012). Software downloadable from http://www.iro.umontreal.ca/simardr/latbuilder/latbuilder.html

  23. L’Ecuyer, P., Munger, D.: On figures of merit for randomly-shifted lattice rules. In: Wozniakowski, H., Plaskota, L. (eds.) Monte Carlo and Quasi-Monte Carlo Methods 2010, pp. 133–159. Springer, Berlin (2012)

    Chapter  Google Scholar 

  24. Mehrotra, S., Papp, D.: Generating nested quadrature formulas for general weight functions with known moments. Tech. Rep. arXiv:1203.1554v1 [math.NA], Northwestern University (2012). http://arxiv.org/abs/1203.1554v1

  25. Mehrotra, S., Papp, D.: Generating moment matching scenarios using optimization techniques. SIAM J. Optim. 23(2), 963–999 (2013). doi:10.1137/110858082

    Article  MATH  MathSciNet  Google Scholar 

  26. Monegato, G.: Stieltjes polynomials and related quadrature rules. SIAM Rev. 24(2), 137–158 (1982)

    Article  MATH  MathSciNet  Google Scholar 

  27. Neumaier, A.: Introduction to Numerical Analysis. Cambridge University Press, Cambridge (2001)

    Book  MATH  Google Scholar 

  28. Niederreiter, H.: Random Number Generation and Quasi-Monte Carlo Methods, CBMS-NSF Regional Conference Series in Applied Mathematics, vol. 63. SIAM, Philadelphia (1992)

    Book  Google Scholar 

  29. Niederreiter, H., Talay, D. (eds.): Monte Carlo and Quasi-Monte Carlo Methods. Springer, Berlin (2006)

    MATH  Google Scholar 

  30. Norkin, V., Pflug, G., Ruszczyński, A.: A branch and bound method for stochastic global optimization. Math. Program. 83, 425–450 (1998). doi:10.1007/BF02680569

    MATH  Google Scholar 

  31. Novak, E., Ritter, K.: High dimensional integration of smooth functions over cubes. Numer. Math. 75, 79–97 (1996)

    Article  MATH  MathSciNet  Google Scholar 

  32. Novak, E., Ritter, K.: Simple cubature formulas with high polynomial exactness. Constr. Approx. 15, 499–522 (1999)

    Article  MATH  MathSciNet  Google Scholar 

  33. Patterson, T.N.L.: The optimal addition of points to quadrature formulae. Math. Comput. 22(104), 847–856 (1968)

    Article  MATH  Google Scholar 

  34. Patterson, T.N.L.: An algorithm for generating interpolatory quadrature rules of the highest degree of precision with preassigned nodes for general weight functions. ACM Trans. Math. Softw. 15, 123–136 (1989). doi:10.1145/63522.63523

    Article  MATH  Google Scholar 

  35. Pennanen, T.: Epi-convergent discretizations of multistage stochastic programs. Math. Oper. Res. 30, 245–256 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  36. Pennanen, T., Koivu, M.: Integration quadrature in discretization of stochastic programs. Stoch. Program. E-Print Series (2002–11)

  37. Pennanen, T., Koivu, M.: Epi-convergent discretizations of stochastic programs via integration quadratures. Numer. Math. 100, 141–163 (2005). doi:10.1007/s00211-004-0571-4

    Article  MATH  MathSciNet  Google Scholar 

  38. Pflug, G.C.: Scenario tree generation for multiperiod financial optimization by optimal discretization. Math. Program. 89, 251–271 (2001)

    Article  MATH  MathSciNet  Google Scholar 

  39. Rockafellar, R.T., Uryasev, S.: Optimization of conditional value-at-risk. J. Risk 2(3), 21–41 (2000)

    Google Scholar 

  40. Rudin, W.: Real and Complex Analysis. McGraw-Hill, New York (1986)

    Google Scholar 

  41. Smoljak, S.A.: Quadrature and interpolation formulas for tensor products of certain classes of functions. Dokl. Akad. Nauk SSSR 4, 240–243 (1963)

    MathSciNet  Google Scholar 

  42. Sobol, I.M.: A Primer for the Monte Carlo Method. CRC Press, Boca Raton (1994)

    MATH  Google Scholar 

  43. Wallace, S.W., Ziemba, W.T. (eds.): Applications Of Stochastic Programming. Society for Industrial and Applied Mathematics, Philadelphia (2005)

    MATH  Google Scholar 

  44. Wasilkowski, G.W., Wozniakowski, H.: Explicit cost bounds of algorithms for multivariate tensor product problems. J. Complex. 11(1), 1–56 (1995)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Acknowledgments

Research supported in part by Grants DOE DE-FG02-10ER26037, SC0005102; DOE-SP0011568; and ONR N00014210051. We thank David Munger for the discussion on optimal lattice rules for smooth integrands and the help with his LatticeBuilder software. We also thank the associate editor and the referees for the constructive suggestions that helped improve the manuscript.

Author information

Authors and Affiliations

  1. Department of Mathematics and Statistics, York University, Toronto, Canada

    Michael Chen

  2. Department of Industrial Engineering and Management Sciences, Northwestern University, Evanston, IL, 60208, USA

    Sanjay Mehrotra

  3. Department of Mathematics, North Carolina State University, Raleigh, NC, 27695, USA

    Dávid Papp

Authors
  1. Michael Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sanjay Mehrotra
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dávid Papp
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dávid Papp.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, M., Mehrotra, S. & Papp, D. Scenario generation for stochastic optimization problems via the sparse grid method. Comput Optim Appl 62, 669–692 (2015). https://doi.org/10.1007/s10589-015-9751-7

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-015-9751-7

Keywords

Search

Navigation