Skip to main content
SpringerLink
Log in

A Guide to Sample Average Approximation

  • Chapter
  • First Online:
Handbook of Simulation Optimization

Part of the book series: International Series in Operations Research & Management Science ((ISOR,volume 216))

Abstract

This chapter reviews the principles of sample average approximation (SAA) for solving simulation optimization problems. We provide an accessible overview of the area and survey interesting recent developments. We explain when one might want to use SAA and when one might expect it to provide good-quality solutions. We also review some of the key theoretical properties of the solutions obtained through SAA. We contrast SAA with stochastic approximation (SA) methods in terms of the computational effort required to obtain solutions of a given quality, explaining why SA “wins” asymptotically. However, an extension of SAA known as retrospective optimization can match the asymptotic convergence rate of SA, at least up to a multiplicative constant.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 139.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 179.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 179.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. S. Ahmed and A. Shapiro. Solving chance-constrained stochastic programs via sampling and integer programming. In Tutorials in Operations Research, pages 261–269. INFORMS, 2008.

    Google Scholar 

  2. S. Alexander, T. F. Coleman, and Y. Li. Minimizing CVaR and VaR for a portfolio of derivatives. Journal of Banknig and Finance, 30(2):584–605, 2006.

    Google Scholar 

  3. S. Andradóttir. A scaled stochastic approximation algorithm. Management Science, 42:475–498, 1996.

    Article  Google Scholar 

  4. S. Asmussen and P. W. Glynn. Stochastic Simulation: Algorithms and Analysis, volume 57 of Stochastic Modeling and Applied Probability. Springer, New York, 2007.

    Google Scholar 

  5. F. Bastin, C. Cirillo, and P. L. Toint. Convergence theory for nonconvex stochastic programming with an application to mixed logit. Mathematical Programming B, 108:207–234, 2006.

    Article  Google Scholar 

  6. M. S. Bazaara, H. Sherali, and C. M. Shetty. Nonlinear Programming: Theory and Algorithms. John Wiley & Sons, New York, NY., 2006.

    Google Scholar 

  7. P. Billingsley. Probability and Measure. Wiley, New York, NY., 1995.

    Google Scholar 

  8. J. R. Birge and F. Louveaux. Introduction to Stochastic Programming. Springer, New York, 1997.

    Google Scholar 

  9. M. Broadie, D. M. Cicek, and A. Zeevi. General bounds and finite-time improvement for the Kiefer-Wolfowitz stochastic approximation algorithm. Operations Research, 59(5):1211–1224, 2011.

    Article  Google Scholar 

  10. F. H. Clarke. Optimization and Nonsmooth Analysis. Wiley, New York, 1983.

    Google Scholar 

  11. G. Deng and M. C. Ferris. Variable-number sample-path optimization. Mathematical Programming, 117:81–109, 2009.

    Article  Google Scholar 

  12. J. A. Dieudonné. Foundations of Modern Analysis. Academic Press, New York, 1960.

    Google Scholar 

  13. M. C. Fu. Gradient estimation. In S. G. Henderson and B. L. Nelson, editors, Simulation, Handbooks in Operations Research and Management Science, pages 575–616. Elsevier, Amsterdam, 2006.

    Chapter  Google Scholar 

  14. M. C. Fu and K. Healy. Techniques for optimization via simulation: an experimental study on an (s, S) inventory system. IIE Transactions, 29(3):191–199, 1993.

    Google Scholar 

  15. M. C. Fu and J. Q. Hu. Conditional Monte Carlo: Gradient Estimation and Optimization Applications. Kluwer, Boston, 1997.

    Book  Google Scholar 

  16. A. A. Gaivoronski and G. Pflug. Value-at-Risk in portfolio optimization: Properties and computational approach. Journal of Risk, 7(2):1–31, 2005.

    Google Scholar 

  17. P. Glasserman. Gradient Estimation Via Perturbation Analysis. Kluwer, The Netherlands, 1991.

    Google Scholar 

  18. G. Gürkan, A. Y. Özge, and S. M. Robinson. Sample-path solution of stochastic variational inequalities. Mathematical Programming, 84:313–333, 1999.

    Article  Google Scholar 

  19. J. M. Harrison and J. A. Van Mieghem. Multi-resource investment stragies: Operational hedging under demand uncertainty. European Journal of Operational Research, 113:17–29, 1999.

    Article  Google Scholar 

  20. K. Healy and L. W. Schruben. Retrospective simulation response optimization. In B. L. Nelson, D. W. Kelton, and G. M. Clark, editors, Proceedings of the 1991 Winter Simulation Conference, pages 954–957. IEEE, Piscataway, NJ, 1991.

    Google Scholar 

  21. T. Homem-de-Mello. Estimation of derivatives of nonsmooth performance measures in regenerative systems. Mathematics of Operations Research, 26:741–768, 2001.

    Article  Google Scholar 

  22. T. Homem-de-Mello. Variable-sample methods for stochastic optimization. ACM Transactions on Modeling and Computer Simulation, 13:108–133, 2003.

    Article  Google Scholar 

  23. L. J. Hong, Y. Yang, and L. Zhang. Sequential convex approximations to joint chance constrained programs: A Monte Carlo approach. Operations Research, 59(3):617–630, 2011.

    Article  Google Scholar 

  24. H. Hu, T. Homem-de-Mello, and S. Mehrotra. Sample average approximation of stochastic dominance constrained programs. Mathematical Programming, 133(1–2):171–201, 2012.

    Article  Google Scholar 

  25. A. I. Kibzun and Y. S. Kan. Stochastic Programming Problems with Probability and Quantile Functions. Wiley, New York, 1996.

    Google Scholar 

  26. J. Kiefer and J. Wolfowitz. Stochastic estimation of the maximum of a regression function. Annals of Mathematical Statistics, 23:462–466, 1952.

    Article  Google Scholar 

  27. S. Kim and S. G. Henderson. The mathematics of continuous-variable simulation optimization. In S. J. Mason, R. R. Hill, L. Moench, and O. Rose, editors, Proceedings of the 2008 Winter Simulation Conference, pages 122–132. IEEE, Piscataway, NJ, 2008.

    Chapter  Google Scholar 

  28. A. J. Kleywegt, A. Shapiro, and T. Homem-de-Mello. The sample average approximation method for stochastic discrete optimization. SIAM Journal on Optimization, 12:479–502, 2001.

    Article  Google Scholar 

  29. M. Koivu. Variance reduction in sample approximations of stochastic programs. Mathematical Programming, 103(3):463–485, 2005.

    Article  Google Scholar 

  30. H. J. Kushner and G. G. Yin. Stochastic Approximation and Recursive Algorithms and Applications. Springer-Verlag, New York, 2nd edition, 2003.

    Google Scholar 

  31. J. Luedtke and S. Ahmed. A sample approximation approach for optimization with probabilistic constraints. SIAM Journal on Optimization, 19:674–699, 2008.

    Article  Google Scholar 

  32. W. K. Mak, D. P. Morton, and R. K. Wood. Monte Carlo bounding techniques for determining solution quality in stochastic programs. Operations Research Letters, 24:47–56, 1999.

    Article  Google Scholar 

  33. Y. Nesterov. Introductory Lectures on Convex Optimization. Kluwer Academic Publishers, Norwell, MA, 2004.

    Book  Google Scholar 

  34. J. M. Ortega and W. C. Rheinboldt. Iterative Solution of Nonlinear Equations in Several Variables. Academic Press, New York, NY., 1970.

    Google Scholar 

  35. R. Pasupathy. On choosing parameters in retrospective-approximation algorithms for stochastic root finding and simulation optimization. Operations Research, 58:889–901, 2010.

    Article  Google Scholar 

  36. R. Pasupathy and S. Kim. The stochastic root-finding problem: overview, solutions, and open questions. ACM Transactions on Modeling and Computer Simulation, 21(3):23, 2011.

    Google Scholar 

  37. E. L. Plambeck, B.-R. Fu, S. M. Robinson, and R. Suri. Sample-path optimization of convex stochastic performance functions. Mathematical Programming, 75:137–176, 1996.

    Google Scholar 

  38. E. Polak. Optimization: Algorithms and Consistent Approximations. Springer, New York, NY, 1997.

    Book  Google Scholar 

  39. B. T. Polyak. New stochastic approximation type procedures. Automat. i Telemekh., 7:98–107, 1990.

    Google Scholar 

  40. B. T. Polyak and A. B. Juditsky. Acceleration of stochastic approximation by averaging. SIAM Journal on Control and Optimization, 30:838–855, 1992.

    Article  Google Scholar 

  41. A. Ravindran, D. T. Phillips, and J. J. Solberg. Operations Research: Principles and Practice. Wiley, New York, NY, 2nd ed. edition, 1987.

    Google Scholar 

  42. H. Robbins and S. Monro. A stochastic approximation method. Annals of Mathematical Statistics, 22:400–407, 1951.

    Article  Google Scholar 

  43. S. M. Robinson. Analysis of sample-path optimization. Mathematics of Operations Research, 21:513–528, 1996.

    Article  Google Scholar 

  44. S. M. Ross. Stochastic Processes. Wiley, New York, 2nd edition, 1996.

    Google Scholar 

  45. J. Royset. On sample size control in sample average approximations for solving smooth stochastic programs. Computational Optimization and Applications, 55(2):265–309, 2013.

    Article  Google Scholar 

  46. J. Royset and R. Szechtman. Optimal budget allocation for sample average approximation. Operations Research, 61(3):762–776, 2013.

    Article  Google Scholar 

  47. R. Y. Rubinstein and A. Shapiro. Optimization of static simulation models by the score function method. Mathematics and Computers in Simulation, 32:373–392, 1990.

    Article  Google Scholar 

  48. R. Y. Rubinstein and A. Shapiro. Discrete Event Systems: Sensitivity Analysis and Stochastic Optimization by the Score Function Method. Wiley, Chichester, 1993.

    Google Scholar 

  49. D. Ruppert. Stochastic approximation. In B. K. Ghosh and P. K. Sen, editors, Handbook of Sequential Analysis, pages 503–529. Marcel Dekker, New York, 1991.

    Google Scholar 

  50. A. Ruszczyński. A linearization method for nonsmooth stochastic programming problems. Mathematics of Operations Research, 12:32–49, 1987.

    Article  Google Scholar 

  51. A. Ruszczyński and A. Shapiro, editors. Stochastic Programming. Handbook in Operations Research and Management Science. Elsevier, New York, NY, 2003.

    Google Scholar 

  52. R. J. Serfling. Approximation Theorems of Mathematical Statistics. John Wiley & Sons, Inc., New York, NY, 1980.

    Google Scholar 

  53. A. Shapiro. Asymtotic behavior of optimal solutions in stochastic programming. Mathematics of Operations Research, 18:829–845, 1993.

    Article  Google Scholar 

  54. A. Shapiro. Simulation-based optimization – convergence analysis and statistical inference. Stochastic Models, 12(3):425–454, 1996.

    Article  Google Scholar 

  55. A. Shapiro. Monte Carlo sampling methods. In A. Ruszczynski and A. Shapiro, editors, Stochastic Programming, Handbooks in Operations Research and Management Science. Elsevier, 2003.

    Google Scholar 

  56. A. Shapiro. Sample average approximation. In S. I. Gass and M. C. Fu, editors, Encyclopedia of Operations Research and Management Science, pages 1350–1355. Springer, 3rd edition, 2013.

    Google Scholar 

  57. A. Shapiro, D. Dentcheva, and A. Ruszczyński. Lectures on Stochastic Programming: Modeling and Theory. MPS-SIAM Series on Optimization. SIAM-MPS, Philadelphia, PA, 2009.

    Book  Google Scholar 

  58. Stochastic Programming Society. Stochastic programming introduction. www.stoprog.org, 2014.

  59. J. A. Van Mieghem and N. Rudi. Newsvendor networks: Inventory management and capacity investment with discretionarty activities. Manufacturing and Service Operations Management, 4(4):313–335, 2002.

    Article  Google Scholar 

  60. S. Vogel. Stability results for stochastic programming problems. Optimization, 19(2):269–288, 1988.

    Article  Google Scholar 

  61. S. Vogel. A stochastic approach to stability in stochastic programming. Journal of Computational and Applied Mathematics, 45:65–96, 1994.

    Article  Google Scholar 

  62. R. Wets. Stochastic programming. In G. L. Nemhauser, A. H. G.Rinnooy Kan, and M. J. Todd, editors, Optimization, volume 1 of Handbooks in Operations Research and Management Science, pages 573–629. Elsevier, 1989.

    Google Scholar 

  63. H. Xu and D. Zhang. Smooth sample average approximation of stationary point in nonsmooth stochastic optimization and application. Mathematical Programming, 119:371–401, 2009.

    Article  Google Scholar 

Download references

Acknowledgements

This work was partially supported by National Science Foundation grants CMMI-0800688 and CMMI-1200315, and by Singapore MOE Academic Research Fund grant WBS R-266-000-049-133.

Author information

Authors and Affiliations

  1. National University of Singapore, Singapore, Singapore

    Sujin Kim

  2. Purdue University, West Lafayette, IN, USA

    Raghu Pasupathy

  3. Cornell University, Ithaca, NY, USA

    Shane G. Henderson

Authors
  1. Sujin Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Raghu Pasupathy
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shane G. Henderson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shane G. Henderson .

Editor information

Editors and Affiliations

  1. University of Maryland, College Park, Maryland, USA

    Michael C Fu

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer Science+Business Media New York

About this chapter

Cite this chapter

Kim, S., Pasupathy, R., Henderson, S.G. (2015). A Guide to Sample Average Approximation. In: Fu, M. (eds) Handbook of Simulation Optimization. International Series in Operations Research & Management Science, vol 216. Springer, New York, NY. https://doi.org/10.1007/978-1-4939-1384-8_8

Download citation

Publish with us

Policies and ethics

Search

Navigation