Skip to main content
SpringerLink
Log in

Foundations of Ranking & Selection for Simulation Optimization

  • Chapter
  • First Online:
Advances in Modeling and Simulation

  • 503 Accesses

Abstract

In addition to his voluminous and profound research accomplishments, Pierre L’Ecuyer is an extraordinary educator; this includes expository talks and papers, especially in the area of pseudorandom-number generation. This paper is written in that same spirit, covering the foundations of ranking & selection for simulation optimization; simulation optimization is also an area of exceptional accomplishment for Pierre.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Bechhofer, R., Santner, T., Goldsman, D.: Design and Analysis of Experiments for Statistical Selection, Screening, and Multiple Comparisons. Wiley, New York (1995)

    Google Scholar 

  2. Chen, C.H., Chick, S.E., Lee, L.H., Pujowidianto, N.A.: Ranking and selection: efficient simulation budget allocation. In: Fu, M. (ed.) Handbook of Simulation Optimization, pp. 45–80. Springer (2015)

    Google Scholar 

  3. Chen, C.H., Lee, L.H.: Stochastic Simulation Optimization: An Optimal Computing Budget Allocation. World Scientific, Singapore (2011)

    Google Scholar 

  4. Chen, E.J.: Using parallel and distributed computing to increase the capability of selection procedures. In: Proceedings of the 2005 Winter Simulation Conference, pp. 723–731. IEEE (2005)

    Google Scholar 

  5. Chen, Y., Ryzhov, I.O.: Complete expected improvement converges to an optimal budget allocation. Adv. Appl. Probab. 51(1), 209–235 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  6. Dieker, A., Kim, S.H.: Selecting the best by comparing simulated systems in a group of three when variances are known and unequal. In: Proceedings of the 2012 Winter Simulation Conference, pp. 490–496. IEEE (2012)

    Google Scholar 

  7. Eckman, D.J., Henderson, S.G.: Guarantees on the probability of good selection. In: Proceedings of the 2018 Winter Simulation Conference, pp. 351–365. IEEE (2018)

    Google Scholar 

  8. Efron, B.: Large-scale Inference: Empirical Bayes Methods for Estimation, Testing, and Prediction. Cambridge University Press (2012)

    Google Scholar 

  9. Fan, W., Hong, L.J., Nelson, B.L.: Indifference-zone-free selection of the best. Oper. Res. 64(6), 1499–1514 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  10. Frazier, P.: Tutorial: Optimization via simulation with Bayesian statistics and dynamic programming. In: Proceedings of the 2012 Winter Simulation Conference, pp. 79–94. IEEE (2012)

    Google Scholar 

  11. Glynn, P., Juneja, S.: A large deviations perspective on ordinal optimization. In: Proceedings of the 2004 Winter Simulation Conference, pp. 577–585. IEEE (2004)

    Google Scholar 

  12. Goldsman, D.: Ranking and selection in simulation. In: Proceedings of the 1983 Winter Simulation Conference, pp. 387–393. IEEE (1983)

    Google Scholar 

  13. Gupta, S.S., Panchapakesan, S.: Multiple Decision Procedures: Theory and Methodology of Selecting and Ranking Populations. SIAM, Philadelphia (2002)

    Book  MATH  Google Scholar 

  14. Hong, L.J.: Fully sequential indifference-zone selection procedures with variance-dependent sampling. Nav. Res. Logist. 53(5), 464–476 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  15. Hunter, S.R., Nelson, B.L.: Parallel ranking and selection. In: Advances in Modeling and Simulation, pp. 249–275. Springer, New York (2017)

    Google Scholar 

  16. Jamieson, K., Nowak, R.: Best-arm identification algorithms for multi-armed bandits in the fixed confidence setting. In: 48th Annual Conference on Information Sciences and Systems (CISS), pp. 1–6. IEEE (2014)

    Google Scholar 

  17. Jennison, C., Johnstone, I.M., Turnbull, B.W.: Asymptotically optimal procedures for sequential adaptive selection of the best of several normal means. In: Statistical Decision Theory and Related Topics III, pp. 55–86. Elsevier (1982)

    Google Scholar 

  18. Kamiński, B., Szufel, P.: On parallel policies for ranking and selection problems. J. Appl. Stat. 45(9), 1690–1713 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  19. Kim, S.H., Nelson, B.L.: A fully sequential procedure for indifference-zone selection in simulation. ACM Trans. Model. Comput. Simul. (TOMACS) 11(3), 251–273 (2001)

    Article  MATH  Google Scholar 

  20. Kim, S.H., Nelson, B.L.: On the asymptotic validity of fully sequential selection procedures for steady-state simulation. Oper. Res. 54(3), 475–488 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  21. Kim, S.H., Nelson, B.L.: Selecting the best system. In: Henderson, S.G., Nelson, B.L. (eds.) Handbooks in Operations Research and Management Science, vol. 13, pp. 501–534. Elsevier (2006)

    Google Scholar 

  22. L’Ecuyer, P., Simard, R., Chen, E.J., Kelton, W.D.: An object-oriented random-number package with many long streams and substreams. Oper. Res. 50(6), 1073–1075 (2002)

    Article  Google Scholar 

  23. Lee, S., Nelson, B.L.: General-purpose ranking and selection for computer simulation. IIE Trans. 48(6), 555–564 (2016)

    Article  Google Scholar 

  24. Luo, J., Hong, L.J., Nelson, B.L., Wu, Y.: Fully sequential procedures for large-scale ranking-and-selection problems in parallel computing environments. Oper. Res. 63(5), 1177–1194 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  25. Luo, Y.C., Chen, C.H., Yücesan, E., Lee, I.: Distributed web-based simulation optimization. In: Proceedings of the 2000 Winter Simulation Conference, pp. 1785–1793. IEEE (2000)

    Google Scholar 

  26. Mukhopadhyay, N., Solanky, T.K.: Multistage Selection and Ranking Procedures: Second Order Asymptotics. CRC Press, Boca Raton, Florida (1994)

    MATH  Google Scholar 

  27. Nelson, B.L.: Selecting the best simulated system: thinking differently about an old problem. In: International Conference on Monte Carlo and Quasi-Monte Carlo Methods in Scientific Computing, pp. 69–79. Springer, New York (2018)

    Google Scholar 

  28. Nelson, B.L., Matejcik, F.J.: Using common random numbers for indifference-zone selection and multiple comparisons in simulation. Manage. Sci. 41(12), 1935–1945 (1995)

    Article  MATH  Google Scholar 

  29. Nelson, B.L., Pei, L.: Foundations and Methods of Stochastic Simulation: A First Course, 2nd edn. Springer, New York (2021)

    Book  Google Scholar 

  30. Nelson, B.L., Swann, J., Goldsman, D., Song, W.: Simple procedures for selecting the best simulated system when the number of alternatives is large. Oper. Res. 49(6), 950–963 (2001)

    Article  Google Scholar 

  31. Ni, E.C., Ciocan, D.F., Henderson, S.G., Hunter, S.R.: Efficient ranking and selection in parallel computing environments. Oper. Res. 65(3), 821–836 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  32. Paulson, E.: A sequential procedure for selecting the population with the largest mean from \(k\) normal populations. Ann. Math. Stat. 174–180 (1964)

    Google Scholar 

  33. Pei, L., Hunter, S.R., Nelson, B.L.: A new framework for parallel ranking & selection using an adaptive standard. In: Proceedings of the 2018 Winter Simulation Conference, pp. 2201–2212. IEEE (2018)

    Google Scholar 

  34. Pei, L., Nelson, B.L., Hunter, S.R.: Evaluation of bi-PASS for parallel simulation optimization. In: Proceedings of the 2020 Winter Simulation Conference, pp. 2960–2971. IEEE (2020)

    Google Scholar 

  35. Rinott, Y.: On two-stage selection procedures and related probability-inequalities. Commun. Stat.-Theory Methods 7(8), 799–811 (1978)

    Article  MathSciNet  MATH  Google Scholar 

  36. Salemi, P., Song, E., Nelson, B.L., Staum, J.: Gaussian Markov random fields for discrete optimization via simulation: framework and algorithms. Oper. Res. 67(1), 250–266 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  37. Zhong, Y., Hong, L.J.: Fully sequential ranking and selection procedures with PAC guarantee. In: Proceedings of the 2018 Winter Simulation Conference, pp. 1898–1908. IEEE (2018)

    Google Scholar 

  38. Zhong, Y., Hong, L.J.: Knockout-tournament procedures for large-scale ranking and selection in parallel computing environments. Oper. Res. 70(1), 432–453 (2021)

    Google Scholar 

Download references

Acknowledgements

Special thanks to David Eckman and Linda Pei for comments on the structure and specifics of this tutorial, as well as to the two referees for their careful reports. This research was supported by National Science Foundation Grant Number DMS-1854562.

Author information

Authors and Affiliations

  1. Northwestern University, Evanston, IL, USA

    Barry L. Nelson

Authors
  1. Barry L. Nelson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Barry L. Nelson .

Editor information

Editors and Affiliations

  1. Randwick, NSW, Australia

    Zdravko Botev

  2. NVIDIA, Berlin, Germany

    Alexander Keller

  3. University of Waterloo, Waterloo, ON, Canada

    Christiane Lemieux

  4. Rennes Bretagne-Atlantique, INRIA, Rennes Cedex, France

    Bruno Tuffin

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Nelson, B.L. (2022). Foundations of Ranking & Selection for Simulation Optimization. In: Botev, Z., Keller, A., Lemieux, C., Tuffin, B. (eds) Advances in Modeling and Simulation. Springer, Cham. https://doi.org/10.1007/978-3-031-10193-9_18

Download citation

Publish with us

Policies and ethics

Search

Navigation