Skip to main content
SpringerLink
Log in

Optimization via simulation: A review

  • Published:
Annals of Operations Research Aims and scope Submit manuscript

Abstract

We review techniques for optimizing stochastic discrete-event systems via simulation. We discuss both the discrete parameter case and the continuous parameter case, but concentrate on the latter which has dominated most of the recent research in the area. For the discrete parameter case, we focus on the techniques for optimization from a finite set: multiple-comparison procedures and ranking-and-selection procedures. For the continuous parameter case, we focus on gradient-based methods, including perturbation analysis, the likelihood ratio method, and frequency domain experimentation. For illustrative purposes, we compare and contrast the implementation of the techniques for some simple discrete-event systems such as the (s, S) inventory system and theGI/G/1 queue. Finally, we speculate on future directions for the field, particularly in the context of the rapid advances being made in parallel computing.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. V.M. Aleksandrov, V.I. Sysoyev and V.V. Shemeneva, Stochastic optimization, Eng. Cybern. 5(1968)11–16.

    Google Scholar 

  2. A.E. Albert and L.A. Gardner, Jr.,Stochastic Approximation and Nonlinear Regression, Research Monograph No. 42 (M.I.T. Press, Cambridge, MA, 1967).

    Google Scholar 

  3. S. Andradóttir, A new algorithm for stochastic approximation,Proc. 1990 Winter Simulation Conf. (1990) pp. 364–366.

  4. F. Azadivar and J.J. Talavage, Optimization of stochastic simulation models, Math. Comp. Simul. 22(1980)231–241.

    Google Scholar 

  5. J. Banks and J.S. Carson,Discrete-Event System Simulation (Prentice-Hall, Englewood Cliffs, N.J., 1984).

    Google Scholar 

  6. R.E. Bechhofer, A single-sample multiple decision procedure for ranking means of normal populations with known variances, Ann. Math. Statist. 25(1954)16–39.

    Google Scholar 

  7. A. Benveniste, M. Metivier and P. Priouret,Adaptive Algorithms and Stochastic Approximation (Springer, New York, 1990).

    Google Scholar 

  8. W.E. Biles and J.J. Swain,Optimization and Industrial Experimentation (Wiley-Interscience, New York, 1980).

    Google Scholar 

  9. R.E. Bixby, J.W. Gregory, I.J. Lustig, R.E. Marsten and D.F. Shanno, Very large-scale programming: a case study in combining interior point and simplex methods, Oper. Res. 25(1992)885–897.

    Google Scholar 

  10. G.E.P. Box and N.R. Draper,Empirical Model-Building and Response Surfaces (Wiley, New York, 1987).

    Google Scholar 

  11. P. Brémaud and F. Vázquez-Abad, On the pathwise computation of derivatives with respect to the rate of a point process, Queueing Syst. 10(1993)249–270.

    Google Scholar 

  12. M. Caramanis and G. Liberopoulos, Perturbation analysis for the design of flexible manufacturing flow controllers, Oper. Res. 40(1992)1107–1126.

    Google Scholar 

  13. G. Liberopolous and M. Caramanis, Infiniresimal perturbation analysis for second derivative estimation and design of manufacturing flow controllers, J. Optim. Theory Appl. (1994), to appear.

  14. C.G. Cassandras and S.G. Strickland, On-line sensitivity analysis of Markov chains, IEEE Trans. Auto. Control AC-34(1989)76–86.

    Google Scholar 

  15. E.K.P. Chong and P.J. Ramadge, Convergence of recursive optimization algorithms using infinitesimal perturbation analysis, Discr. Event Dyn. Syst. 1(1992)339–372.

    Google Scholar 

  16. E.K.P. Chong and P.J. Ramadge, Optimization of queues using an IPA based stochastic algorithm with general update times, SIAM J. Control Optim. (1993), to appear.

  17. K.L. Chung, On a stochastic approximation method, Ann. Math. Statist. 25(1954)463–483.

    Google Scholar 

  18. W.G. Cochran and G.M. Cox,Experimental Designs, 2nd ed. (Wiley, New York, 1957).

    Google Scholar 

  19. E.J. Dudewicz and S.R. Dalal, Allocation of measurements in ranking and selection with unequal variances, Sankhya B37(1975)28–78.

    Google Scholar 

  20. P. Dupuis and R. Simha, On sampling controlled stochastic approximation, IEEE Trans. Auto. Control AC-36(1991)915–924.

    Google Scholar 

  21. W. Farrell, Literature review and bibliography of simulation optimization,Proc. Winter Simulation Conf. (1977) pp. 117–124.

  22. M.C. Fu, Convergence of a stochastic approximation algorithm for theGI/G/1 queue using infinitesimal perturbation analysis, J. Optim. Theory Appl. 65(1990)149–160.

    Google Scholar 

  23. M.C. Fu, Sample path derivatives for (s, S) inventory systems, Oper. Res. (1993), to appear.

  24. M.C. Fu and K. Healy, Simulation optimization of (s, S) inventory systems,Proc. Winter Simulation Conf. (1992) pp. 506–514.

  25. M.C. Fu and Y.C. Ho, Using perturbation analysis for gradient estimation, averaging, and updating in a stochastic approximation algorithm,Proc. Winter Simulation Conf. (1988) pp. 509–517.

  26. M.C. Fu and J.Q. Hu, Extensions and generalizations of smoothed perturbation analysis in a generalized semi-Markov process framework, IEEE Trans. Auto. Control AC-37(1992)1483–1500.

    Google Scholar 

  27. M.C. Fu and J.Q. Hu, On choosing the characterization for smoothed perturbation analysis, IEEE Trans. Auto. Control AC-36(1991)1331–1336.

    Google Scholar 

  28. M.C. Fu and J.Q. Hu, Second derivative sample path estimators for theGI/G/m queue, Manag. Sci. 39(1993)359–383.

    Google Scholar 

  29. M.C. Fu, J.Q. Hu and L. Shi, Likelihood ratio methods with conditional Monte Carlo and splitting,Proc. 1993 Allerton Conf., to appear.

  30. R. Fujimoto, Parallel discrete event simulation, Commun. ACM 33(1990)30–53.

    Google Scholar 

  31. A. Gaivoronski, Optimization of stochastic discrete event dynamic systems: a survey of some recent results, Lecture Notes in Economics and Mathematical Systems 374,Proc. Workshop on Simulation and Optimization, Laxenburg, Austria, ed. G. Pflug and U. Dieter (1992) pp. 24–44.

  32. P. Glasserman,Gradient Estimation Via Perturbation Analysis (Kluwer Academic, 1991).

  33. P. Glasserman, Structural conditions for perturbation analysis derivative estimation: finite-time performance indices, Oper. Res. 39(1991)724–738.

    Google Scholar 

  34. P. Glasserman and W.B. Gong, Smoothed perturbation analysis for a class of discrete event systems, IEEE Trans. Auto. Control AC-35(1990)1218–1230.

    Google Scholar 

  35. P. Glasserman and D.D. Yao, Some guidelines and guarantees for common random numbers, Manag. Sci. 38(1992)884–908.

    Google Scholar 

  36. P.W. Glynn, Likelihood ratio gradient estimation: an overview,Proc. Winter Simulation Conf. (1987) pp. 366–374.

  37. P.W. Glynn, Likelihood ratio gradient estimation for stochastic systems, Commun. ACM 33(1990)75–84.

    Google Scholar 

  38. P.W. Glynn, Optimization of stochastic systems,Proc. Winter Simulation Conf. (1986) pp. 52–59.

  39. P.W. Glynn, Optimization of stochastic systems via simulation,Proc. Winter Simulation Conf. (1989) pp. 90–105.

  40. P.W. Glynn, Stochastic approximation for Monte Carlo optimization,Proc. Winter Simulation Conf. (1986) pp. 356–364.

  41. P.W. Glynn and P. Heidelberger, Analysis of parallel replicated simulations under a completion time constraint, ACM Trans. Modeling Comp. Simul. 1(1991)3–23.

    Google Scholar 

  42. D. Goldsman, B.L. Nelson and B. Schmeiser, Methods for selecting the best system,Proc. Winter Simulation Conf. (1991) pp. 177–186.

  43. W.B. Gong and Y.C. Ho, Smoothed perturbation analysis of discrete-event dynamic systems, IEEE Trans. Auto. Control AC-32(1987)858–867.

    Google Scholar 

  44. S.S. Gupta, On a decision rule for ranking means, Inst. Statist. Mimeo. Ser. No. 150, University of North Carolina, Chapel Hill, NC (1956).

    Google Scholar 

  45. J. Haddock and G. Bengu, Application of a simulation optimization system for a continuous review inventory model,Proc. Winter Simulation Conf. (1987) pp. 383–390.

  46. K. Healy and L.W. Schruben, Retrospective simulation response optimization,Proc. Winter Simulation Conf. (1991) pp. 901–906.

  47. K. Healy, Retrospective simulation response optimization, Ph.D. Dissertation, Cornell University (1992).

  48. Y.C. Ho, Special issue on dynamics of discrete event systems, Proc. IEEE 77 (1989).

  49. Y.C. Ho, Perturbation analysis: concepts and algorithms,Proc. Winter Simulation Conf. (1992) pp. 231–240.

  50. Y.C. Ho amd X.R. Cao,Discrete Event Dynamic Systems and Perturbation Analysis (Kluwer Academic, 1991).

  51. Y.C. Ho and X.R. Cao, Optimization and perturbation analysis of queueing networks, J. Optim. Theory Appl. 40(1983)559–582.

    Google Scholar 

  52. Y.C. Ho, X.R. Cao and C.G. Cassandras, Infinitesimal and finite perturbation analysis for queueing networks, Automatica 19(1983)439–445.

    Google Scholar 

  53. Y.C. Ho and C. Cassandras, A new approach to the analysis of discrete event dynamic systems, Automatica 19(1983)149–167.

    Google Scholar 

  54. Y.C. Ho, A. Eyler and T.T. Chien, A gradient technique for general buffer storage design in a serial production line, Int. J. Prod. Res. 17(1979)557–580.

    Google Scholar 

  55. Y.C. Ho, M.A. Eyler and T.T. Chien, A new approach to determine parameter sensitivity of transfer lines, Manag. Sci. 29(1983)700–714.

    Google Scholar 

  56. Y.C. Ho and S. Li, Extensions of perturbation analysis of discrete-event dynamic systems, IEEE Trans. Auto. Control AC-33(1988)427–438.

    Google Scholar 

  57. Y.C. Ho, S. Li and P. Vakili, On the efficient generation of discrete event sample paths under different system parameters, Math. Comp. Simul. 30(1988)347–370.

    Google Scholar 

  58. Y.C. Ho, L. Shi, L. Dai and W.B. Gong, Optimizing discrete event systems via the gradient surface method, Discr. Event Dyn. Syst. 2(1992)99–120.

    Google Scholar 

  59. Y.C. Ho, R. Sreenevas and P. Vakili, Ordinal optimization of DEDS, Discr. Event Dyn. Syst. 2(1992)61–88.

    Google Scholar 

  60. Y. Hochberg and A.C. Tamhane,Multiple Comparison Procedures (Wiley, New York, 1987).

    Google Scholar 

  61. J.C. Hsu, Simultaneous confidence levels for all distributions from the best, Ann. Statist. 9(1981)1026–1034.

    Google Scholar 

  62. J.C. Hsu and B.L. Nelson, Optimization over a finite number of system designs with one-stage sampling and multiple comparisons with the best,Proc. Winter Simulation Conf. (1988) pp. 451–457.

  63. J.S. Hunter and T.H. Naylor, Experimental designs for computer simulation experiments, Manag. Sci. 16(1970)422–434.

    Google Scholar 

  64. S.H. Jacobson, Optimal mean squared error analysis of the harmonic gradient estimators, J. Optim. Theory Appl. (1994), to appear.

  65. S.H. Jacobson, Oscillation amplitude consideration in frequency domain experiments,Proc. Winter Simulation Conf. (1989) pp. 406–410.

  66. S.H. Jacobson, Variance and bias reduction techniques for harmonic gradient estimators, Appl. Math. Comp. (1993), to appear.

  67. S.H. Jacobson and L.W. Schruben, A review of techniques for simulation optimization, Oper. Res. Lett. 8(1989)1–9.

    Google Scholar 

  68. S.H. Jacobson and L.W. Schruben, A simulation optimization procedure using harmonic analysis, Oper. Res. (1992), submitted.

  69. S.H. Jacobson, A. Buss and L.W. Schruben, Driving frequency selection for frequency domain simulation experiments, Oper. Res. 39(1991)917–924.

    Google Scholar 

  70. S.H. Jacobson, D. Morrice and L.W. Schruben, The global simulation clock as the frequency domain experiment index,Proc. Winter Simulation Conf. (1988) pp. 558–563.

  71. H. Kesten, Accelerated stochastic approximation, Ann. Math. Statist. 29(1958)41–59.

    Google Scholar 

  72. A.I. Khuri and J.A. Cornell,Response Surfaces (Marcel Dekker, 1987).

  73. J. Kiefer and J. Wolfowitz, Stochastic estimation of the maximum of a regression function, Ann. Math. Statist. 23(1952)462–466.

    Google Scholar 

  74. J.P.C. Kleijnen,Statistical Tools for Simulation Practitioners (Marcel Dekker, New York, 1987).

    Google Scholar 

  75. H.J. Kushner and D.C. Clark,Stochastic Approximation Methods for Constrained and Unconstrained Systems (Springer, New York, 1978).

    Google Scholar 

  76. A.M. Law and W.D. Kelton,Simulation Modeling and Analysis, 2nd ed. (McGraw-Hill, New York, 1991).

    Google Scholar 

  77. P. L'Ecuyer, A unified view of the IPA, SA, and LR gradient estimation techniques, Manag. Sci. 36(1990)1364–1383.

    Google Scholar 

  78. P. L'Ecuyer, An overview of derivative estimation,Proc. Winter Simulation Conf. (1991) pp. 207–217.

  79. P. L'Ecuyer, Convergence rates for steady-state derivative estimators, Ann. Oper. Res. 39(1992)121–136.

    Google Scholar 

  80. P. L'Ecuyer and P.W. Glynn, Stochastic optimization by simulation: convergence proofs for theGI/G/1 queue in steady state, Manag. Sci. (1994), to appear.

  81. P. L'Ecuyer, N. Giroux and P.W. Glynn, Stochastic optimization by simulation: numerical experiments with a simple queue in steady state, Manag. Sci. (1994), to appear.

  82. P. L'Ecuyer and G. Perron, On the convergence rates of IPA and FDC derivative estimators for finite-horizon stochastic simulations, Oper. Res. (1993), submitted.

  83. Y.T. Leung and R. Suri, Finite-time behavior of two simulation optimization algorithms,Proc. Winter Simulation Conf. (1990) pp. 372–376.

  84. L. Ljung, Analysis of recursive stochastic algorithms, IEEE Trans. Auto. Control AC-22(1977)551–575.

    Google Scholar 

  85. L. Ljung, Strong convergence of a stochastic approximation algorithm, Ann. Statist. 6(1978)680–696.

    Google Scholar 

  86. M. Mitra and S.K. Park, Solution to the indexing problem of frequency domain simulation experiments,Proc. Winter Simulation Conf. (1991) pp. 907–915.

  87. R.H. Myers, A.I. Khuri and W.H. Carter, Response surface methodology: 1966–1988, Technometrics 31(1989)137–157.

    Google Scholar 

  88. R.H. Myers,Response Surface Methodology (Allyn and Bacon, Boston, 1971).

    Google Scholar 

  89. M.B. Nevelson and R. Zalmanovich, Stochastic approximation and recursive estimation, translation of mathematical monographs, Vol. 47 (1973).

  90. G.Ch. Pflug, On-line optimization of simulated Markovian processes, Math. Oper. Res. 15(1990)381–395.

    Google Scholar 

  91. G.Ch. Pflug, Sampling derivatives of probabilities, Computing 42(1989)315–328.

    Google Scholar 

  92. B.T. Polyak, New method of stochastic approximation type, Auto. Remote Control 51(1990)937–946.

    Google Scholar 

  93. M.I. Reiman and A. Weiss, Sensitivity analysis via likelihood ratios,Proc. Winter Simulation Conf. (1986) pp. 285–289.

  94. M.I. Reiman and A. Weiss, Sensitivity analysis for simulations via likelihood ratios, Oper. Res. 37(1989)830–844.

    Google Scholar 

  95. H. Robbins and S. Monro, A stochastic approximation method, Ann. Math. Statist. 22(1951)400–407.

    Google Scholar 

  96. K.W. Ross and J. Wang, Solving product form stochastic networks with Monte Carlo summation,Proc. Winter Simulation Conf. (1990) pp. 270–275.

  97. R.Y. Rubinstein, How to optimize discrete-event systems from a single sample path by the score function method, Ann. Oper. Res. 27(1991)175–212.

    Google Scholar 

  98. R.Y. Rubinstein, The push-out method for sensitivity analysis of discrete event systems, Ann. Oper. Res. (1993), to appear.

  99. R.Y. Rubinstein, Sensitivity analysis and performance extrapolation for computer simulation models, Oper. Res. 37(1989)72–81.

    Google Scholar 

  100. R.Y. Rubinstein,Monte Carlo Optimization: Simulation and Sensitivity of Queueing Networks (Wiley, New York, 1986).

    Google Scholar 

  101. R.Y. Rubinstein and A. Shapiro,Discrete Event Systems: Sensitivity Analysis and Stochastic Optimization by the Score Function Method (Wiley, 1992).

  102. D. Ruppert, Almost sure approximations to the Robbins-Monro and Kiefer-Wolfowitz processes with dependent noise, Ann. Statist. 16(1982)178–187.

    Google Scholar 

  103. A. Ruszczynski and W. Syski, Stochastic approximation method with gradient averaging for unconstrained problems, IEEE Trans. Auto. Control AC-28(1983)1097–1105.

    Google Scholar 

  104. J. Sacks, Asymptotic distribution of stochastic approximation procedures, Ann. Math. Statist. 29(1958)373–405.

    Google Scholar 

  105. M.H. Safizadeh, Optimization in simulation: current issues and the future outlook, Naval Res. Logist. Quart. 37(1990)807–825.

    Google Scholar 

  106. M.H. Safizadeh and B.M. Thornton, Optimization in simulation experiments using response surface methodology, Comp. Ind. Eng. 8(1984)11–27.

    Google Scholar 

  107. J. Santner and A.C. Tamhane,Design of Experiments: Ranking and Selection (Marcel Dekker, 1984).

  108. R.G. Sargent, Research issues in metamodeling,Proc. Winter Simulation Conf. (1991) pp. 888–893.

  109. R.G. Sargent and T.K. Som, Current issues in frequency domain experimentation, Manag. Sci. 38(1992)667–687.

    Google Scholar 

  110. L.W. Schruben, Simulation optimization using frequency domain methods,Proc. Winter Simulation Conf. (1986) pp. 366–369.

  111. L.W. Schruben and V.J. Cogliano, An experimental procedure for simulation response surface model identification, Commun. ACM 30(1987)716–730.

    Google Scholar 

  112. L.W. Schruben and V.J. Cogliano, Simulation sensitivity analysis: a frequency domain approach,Proc. Winter Simulation Conf. (1981) pp. 455–459.

  113. B. Schmeiser, Simulation output data analysis, Ann. Oper. Res. (1994), this volume.

  114. L.Y. Shi, Discontinuous perturbation analysis of discrete event dynamic systems, IEEE Trans. Auto. Control (1993), submitted.

  115. D.E. Smith, An empirical investigation of optimum-seeking in the computer simulation situation, Oper. Res. 21(1973)475–497.

    Google Scholar 

  116. D.E. Smith, Automatic optimum-seeking program for digital simulation, Simulation 27(1976)27–32.

    Google Scholar 

  117. D.E. Smith, Requirements of an optimizer for computer simulation, Naval Res Logist. Quart. 20(1973)161–179.

    Google Scholar 

  118. D.W. Sullivan and J.R. Wilson, Restricted subset selection procedures for simulation, Oper. Res. 37(1989)52–71.

    Google Scholar 

  119. R. Suri, Perturbation analysis: the state of the art and research issues explained via theGI/G/1 queue, Proc. IEEE 77(1989)114–137.

    Google Scholar 

  120. R. Suri and Y.T. Leung, Single run optimization of discrete event simulations — an empirical study using theM/M/1 queue, IIE Trans. 21(1991)35–49.

    Google Scholar 

  121. R. Suri and M. Zazanis, Perturbation analysis gives strongly consistent sensitivity estimates for theM/G/1 queue, Manag. Sci. 34(1988)39–64.

    Google Scholar 

  122. P. Vakili, Using a standard clock technique for efficient simulation, Oper. Res. Lett. (1991) 445–452.

  123. P. Vakili, Massively parallel and distributed simulation of a class of discrete event systems: a different perspective, ACM Trans. Modeling Comp. Simul. (1993), to appear.

  124. F. Vázquez-Abad and P. L'Ecuyer, Comparing alternative methods for derivative estimation when IPA does not apply directly,Proc. Winter Simulation Conf. (1991) pp. 1004–1011.

  125. Y. Wardi, Simulation-based stochastic algorithm for optimizingGI/G/1 queues, Manuscript, Department of Industrial Engineering, Ben Gurion University (1988).

  126. M.T. Wasan,Stochastic Approximation (Cambridge University Press, 1969).

  127. J.R. Wilson, Future directions in response surface methodology for simulation,Proc. Winter Simulation Conf. (1987) pp. 378–381.

  128. S. Yakowitz, A globally convergent stochastic approximation, SIAM J. Control Optim. (1993), to appear.

  129. S. Yakowitz, T. Jayawadena and S. Li, Theory for automatic learning under partially observed Markov-dependent noise, IEEE Trans. Auto. Control AC-37(1992)1316–1324.

    Google Scholar 

  130. W.N. Yang and B.L. Nelson, Using common random numbers and control variates in multiple-comparison procedures, Oper. Res. 39(1991)583–591.

    Google Scholar 

  131. G. Yin, Stochastic approximation via averaging: the Polyak approach revisited, Lecture Notes in Economics and Mathematical Systems Vol. 374,Proc. Workshop on Simulation and Optimization, Laxenburg, Austria, ed. G. Pflug and U. Dieter (1990) pp. 119–134.

  132. M. Zazanis and R. Suri, Estimating first and second derivatives of response time forG/G/1 queues from a single sample path, Technical Report, Northwestern University, Evanston, IL; also Queueing Syst. (1986).

  133. B. Zhang and Y.C. Ho, Performance gradient estimation for very large finite Markov chains, IEEE Trans. Auto. Control AC-36(1991)1218–1227.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Maryland, 20742, College Park, MD, USA

    Michael C. Fu

Authors
  1. Michael C. Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Fu, M.C. Optimization via simulation: A review. Ann Oper Res 53, 199–247 (1994). https://doi.org/10.1007/BF02136830

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02136830

Keywords

Search

Navigation