Skip to main content
SpringerLink
Log in

Variable-Number Sample-Path Optimization

  • FULL LENGTH PAPER
  • Published:
Mathematical Programming Submit manuscript

Abstract

The sample-path method is one of the most important tools in simulation-based optimization. The basic idea of the method is to approximate the expected simulation output by the average of sample observations with a common random number sequence. In this paper, we describe a new variant of Powell’s unconstrained optimization by quadratic approximation (UOBYQA) method, which integrates a Bayesian variable-number sample-path (VNSP) scheme to choose appropriate number of samples at each iteration. The statistically accurate scheme determines the number of simulation runs, and guarantees the global convergence of the algorithm. The VNSP scheme saves a significant amount of simulation operations compared to general purpose ‘fixed-number’ sample-path methods. We present numerical results based on the new algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Chen, H.-C., Chen, C.-H., Yucesan, E.: An asymptotic allocation for simultaneous simulation experiments. In: Proceedings of the 1999 Winter Simulation Conference, pp. 359–366 (1999)

  2. Chick S.E., Inoue K. (2001). New procedures to select the best simulation system using common random numbers. Manage. Sci. 47(8): 1133–1149

    Article  Google Scholar 

  3. Chick S.E., Inoue K. (2003). New two-stage and sequential procedures for selecting the best simulated system. Oper. Res. 49: 1609–1624

    Google Scholar 

  4. Conn, A.R., Scheinberg, K., Toint, P.L.: On the convergence of derivative-free methods for unconstrained optimization. Approximation Theory and Optimization, Tributes to M.J.D. Powell, Buhmann, M.D., Iserles, A. (eds.) pp. 83–108 (1996)

  5. Conn, A.R., Toint, Ph.L.: An algorithm using quadratic interpolation for unconstrained derivative free optimization. In: Di Pillo, G., Giannessi, F. (eds.) Nonlinear Optimization and Applications, pp. 27–47. Plenum Press, New York (1996)

    Google Scholar 

  6. Dawid A.P. (1981). Some matrix-variate distribution theory: Notational considerations and a bayesian application. Biometrika 68(1): 265–74

    Article  MathSciNet  MATH  Google Scholar 

  7. DeGroot M.H. (1970). Optimal Statistical Decisions. McGraw-Hill Inc, New York

    MATH  Google Scholar 

  8. Deng, G., Ferris, M.C.: Adaptation of the UOBYQA algorithm for noisy functions. In: Perrone, L.F., Wieland, F.P., Lawson, B.G., Liu, J., Nicol, D.M., Fujimoto, R.M. (eds.) Proceedings of the 2006 Winter Simulation Conference, pp. 312–319 (2006)

  9. Deng, G., Ferris, M.C.: Extension of the DIRECT optimization algorithm for noisy functions. (2007, Submitted to Winter Simulation Conference)

  10. Durrett R. (2004). Probability: Theory and Examples, 3rd edn. Duxbury Press, Pacific Grove

    Google Scholar 

  11. Ferris M.C., Munson T.S., Sinapiromsaran K. (2000). A practical approach to sample-path simulation optimization. In: Joines, J.A., Barton, R.R., Kang, K., and Fishwick, P.A. (eds) Proceedings of the 2000 Winter Simulation Conference, pp 795–804. Omnipress, Orlando

    Google Scholar 

  12. Fu M. (1994). Optimization via simulation: a review. Ann. Oper. Res. 53: 199–248

    Article  MathSciNet  MATH  Google Scholar 

  13. Gosavi A. (2003). Simulation-Based Optimization: Parametric Optimization Techniques and Reinforcement Learning. Kluwer, Norwell

    MATH  Google Scholar 

  14. Gürkan, G., Yonca Özge, A., Robinson, S.M.: Sample-path solution of stochastic variational inequalities, with applications to option pricing. In: Brunner, D.T., Charnes, J.M., Morrice, D.J., Swain, J.J. (eds.) Proceedings of the 1996 Winter Simulation Conference, pp. 337–344 (1996)

  15. Gürkan, G., Yonca Özge, A., Robinson, S.M.: Solving stochastic optimization problems with stochastic constraints: An application in network design. In: Farrington, P.A., Nembhard, H.B., Sturrock, D.T., Evans, G.W. (eds.) Proceedings of the 1999 Winter Simulation Conference, pp. 471–478 (1999)

  16. Homem–de–Mello, T.: Variable-sample methods for stochastic optimization. ACM Trans. Model. Comput. Simul. 13(2), (2003)

  17. Homem–de–Mello, T.: On rates of convergence for stochastic optimization problems under non-I.I.D. sampling. (2006, submitted for publication)

  18. Kim, S.-H., Nelson, B.L.: Selecting the best system: Theory and methods. In: Proceedings of the 2003 Winter Simulation Conference, pp. 101–112 (2003)

  19. Kleywegt A.J., Shapiro A., Homem–de–Mello T. (2001). The sample average approximation method for stochastic discrete optimization. SIAM. J. Optim. 12: 479–502

    Article  MathSciNet  MATH  Google Scholar 

  20. Law A., Kelton W. (2000). Simulation Modeling and Analysis, 3rd edn. McGraw-Hill, New York

    Google Scholar 

  21. Marazzi M., Nocedal J. (2002). Wedge trust region methods for derivative free optimization. Math. Program. 91: 289–305

    Article  MathSciNet  MATH  Google Scholar 

  22. Moré, J.J.: Recent developments in algorithms and software for trust region methods. In: Mathematical Programming: The State of the Art. Springer, Heidelberg, pp. 258–287 (1983)

  23. Moré J.J., Sorensen D.C. (1983). Computing a trust region step. SIAM J. Sci. Stat. Comput. 4: 553–572

    Article  MATH  Google Scholar 

  24. Nocedal J., Wright S.J. (2006). Numerical Optimization, 2nd edn. Springer, New York

    MATH  Google Scholar 

  25. Plambeck E.L., Fu B.R., Robinson S.M., Suri R. (1993). Throughput optimization in tandem production lines via nonsmooth programming. In: Schoen, J. (eds) Proceedings of the 1993 Summer Computer Simulation Conference., pp 70–75. Society for Computer Simulation, San Diego

    Google Scholar 

  26. Plambeck E.L., Fu B.R., Robinson S.M., Suri R. (1996). Sample-path optimization of convex stochastic performance functions. Math. Program. 75: 137–176

    MathSciNet  Google Scholar 

  27. Powell M.J.D. (2002). UOBYQA: Unconstrained optimization by quadratic approximation. Math. Program. 92: 555–582

    Article  MathSciNet  MATH  Google Scholar 

  28. Powell, M.J.D.: The NEWUOA software for unconstrained optimization with derivatives. DAMTP Report 2004/NA05, University of Cambridge (2004)

  29. Prakash, P., Deng, G., Converse, M.C., Webster, J.G., Mahvi, D.M., Ferris, M.C.: Design optimization of a robust sleeve antenna for hepatic microwave ablation (Working paper)

  30. Robinson S.M. (1996). Analysis of sample-path optimization. Math. Oper. Res. 21: 513–528

    MathSciNet  MATH  Google Scholar 

  31. Ruszczyński, A., Shapiro, A.: Stochastic programming models. In: Stochastic Programming. Elsevier, Amsterdam, pp. 1–63 (2003)

  32. Shapiro A. (1993). Asymptotic behavior of optimal solutions in stochastic programming. Math. Oper. Res. 18: 829–845

    Article  MathSciNet  MATH  Google Scholar 

  33. Shapiro A. (2000). Statistical inference of stochastic optimization problems. In: Uryasev, S.P. (eds) Probabilistic Constrained Optimization: Methodology and Applications, pp 91–116. Kluwer, Dordrecht

    Google Scholar 

  34. Shapiro, A.: Monte Carlo sampling approach to stochastic programming. In: Penot, J.P. (ed.) ESAIM: Proceedings, vol. 13, pp. 65–73 (2003)

  35. Shapiro, A.: Monte Carlo sampling methods. In: Stochastic Programming, pp. 1–63. Elsevier, Amsterdam (2003)

  36. Shapiro A., Homem–de–Mello T. (1998). A simulation-based approach to two-stage stochastic programming with recourse. Math. Program. Ser. A 81(3): 301–325

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, University of Wisconsin, 480 Lincoln Drive, Madison, WI, 53706, USA

    Geng Deng

  2. Computer Sciences Department, University of Wisconsin, 1210 West Dayton Street, Madison, WI, 53706, USA

    Michael C. Ferris

Authors
  1. Geng Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael C. Ferris
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael C. Ferris.

Additional information

This paper is dedicated to Stephen Robinson on the occasion of his 65th birthday. The authors are grateful for his encouragement and guidance over the past two decades, and the inspirational work he has done in the topic of this paper.

This material is based on research partially supported by the National Science Foundation Grants DMI-0521953, DMS-0427689 and IIS-0511905 and the Air Force Office of Scientific Research Grant FA9550-04-1-0192.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Deng, G., Ferris, M.C. Variable-Number Sample-Path Optimization. Math. Program. 117, 81–109 (2009). https://doi.org/10.1007/s10107-007-0164-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10107-007-0164-y

Keywords

Mathematics Subject Classification (2000)

Search

Navigation