Skip to main content
SpringerLink
Log in

Stochastic Optimization

  • Chapter
  • First Online:
Book cover Handbook of Computational Statistics

Part of the book series: Springer Handbooks of Computational Statistics ((SHCS))

Abstract

Stochastic optimization algorithms have been growing rapidly in popularityover the last decade or two, with a number of methods now becomingindustry standard approaches for solving challenging optimization problems.This chapter provides a synopsis of some of the critical issues associatedwith stochastic optimization and a gives a summary of several popularalgorithms. Much more complete discussions are available in the indicatedreferences.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 259.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 329.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 329.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Arsham, H.: Techniques for Monte Carlo optimizing. Monte Carlo Methods Appl. 4, 181–229 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  • Baba, N., Shoman, T., Sawaragi, Y.: A modified convergence theorem for a random optimization method. Inf. Sci. 13, 159–166 (1977)

    Article  MathSciNet  MATH  Google Scholar 

  • Bazaraa, M.S., Sherali, H.D., Shetty, C.M.: Nonlinear programming: theory and algorithms. Wiley, New York (1993)

    MATH  Google Scholar 

  • Blum, J.R.: Multidimensional stochastic approximation methods. Ann. Math. Stat. 25, 737–744 (1954)

    Article  MATH  Google Scholar 

  • Box, G.E.P.: Evolutionary operation: a method for increasing industrial productivity. J. R. Stat. Soc. Ser. C: Appl. Stat. 6, 81–101 (1957)

    Google Scholar 

  • Cochran, J.J. (ed.): Encyclopedia of operations research and management science. Wiley, Hoboken (2011)

    Google Scholar 

  • Das, S., Spall, J.C., Ghanem, R.: Efficient Monte Carlo computation of Fisher information matrix using prior information. Comput. Stat. Data Anal. 54(2), 272–289 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  • De Jong, K.A.: An analysis of the behavior of a class of genetic adaptive systems. Ph.D. dissertation, University of Michigan (University Microfilms no. 76–9381) (1975)

    Google Scholar 

  • Dippon, J., Renz, J.: Weighted means in stochastic approximation of minima. SIAM J. Contr. Optim. 35, 1811–1827 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  • Fabian, V.: Stochastic approximation. In: Rustigi, J.S. (ed.) Optimizing methods in statistics, pp. 439–470. Academic, New York (1971)

    Google Scholar 

  • Fogel, D.B.: Evolutionary computation: toward a new philosophy of machine intelligence. IEEE, Piscataway (2000)

    Google Scholar 

  • Fouskakis, D., Draper, D.: Stochastic optimization: a review. Int. Stat. Rev. 70, 315–349 (2002)

    Article  MATH  Google Scholar 

  • Fu, M.C.: Optimization for simulation: theory vs. practice. INFORMS J. Comput. 14, 192–227 (2002)

    Google Scholar 

  • Gentle, J.E.: Random number generation and Monte Carlo methods. Springer, New York (2003)

    MATH  Google Scholar 

  • Gerencsér, L.: Convergence rate of moments in stochastic approximation with simultaneous perturbation gradient approximation and resetting. IEEE Trans. Autom. Control 44, 894–905 (1999)

    Article  MATH  Google Scholar 

  • Gerencsér, L., Hill, S.D., Vágó, Z.: Optimization over discrete sets via SPSA. In: Proceedings of the IEEE Conference on Decision and Control, Phoenix, 7–10 December 1999, 1791–1795

    Google Scholar 

  • Goldberg, D.E.: Genetic algorithms in search, optimization, and machine learning. Addison-Wesley, Reading (1989)

    MATH  Google Scholar 

  • Goldberg, D.E.: The design of innovation: lessons from and for competent genetic algorithms. Kluwer Academic, Boston (2002)

    MATH  Google Scholar 

  • Gosavi, A.: Simulation-based optimization: parametric optimization techniques and reinforcement learning. Kluwer Academic, Boston (2003)

    MATH  Google Scholar 

  • Hill, S.D.: Discrete stochastic approximation with application to resource allocation. Johns Hopkins APL Tech. Dig. 26, 15–21 (2005)

    Google Scholar 

  • Holland, J.H.: Adaptation in natural and artificial systems. University of Michigan Press, Ann Arbor (1975)

    Google Scholar 

  • Jeffreys, H.: An invariant form for the prior probability in estimation problems. Proc. R. Soc. Lond. A: Math. Phys. Sci. 186, 453–461 (1946)

    Article  MathSciNet  MATH  Google Scholar 

  • Karnopp, D.C.: Random search techniques for optimization problems. Automatica 1, 111–121 (1963)

    Article  Google Scholar 

  • Kiefer, J., Wolfowitz, J.: Stochastic estimation of a regression function. Ann. Math. Stat. 23, 462–466 (1952)

    Article  MathSciNet  MATH  Google Scholar 

  • Kolda, T.G., Lewis, R.M., Torczon, V.: Optimization by direct search: new perspectives on some classical and modern methods. SIAM Rev. 45, 385–482 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  • Kushner, H.J., Yin, G.G.: Stochastic approximation and recursive algorithms and applications. Springer, New York (2003)

    MATH  Google Scholar 

  • Ljung, L.: System identification – theory for the user. Prentice Hall PTR, Upper Saddle River (1999)

    Google Scholar 

  • Maryak, J.L., Chin, D.C.: Global random optimization by simultaneous perturbation stochastic approximation. IEEE Trans. Autom. Control 53, 780–783 (2008)

    Article  MathSciNet  Google Scholar 

  • Matyas, J.: Random optimization. Autom. Remote Control 26, 244–251 (1965)

    MathSciNet  MATH  Google Scholar 

  • Michalewicz, Z.: Genetic algorithms + data structures = evolution programs. Springer, New York (1996)

    MATH  Google Scholar 

  • Michalewicz, Z., Fogel, D.B.: How to solve it: modern heuristics. Springer, New York (2000)

    MATH  Google Scholar 

  • Mitchell, M.: An introduction to genetic algorithms. MIT Press, Cambridge (1996)

    Google Scholar 

  • Nelder, J. A., Mead, R., A simplex method for function minimization. Comput. J., 7, 308–313 (1965)

    Article  MATH  Google Scholar 

  • Pflug, G.C.h.: Optimization of stochastic models: the interface between simulation and optimization. Kluwer Academic, Boston (1996)

    Google Scholar 

  • Reeves, C.R., Rowe, J.E.: Genetic algorithms – principles and perspectives: a guide to GA theory. Kluwer Academic, Boston (2003)

    MATH  Google Scholar 

  • Robbins, H., Monro, S.: A stochastic approximation method. Ann. Math. Stat. 22, 400–407 (1951)

    Article  MathSciNet  MATH  Google Scholar 

  • Rudolph, G.: Convergence analysis of Canonical genetic algorithms. IEEE Trans. Neural Netw. 5, 96–101 (1994)

    Article  Google Scholar 

  • Rudolph, G.: Convergence properties of evolutionary algorithms. Kovac, Hamburg (1997)

    Google Scholar 

  • Ruppert, D.: Stochastic approximation. In: Ghosh, B.K., Sen, P.K. (eds.) Handbook of sequential analysis, pp. 503–529. Dekker, New York (1991)

    Google Scholar 

  • Schwefel, H.P.: Evolution and optimum seeking. Wiley, New York (1995)

    Google Scholar 

  • Solis, F.J., Wets, J.B.: Minimization by random search techniques. Math. Oper. Res. 6, 19–30 (1981)

    Article  MathSciNet  MATH  Google Scholar 

  • Spall, J.C.: Multivariate stochastic approximation using a simultaneous perturbation gradient approximation. IEEE Trans. Autom. Control 37, 332–341 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  • Spall, J.C.: Adaptive stochastic approximation by the simultaneous perturbation method. IEEE Trans. Autom. Control 45, 1839–1853 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  • Spall, J.C.: Introduction to stochastic search and optimization: estimation, simulation, and control. Wiley, Hoboken (2003)

    MATH  Google Scholar 

  • Spall, J.C.: Monte Carlo computation of the Fisher information matrix in nonstandard settings. J. Comput. Graph. Stat. 14(4), 889–909 (2005)

    Article  MathSciNet  Google Scholar 

  • Spall, J.C.: Feedback and weighting mechanisms for improving Jacobian estimates in the adaptive simultaneous perturbation algorithm. IEEE Trans. Autom. Control 54(6), 1216–1229 (2009)

    Article  MathSciNet  Google Scholar 

  • Spall, J.C.: Factorial design for choosing input values in experimentation: generating informative data for system identification. IEEE Control Syst. Mag. 30(5), 38–53 (2010)

    Article  MathSciNet  Google Scholar 

  • Stark, D.R., Spall, J.C.: Rate of convergence in evolutionary computation. In: Proceedings of the American Control Conference, Denver, 4–6 June 2003

    Google Scholar 

  • Suzuki, J.: A Markov chain analysis on simple genetic algorithms. IEEE Trans. Syst. Man. Cybern. 25, 655–659 (1995)

    Article  Google Scholar 

  • Vose, M.: The simple genetic algorithm. MIT Press, Cambridge (1999)

    MATH  Google Scholar 

  • Wang, Q., Spall, J.C.: Discrete simultaneous perturbation stochastic approximation on loss functions with noisy measurements. In: Proceedings of the American Control Conference, San Francisco, 29 June–1 July 2011, pp. 4520–4525 (paper FrB10.3) (2011)

    Google Scholar 

  • Wolpert, D.H., Macready, W.G.: No free lunch theorems for optimization. IEEE Trans. Evol. Comput. 1, 67–82 (1997)

    Article  Google Scholar 

  • Yakowitz, S.J., Fisher, L.: On sequential search for the maximum of an unknown function. J. Math. Anal, Appl. 41, 234–259 (1973)

    Google Scholar 

  • Yin, G.: Rates of convergence for a class of global stochastic optimization algorithms. SIAM J. Optim. 10, 99–120 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  • Zhigljavsky, A.A.: Theory of global random search. Kluwer Academic, Boston (1991)

    Book  Google Scholar 

Download references

Acknowledgements

I appreciate the helpful comments of Dr. Stacy Hill on a draft version of this chapter. Funding was provided by the U.S. Navy (contract N00024-03-D-6606) and the JHU/APL Independent Research and Development (IRAD) Program. Selected parts of this article have been reprinted, by permission, from J.C. Spall, Introduction to Stochastic Search and Optimization, ©2003 by John Wiley and Sons, Inc.

Author information

Authors and Affiliations

  1. Applied Physics Laboratory Laurel, The Johns Hopkins University, Baltimore, MD, USA

    James C. Spall

Authors
  1. James C. Spall
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to James C. Spall .

Editor information

Editors and Affiliations

  1. Dept. Computational & Data, Sciences, George Mason University, University Drive 4400, Fairfax, 22030-4444, Virginia, USA

    James E. Gentle

  2. L.v.Bortkiewicz Chair of Statistics, C.A.S.E. Centre f. Appl. Stat. & Econ., Humboldt-Universität zu Berlin, Unter den Linden 6, Berlin, 10099, Germany

    Wolfgang Karl Härdle

  3. Dept. Socioinformation, Okayama University, Ridai-cho 1-1, Okayama, 700-0005, Japan

    Yuichi Mori

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Spall, J.C. (2012). Stochastic Optimization. In: Gentle, J., Härdle, W., Mori, Y. (eds) Handbook of Computational Statistics. Springer Handbooks of Computational Statistics. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-21551-3_7

Download citation

Publish with us

Policies and ethics

Search

Navigation