Skip to main content

Stochastic Optimization

Part of the Springer Handbooks of Computational Statistics book series (SHCS)

Abstract

Stochastic optimization algorithms have been growing rapidly in popularityover the last decade or two, with a number of methods now becomingindustry standard approaches for solving challenging optimization problems.This chapter provides a synopsis of some of the critical issues associatedwith stochastic optimization and a gives a summary of several popularalgorithms. Much more complete discussions are available in the indicatedreferences.

Keywords

  • Loss Function
  • Stochastic Optimization
  • Random Search
  • Stochastic Approximation
  • Fisher Information Matrix

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-642-21551-3_7
  • Chapter length: 29 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   269.00
Price excludes VAT (USA)
  • ISBN: 978-3-642-21551-3
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Fig. 7.1
Fig. 7.2
Fig. 7.3
Fig. 7.4

References

  • Arsham, H.: Techniques for Monte Carlo optimizing. Monte Carlo Methods Appl. 4, 181–229 (1998)

    MathSciNet  MATH  CrossRef  Google Scholar 

  • Baba, N., Shoman, T., Sawaragi, Y.: A modified convergence theorem for a random optimization method. Inf. Sci. 13, 159–166 (1977)

    MathSciNet  MATH  CrossRef  Google Scholar 

  • Bazaraa, M.S., Sherali, H.D., Shetty, C.M.: Nonlinear programming: theory and algorithms. Wiley, New York (1993)

    MATH  Google Scholar 

  • Blum, J.R.: Multidimensional stochastic approximation methods. Ann. Math. Stat. 25, 737–744 (1954)

    MATH  CrossRef  Google Scholar 

  • Box, G.E.P.: Evolutionary operation: a method for increasing industrial productivity. J. R. Stat. Soc. Ser. C: Appl. Stat. 6, 81–101 (1957)

    Google Scholar 

  • Cochran, J.J. (ed.): Encyclopedia of operations research and management science. Wiley, Hoboken (2011)

    Google Scholar 

  • Das, S., Spall, J.C., Ghanem, R.: Efficient Monte Carlo computation of Fisher information matrix using prior information. Comput. Stat. Data Anal. 54(2), 272–289 (2010)

    MathSciNet  MATH  CrossRef  Google Scholar 

  • De Jong, K.A.: An analysis of the behavior of a class of genetic adaptive systems. Ph.D. dissertation, University of Michigan (University Microfilms no. 76–9381) (1975)

    Google Scholar 

  • Dippon, J., Renz, J.: Weighted means in stochastic approximation of minima. SIAM J. Contr. Optim. 35, 1811–1827 (1997)

    MathSciNet  MATH  CrossRef  Google Scholar 

  • Fabian, V.: Stochastic approximation. In: Rustigi, J.S. (ed.) Optimizing methods in statistics, pp. 439–470. Academic, New York (1971)

    Google Scholar 

  • Fogel, D.B.: Evolutionary computation: toward a new philosophy of machine intelligence. IEEE, Piscataway (2000)

    Google Scholar 

  • Fouskakis, D., Draper, D.: Stochastic optimization: a review. Int. Stat. Rev. 70, 315–349 (2002)

    MATH  CrossRef  Google Scholar 

  • Fu, M.C.: Optimization for simulation: theory vs. practice. INFORMS J. Comput. 14, 192–227 (2002)

    Google Scholar 

  • Gentle, J.E.: Random number generation and Monte Carlo methods. Springer, New York (2003)

    MATH  Google Scholar 

  • Gerencsér, L.: Convergence rate of moments in stochastic approximation with simultaneous perturbation gradient approximation and resetting. IEEE Trans. Autom. Control 44, 894–905 (1999)

    MATH  CrossRef  Google Scholar 

  • Gerencsér, L., Hill, S.D., Vágó, Z.: Optimization over discrete sets via SPSA. In: Proceedings of the IEEE Conference on Decision and Control, Phoenix, 7–10 December 1999, 1791–1795

    Google Scholar 

  • Goldberg, D.E.: Genetic algorithms in search, optimization, and machine learning. Addison-Wesley, Reading (1989)

    MATH  Google Scholar 

  • Goldberg, D.E.: The design of innovation: lessons from and for competent genetic algorithms. Kluwer Academic, Boston (2002)

    MATH  Google Scholar 

  • Gosavi, A.: Simulation-based optimization: parametric optimization techniques and reinforcement learning. Kluwer Academic, Boston (2003)

    MATH  Google Scholar 

  • Hill, S.D.: Discrete stochastic approximation with application to resource allocation. Johns Hopkins APL Tech. Dig. 26, 15–21 (2005)

    Google Scholar 

  • Holland, J.H.: Adaptation in natural and artificial systems. University of Michigan Press, Ann Arbor (1975)

    Google Scholar 

  • Jeffreys, H.: An invariant form for the prior probability in estimation problems. Proc. R. Soc. Lond. A: Math. Phys. Sci. 186, 453–461 (1946)

    MathSciNet  MATH  CrossRef  Google Scholar 

  • Karnopp, D.C.: Random search techniques for optimization problems. Automatica 1, 111–121 (1963)

    CrossRef  Google Scholar 

  • Kiefer, J., Wolfowitz, J.: Stochastic estimation of a regression function. Ann. Math. Stat. 23, 462–466 (1952)

    MathSciNet  MATH  CrossRef  Google Scholar 

  • Kolda, T.G., Lewis, R.M., Torczon, V.: Optimization by direct search: new perspectives on some classical and modern methods. SIAM Rev. 45, 385–482 (2003)

    MathSciNet  MATH  CrossRef  Google Scholar 

  • Kushner, H.J., Yin, G.G.: Stochastic approximation and recursive algorithms and applications. Springer, New York (2003)

    MATH  Google Scholar 

  • Ljung, L.: System identification – theory for the user. Prentice Hall PTR, Upper Saddle River (1999)

    Google Scholar 

  • Maryak, J.L., Chin, D.C.: Global random optimization by simultaneous perturbation stochastic approximation. IEEE Trans. Autom. Control 53, 780–783 (2008)

    MathSciNet  CrossRef  Google Scholar 

  • Matyas, J.: Random optimization. Autom. Remote Control 26, 244–251 (1965)

    MathSciNet  MATH  Google Scholar 

  • Michalewicz, Z.: Genetic algorithms + data structures = evolution programs. Springer, New York (1996)

    MATH  Google Scholar 

  • Michalewicz, Z., Fogel, D.B.: How to solve it: modern heuristics. Springer, New York (2000)

    MATH  Google Scholar 

  • Mitchell, M.: An introduction to genetic algorithms. MIT Press, Cambridge (1996)

    Google Scholar 

  • Nelder, J. A., Mead, R., A simplex method for function minimization. Comput. J., 7, 308–313 (1965)

    MATH  CrossRef  Google Scholar 

  • Pflug, G.C.h.: Optimization of stochastic models: the interface between simulation and optimization. Kluwer Academic, Boston (1996)

    Google Scholar 

  • Reeves, C.R., Rowe, J.E.: Genetic algorithms – principles and perspectives: a guide to GA theory. Kluwer Academic, Boston (2003)

    MATH  Google Scholar 

  • Robbins, H., Monro, S.: A stochastic approximation method. Ann. Math. Stat. 22, 400–407 (1951)

    MathSciNet  MATH  CrossRef  Google Scholar 

  • Rudolph, G.: Convergence analysis of Canonical genetic algorithms. IEEE Trans. Neural Netw. 5, 96–101 (1994)

    CrossRef  Google Scholar 

  • Rudolph, G.: Convergence properties of evolutionary algorithms. Kovac, Hamburg (1997)

    Google Scholar 

  • Ruppert, D.: Stochastic approximation. In: Ghosh, B.K., Sen, P.K. (eds.) Handbook of sequential analysis, pp. 503–529. Dekker, New York (1991)

    Google Scholar 

  • Schwefel, H.P.: Evolution and optimum seeking. Wiley, New York (1995)

    Google Scholar 

  • Solis, F.J., Wets, J.B.: Minimization by random search techniques. Math. Oper. Res. 6, 19–30 (1981)

    MathSciNet  MATH  CrossRef  Google Scholar 

  • Spall, J.C.: Multivariate stochastic approximation using a simultaneous perturbation gradient approximation. IEEE Trans. Autom. Control 37, 332–341 (1992)

    MathSciNet  MATH  CrossRef  Google Scholar 

  • Spall, J.C.: Adaptive stochastic approximation by the simultaneous perturbation method. IEEE Trans. Autom. Control 45, 1839–1853 (2000)

    MathSciNet  MATH  CrossRef  Google Scholar 

  • Spall, J.C.: Introduction to stochastic search and optimization: estimation, simulation, and control. Wiley, Hoboken (2003)

    MATH  Google Scholar 

  • Spall, J.C.: Monte Carlo computation of the Fisher information matrix in nonstandard settings. J. Comput. Graph. Stat. 14(4), 889–909 (2005)

    MathSciNet  CrossRef  Google Scholar 

  • Spall, J.C.: Feedback and weighting mechanisms for improving Jacobian estimates in the adaptive simultaneous perturbation algorithm. IEEE Trans. Autom. Control 54(6), 1216–1229 (2009)

    MathSciNet  CrossRef  Google Scholar 

  • Spall, J.C.: Factorial design for choosing input values in experimentation: generating informative data for system identification. IEEE Control Syst. Mag. 30(5), 38–53 (2010)

    MathSciNet  CrossRef  Google Scholar 

  • Stark, D.R., Spall, J.C.: Rate of convergence in evolutionary computation. In: Proceedings of the American Control Conference, Denver, 4–6 June 2003

    Google Scholar 

  • Suzuki, J.: A Markov chain analysis on simple genetic algorithms. IEEE Trans. Syst. Man. Cybern. 25, 655–659 (1995)

    CrossRef  Google Scholar 

  • Vose, M.: The simple genetic algorithm. MIT Press, Cambridge (1999)

    MATH  Google Scholar 

  • Wang, Q., Spall, J.C.: Discrete simultaneous perturbation stochastic approximation on loss functions with noisy measurements. In: Proceedings of the American Control Conference, San Francisco, 29 June–1 July 2011, pp. 4520–4525 (paper FrB10.3) (2011)

    Google Scholar 

  • Wolpert, D.H., Macready, W.G.: No free lunch theorems for optimization. IEEE Trans. Evol. Comput. 1, 67–82 (1997)

    CrossRef  Google Scholar 

  • Yakowitz, S.J., Fisher, L.: On sequential search for the maximum of an unknown function. J. Math. Anal, Appl. 41, 234–259 (1973)

    Google Scholar 

  • Yin, G.: Rates of convergence for a class of global stochastic optimization algorithms. SIAM J. Optim. 10, 99–120 (1999)

    MathSciNet  MATH  CrossRef  Google Scholar 

  • Zhigljavsky, A.A.: Theory of global random search. Kluwer Academic, Boston (1991)

    CrossRef  Google Scholar 

Download references

Acknowledgements

I appreciate the helpful comments of Dr. Stacy Hill on a draft version of this chapter. Funding was provided by the U.S. Navy (contract N00024-03-D-6606) and the JHU/APL Independent Research and Development (IRAD) Program. Selected parts of this article have been reprinted, by permission, from J.C. Spall, Introduction to Stochastic Search and Optimization, ©2003 by John Wiley and Sons, Inc.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to James C. Spall .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Spall, J.C. (2012). Stochastic Optimization. In: Gentle, J., Härdle, W., Mori, Y. (eds) Handbook of Computational Statistics. Springer Handbooks of Computational Statistics. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-21551-3_7

Download citation