A Survey of Some Model-Based Methods for Global Optimization

  • Jiaqiao Hu
  • Yongqiang Wang
  • Enlu Zhou
  • Michael C. Fu
  • Steven I. MarcusEmail author
Part of the Systems & Control: Foundations & Applications book series (SCFA)


We review some recent developments of a class of random search methods: model-based methods for global optimization problems. Probability models are used to guide the construction of candidate solutions in model-based methods, which makes them easy to implement and applicable to problems with little structure. We have developed various frameworks for model-based algorithms to guide the updating of probabilistic models and to facilitate convergence proofs. Specific methods covered in this survey include model reference adaptive search, a particle-filtering approach, an evolutionary games approach, and a stochastic approximation-based gradient approach.


Candidate Solution Global Optimal Solution Evolutionary Game Stochastic Approximation Replicator Dynamic 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This work was supported in part by the National Science Foundation (NSF) under Grants CNS-0926194, CMMI-0856256, CMMI-0900332, CMMI-1130273, CMMI-1130761, EECS-0901543, and by the Air Force Office of Scientific Research (AFOSR) under Grant FA9550-10-1-0340.


  1. 1.
    Arulampalam, S., Maskell, S., Gordon, N.J., Clapp, T.: A tutorial on particle filters for on-line non-linear/non-Gaussian Bayesian tracking. IEEE Transactions on Signal Processing 50(2), 174–188 (2002)CrossRefGoogle Scholar
  2. 2.
    Benaim, M.: A dynamical system approach to stochastic approximations. SIAM Journal on Control and Optimization 34, 437–472 (1996)MathSciNetzbMATHCrossRefGoogle Scholar
  3. 3.
    Billingsley, P.: Convergence of Probability Measures. John Wiley & Sons, Inc., New York (1999)zbMATHCrossRefGoogle Scholar
  4. 4.
    Borkar, V.: Stochastic approximation: a dynamical systems viewpoint. Cambridge University Press; New Delhi: Hindustan Book Agency (2008)Google Scholar
  5. 5.
    Budhiraja, A., Chen, L., Lee, C.: A survey of numerical methods for nonlinear filtering problems. Physica D: Nonlinear Phenomena 230, 27–36 (2007)MathSciNetzbMATHCrossRefGoogle Scholar
  6. 6.
    Chopin, N.: Central limit theorem for sequential Monte Carlo and its applications to Bayesian inference. The Annals of Statistics 32(6), 2385–2411 (2004)MathSciNetzbMATHCrossRefGoogle Scholar
  7. 7.
    Costa, A., Jones, O., Kroese, D.: Convergence properties of the cross-entropy method for discrete optimization. Operations Research Letters (35), 573–580 (2007)MathSciNetzbMATHCrossRefGoogle Scholar
  8. 8.
    Davis, M.H.A., Marcus, S.I.: An introduction to nonlinear filtering. The Mathematics of Filtering and Identification and Applications. Amsterdam, The Netherlands, Reidel (1981)Google Scholar
  9. 9.
    Dorigo, M., Gambardella, L.M.: Ant colony system: A cooperative learning approach to the traveling salesman problem. IEEE Trans. on Evolutionary Computation 1, 53–66 (1997)CrossRefGoogle Scholar
  10. 10.
    Doucet, A., deFreitas, J.F.G., Gordon, N.J. (eds.): Sequential Monte Carlo Methods In Practice. Springer, New York (2001)Google Scholar
  11. 11.
    Doucet, A., Johansen, A.M.: A tutorial on particle filtering and smoothing: Fifteen years later. Handbook of Nonlinear Filtering. Cambridge University Press, Cambridge (2009)Google Scholar
  12. 12.
    Eiben, A., Smith, J.: Introduction to Evolutionary Computing. Natural Computing Series, Springer (2003)zbMATHGoogle Scholar
  13. 13.
    Gilks, W., Berzuini, C.: Following a moving target - Monte Carlo inference for dynamic Bayesian models. Journal of the Royal Statistical Society 63(1), 127–146 (2001)MathSciNetzbMATHCrossRefGoogle Scholar
  14. 14.
    Gland, F.L., Oudjane, N.: Stability and uniform approximation of nonlinear filters using the Hilbert metric and application to particle filter. The Annals of Applied Probability 14(1), 144–187 (2004)MathSciNetzbMATHCrossRefGoogle Scholar
  15. 15.
    Glover, F.W.: Tabu search: A tutorial. Interfaces 20, 74–94 (1990)Google Scholar
  16. 16.
    Goldberg, D.E.: Genetic Algorithms in Search, Optimization, and Machine Learning. Addison-Wesley, Boston, MA (1989)zbMATHGoogle Scholar
  17. 17.
    Homem-De-Mello, T.: A study on the cross-entropy method for rare-event probability estimation. INFORMS Journal on Computing 19, 381–394 (2007)MathSciNetzbMATHCrossRefGoogle Scholar
  18. 18.
    Hu, J., Fu, M.C., Marcus, S.I.: A model reference adaptive search method for global optimization. Operations Research 55(3), 549–568 (2007)MathSciNetzbMATHCrossRefGoogle Scholar
  19. 19.
    Hu, J., Hu, P.: On the performance of the cross-entropy method. In: Proceedings of the 2009 Winter Simulation Conference, pp. 459–468 (2009)Google Scholar
  20. 20.
    Hu, J., Hu, P.: Annealing adaptive search, cross-entropy, and stochastic approximation in global optimization. Naval Research Logistics 58, 457–477 (2011)MathSciNetzbMATHCrossRefGoogle Scholar
  21. 21.
    Hu, J., Hu, P., Chang, H.: A stochastic approximation framework for a class of randomized optimization algorithms. IEEE Transactions on Automatic Control (forthcoming) (2012)Google Scholar
  22. 22.
    Jacobson, S., Sullivan, K., Johnson, A.: Discrete manufacturing process design optimization using computer simulation and generalized hill climbing algorithms. Engineering Optimization 31, 247–260 (1998)CrossRefGoogle Scholar
  23. 23.
    Johnson, A., Jacobson, S.: A class of convergent generalized hill climbing algorithms. Applied Mathematics and Computation 125, 359–373 (2001)MathSciNetCrossRefGoogle Scholar
  24. 24.
    Kennedy, J., Eberhart, R.: Particle swarm optimization. In Proceedings of IEEE International Conference on Neural Networks. IEEE Press, Piscataway, NJ, pp. 1942–1948 (1995)Google Scholar
  25. 25.
    Kirkpatrick, S., Gelatt, C.D., Vecchi, M.P.: Optimization by simulated annealing. Science 220, 671–680 (1983)MathSciNetzbMATHGoogle Scholar
  26. 26.
    Kushner, H.J., Clark, D.S.: Stochastic Approximation Methods for Constrained and Unconstrained Systems and Applications. Springer-Verlag, New York (1978)CrossRefGoogle Scholar
  27. 27.
    Larrañaga, P., Lozano, J. (eds.): Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation. Kluwer Academic Publisher, Boston, MA (2002)zbMATHGoogle Scholar
  28. 28.
    Musso, C., Oudjane, N., Gland, F.L.: Sequential Monte Carlo Methods in Practice. Springer-Verlag, New York (2001)Google Scholar
  29. 29.
    Oechssler, J., Riedel, F.: On the dynamics foundation of evolutionary stability in continuous models. Journal of Economic Theory 107, 223–252 (2002)MathSciNetzbMATHCrossRefGoogle Scholar
  30. 30.
    Robbins, H., Monro, S.: A stochastic approximation method. Annals of Mathematical Statistics 22, 400–407 (1951)MathSciNetzbMATHCrossRefGoogle Scholar
  31. 31.
    Romeijn, H., Smith, R.: Simulated annealing and adaptive search in global optimization. Probability in the Engineering and Informational Sciences 8, 571–590 (1994)CrossRefGoogle Scholar
  32. 32.
    Rubinstein, R.Y.: Optimization of computer simulation models with rare events. European Journal of Operations Research 99, 89–112 (1997)CrossRefGoogle Scholar
  33. 33.
    Rubinstein, R.Y.: The cross-entropy method for combinatorial and continuous optimization. Methodology and Computing in Applied Probability 2, 127–190 (1999)CrossRefGoogle Scholar
  34. 34.
    Rubinstein, R.Y., Kroese, D.P.: The Cross-Entropy Method: A Unified Approach to Combinatorial Optimization, Monte-Carlo Simulation, and Machine Learning. Springer, New York, NY (2004)zbMATHGoogle Scholar
  35. 35.
    Shi, L., Olafsson, S.: Nested partitions method for global optimization. Operations Research 48(3), 390–407 (2000)MathSciNetzbMATHCrossRefGoogle Scholar
  36. 36.
    Srinivas, M., Patnaik, L.M.: Genetic algorithms: A survey. IEEE Computer 27, 17–26 (1994)Google Scholar
  37. 37.
    Wang Y., Fu, M.C., Marcus, S.I.: Model-based evolutionary optimization. In Proceedings of the 2010 Winter Simulation Conference. IEEE Press, Piscataway, NJ, pp. 1199–1210 (2010)Google Scholar
  38. 38.
    Wang, Y., Fu, M.C., Marcus, S.I.: An evolutionary game approach for model-based optimization. Working paper (2011)Google Scholar
  39. 39.
    Wolpert, D.H. Finding bounded rational equilibria part i: Iterative focusing. In Proceedings of the Eleventh International Symposium on Dynamic Games and Applications, T. Vincent (Editor), Tucson AZ, USA (2004)Google Scholar
  40. 40.
    Zabinsky, Z., Smith, R., McDonald, J., Romeijn, H., Kaufman, D.: Improving hit-and-run for global optimization. Journal of Global Optimization 3, 171–192 (1993)MathSciNetzbMATHCrossRefGoogle Scholar
  41. 41.
    Zabinsky, Z.B.: Stochastic Adaptive Search for Global Optimization. Kluwer, The Netherlands (2003)zbMATHGoogle Scholar
  42. 42.
    Zhang, Q., Mühlenbein, H.: On the convergence of a class of estimation of distribution algorithm. IEEE Trans. on Evolutionary Computation 8, 127–136 (2004)CrossRefGoogle Scholar
  43. 43.
    Zhou, E., Fu, M.C., Marcus, S.I.: A particle filtering framework for randomized optimization algorithms. In Proceedings of the 2008 Winter Simulation Conference. IEEE Press, Piscataway, NJ, pp. 647–654 (2008)Google Scholar
  44. 44.
    Zhou, E., Fu, M.C., Marcus, S.I.: Solving continuous-state POMDPs via density projection. IEEE Transactions on Automatic Control 55(5), 1101–1116 (2010)MathSciNetCrossRefGoogle Scholar
  45. 45.
    Zhou, E., Fu, M.C., Marcus, S.I.: Particle filtering framework for a class of randomized optimization algorithms. Under review (2012)Google Scholar
  46. 46.
    Zlochin, M., Birattari, M., Meuleau, N., Dorigo, M.: Model-based search for combinatorial optimization: A critical survey. Annals of Operations Research 131, 373–395 (2004)MathSciNetzbMATHCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  • Jiaqiao Hu
    • 1
  • Yongqiang Wang
    • 2
  • Enlu Zhou
    • 3
  • Michael C. Fu
    • 4
  • Steven I. Marcus
    • 2
    Email author
  1. 1.Department of Applied Mathematics and StatisticsState University at Stony BrookStony BrookUSA
  2. 2.Department of Electrical and Computer Engineering & Institute for Systems ResearchUniversity of MarylandCollege ParkUSA
  3. 3.Department of Industrial and Enterprise Systems EngineeringUniversity of Illinois at Urbana-ChampaignChicagoUSA
  4. 4.The Robert H. Smith School of Business & Institute for Systems ResearchUniversity of MarylandCollege ParkUSA

Personalised recommendations