Skip to main content
SpringerLink
Log in

Stochastic Search in Metaheuristics

  • Chapter
  • First Online:
Handbook of Metaheuristics

Part of the book series: International Series in Operations Research & Management Science ((ISOR,volume 146))

  • 11k Accesses

Abstract

Stochastic search is a key mechanism underlying many metaheuristics. The chapter starts with the presentation of a general framework algorithm in the form of a stochastic search process that contains a large variety of familiar metaheuristic techniques as special cases. Based on this unified view, questions concerning convergence and runtime are discussed on the level of a theoretical analysis. Concrete examples from diverse metaheuristic fields are given. In connection with runtime results, important topics as instance difficulty, phase transitions, parameter choice, No-Free-Lunch theorems, or fitness landscape analysis are addressed. Furthermore, a short sketch of the theory of black-box optimization is given, and generalizations of results to stochastic search under noise are outlined.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Notes

  1. 1.

    Since g and h do not depend on the iteration counter t, the Markov process is homogeneous. Dependence on t can easily be modeled by adding t as a component to the memory M t .

  2. 2.

    Elitism as a mechanism ensuring convergence of a GA has already been analyzed in [39], which appears to be the first paper on GA convergence.

  3. 3.

    To ask, say, for the expected time until first hitting an optimal solution without being sure that the optimum will be reached, is as meaningless as to ask “How much training time would it take in the average for a randomly selected person to win an olympic gold medal?” Also by being content with an approximate solution of a certain minimum quality (call it the “silver medal”) instead of the optimal solution, one does not escape this difficulty.

  4. 4.

    The relevance of this distribution in the field of stochastic search is also underlined by the fact that one of the oldest general-purpose stochastic search techniques, namely SA, approximates at each fixed temperature level T the corresponding Boltzmann distribution.

  5. 5.

    There seem to be close relations between NFL theorems and the well-known philosophical induction problem that plays also a role in AI approaches to inductive reasoning. Suppose that the function evaluation of \(f(x)\) in the solution \(x \in S\) is not done by an algorithmic computation, but rather by the observation of some real-world system (say, x is a control vector for a chemical plant, and \(f(x)\) is the observed value of an outcome variable). Then the “NFL insight” that there is no logical argument why the observation of \(f(x_1),\ldots,f(x_{t-1})\) for some sample solutions \(x_1,\ldots,x_{t-1}\) should provide any information on \(f(x_t)\) for the sample solution \(x_t \notin \{x_1,\ldots,x_{t-1}\}\), basically amounts to the intriguing claim by David Hume that we do not have any logical justification for the step called “inductive conclusion,” although this step is indispensable in science as in everyday life.

  6. 6.

    Note that both EA and ACO algorithms typically solve this problem in \(O(n \log n)\) time [11, 37], which differs from the lower bound by a factor of order \(O((\log n)^2)\). This overhead may partly be explained by the effort for re-sampling already visited solutions.

  7. 7.

    For approaches using “white-box” mathematical programming techniques such as the Integer L-Shaped Method, see, e.g., [23].

References

  1. Achlioptas, D., Naor, A., Peres, Y.: Rigorous location of phase transitions in hard optimization problems. Nature 435, 759–764 (2005)

    Article  Google Scholar 

  2. A simulated annealing algorithm with constant temperature for discrete stochastic optimization. Manage. Sci. 45, 748–764 (1999)

    Google Scholar 

  3. Barbosa, V.C., Ferreira, R.G.: On the phase transitions of graph coloring and independent sets. Physika A 343, 401–423 (2004)

    Google Scholar 

  4. Bianchi, L., Dorigo, M., Gambardella, L.M., Gutjahr, W.J.: A survey on metaheuristics for stochastic combinatorial optimization. To appear In: Natural Computing 8, 239–287 (2009)

    Google Scholar 

  5. Birattari, M., Balaprakash, P., Dorigo, M.: The ACO/FRACE algorithm for combinatorial optimization under uncertainty. In: Doerner K. et al. (ed.) Metaheuristics – Progress in Complex Systems Optimization, Springer: Berlin, Germany (2006)

    Google Scholar 

  6. Borenstein, Y., Poli, R.: Information Perspective of Optimization. Proceedings of 9th Conference on Parallel Problem Solving from Nature, Springer LNCS, vol. 4193, pp. 102–111. Berlin Heidelberg (2006)

    Google Scholar 

  7. Borenstein, Y. Poli, R.: Structure and Metaheuristics. Proceedings of Genetic and Evolutionary Computation Conference ’06, pp. 1087–1093 (2006)

    Google Scholar 

  8. Borisovsky, P.A., Eremeev, A.V.: A study on performance of the (1+1)-evolutionary algorithm. In: Proceedings of Foundations of Genetic Algorithms, vol. 7, pp. 271–287. Morgan Kaufmann, San Francisco (2003)

    Google Scholar 

  9. Cheesman, P., Kenafsky, B., Taylor, W.M.: Where the really hard problems are. In: Morgan Kaufmann (ed.) Proceedings of IJCAI ’91, pp. 331–337 (1991)

    Google Scholar 

  10. Droste, S., Jansen, T., Wegener, I.: Perhaps not a free lunch but at least a free appetizer. Proceedings of Genetic and Evolutionary Computation Conference ’99, Orlando, FL, USA pp. 833–839 (1999)

    Google Scholar 

  11. Droste, S., Jansen, T., Wegener, I.: On the analysis of the (1+1) evolutionary algorithm. Theor. Comput. Sci. 276, 51–81 (2002)

    Article  Google Scholar 

  12. Droste, S., Jansen, T., Wegener, I.: Upper and lower bounds for randomized search heuristics in black-box optimization. Theory Comput. Syst. 39 525–544 (2006)

    Article  Google Scholar 

  13. Dorigo, M., Maniezzo, V., Colorni, A.: Ant System: optimization by a colony of cooperating agents. IEEE Trans. Syst, Man Cybern 26, 1–13 (1996)

    Google Scholar 

  14. English, T.: Optimization is easy and learning is hard in the typical function. Proceedings of Congress in Evolutionary Computation ’00, La Jolla, USA pp. 924–931 (2000)

    Google Scholar 

  15. English, T.: On the structure of sequential search: beyond “no free lunch”. Proc. EvoCOP ’04, Springer LNCS, Coimbra, Portugal 3004, 95–103 (2004)

    Google Scholar 

  16. Eremeev, A.V., Reeves, C.R.: On confidence intervals for the number of local optima. Applications of Evolutionary Computing, Springer LNCS, Berlin Heidelberg vol. 2611, pp. 224–235 (2003)

    Google Scholar 

  17. Ferreira, F.F., Fontanari, J.F.: Probabilistic analysis of the number partitioning problem. J. Phys. A: Math. Gen. 31, 3417–3428 (1998)

    Article  Google Scholar 

  18. Fischetti, M., Lodi, A.: Local branching. Mathematical Programming Ser. B 98, 23–47 (2003)

    Article  Google Scholar 

  19. Friedrich, T., He, J., Hebbinghaus, N., Neumann, F., Witt, C.: Approximating covering problems by randomized search heuristics using multi-objective models. Proceedings of 9th Annual Conference on Genetic and Evolutionary Computation, pp. 797–804 (2007)

    Google Scholar 

  20. Garnier, J., Kalel, L., Schoenauer, M.: Rigorous hitting times for binary mutations. Evol. Comput. 7, 45–68 (1999)

    Article  Google Scholar 

  21. Garnier, J., Kallel, L.: Efficiency of local search with multiple local optima. SIAM J. Discrete Math. 15, 122–141 (2002)

    Article  Google Scholar 

  22. Gelfand, S.B., Mitter, S.K.: Simulated annealing with noisy or imprecise measurements. J. Optim. Theor. Appl. 69, 49–62 (1989)

    Article  Google Scholar 

  23. Gendreau, M., Laporte, G., Seguin, R.: An exact algorithm for the vehicle routing problem with stochastic demands and customers. Transport. Sci. 29, 143–155 (1995)

    Article  Google Scholar 

  24. Gent, I.P., Walsh, T.: Analysis of heuristics for number partitioning. Comput. Intell. 14, 430–450 (1998)

    Article  Google Scholar 

  25. Giel, O., Wegener, I.: Evolutionary algorithms and the maximum matching problem. Proceedings of 20th Annual Symposium on Theoretical Aspects of Computer Science, Seattle, Washington, USA pp. 415–426 (2003)

    Google Scholar 

  26. González, C., Lozano, J.A., Larrañaga, P.: Analyzing the PBIL algorithm by means of discrete dynamical systems. Complex Syst. 11, 1–15 (1997)

    Google Scholar 

  27. González, C., Lozano, J.A., Larrañaga, P.: Mathematical modelling of discrete estimation of distribution algorithms. In: Larrañaga et al. (eds.) Estimation of Distribution Algorithms, A New Tool for Evolutionary Computation, Kluwer, Dordrecht 147–163 (2002)

    Google Scholar 

  28. Gutjahr, W.J.: A graph–based ant system and its convergence. Future Gen. Comput. Syst. 16, 873–888 (2000)

    Article  Google Scholar 

  29. Gutjahr, W.J.: ACO algorithms with guaranteed convergence to the optimal solution. In. Process. Lett. 82, 145–153 (2002)

    Article  Google Scholar 

  30. Gutjahr, W.J.: A converging ACO algorithm for stochastic combinatorial optimization. Proceedings of 2nd Symposium on Stochastic Algorithms, Foundations and Applications, Springer LNCS, Berlin Heidelberg vol. 2827, pp. 10–25 (2003)

    Google Scholar 

  31. Gutjahr, W.J.: S-ACO: An ant-based approach to combinatorial optimization under uncertainty. Proceedings of 4nd Int. Workshop on Ant Colony Optimization and Swarm Intelligence, Springer LNCS, Berlin Heidelberg New York vol. 3172, pp. 238–249 (2004)

    Google Scholar 

  32. Gutjahr, W.J.: On the finite-time dynamics of ant colony optimization. Methodol. Comput. Appl. Probability 8, 105–133 (2006)

    Article  Google Scholar 

  33. Gutjahr, W.J.: Mathematical runtime analysis of ACO algorithms: survey on an emerging issue. Swarm Intell. 1, 59–79 (2007)

    Article  Google Scholar 

  34. Gutjahr, W.J.: First steps to the runtime complexity analysis of ant colony optimization. Comput. Oper. Res. 35, 2711–2727 (2008)

    Article  Google Scholar 

  35. Gutjahr, W.J., Katzensteiner, S., Reiter, P.: A VNS algorithm for noisy problems and its application to project portfolio analysis. Proceedings of SAGA 2007 (Stochastic Algorithms: Foundations and Applications), Springer LNCS, Berlin Heidelberg vol. 4665, pp. 93–104 (2007)

    Google Scholar 

  36. Gutjahr, W.J., Pflug, G.: Simulated annealing for noisy cost functions. J. Global Optim. 8, 1–13 (1996)

    Article  Google Scholar 

  37. Gutjahr, W.J., Sebastiani, G.: Runtime analysis of ant colony optimization with best-so-far reinforcement. Methodology and Computing in Applied Probability 10, 409–433 (2008)

    Article  Google Scholar 

  38. Hajek, B.: Cooling schedules for optimal annealing. Math. of Operat. Res. 13, 311–329 (1988)

    Article  Google Scholar 

  39. Hartl, R.F.: A global convergence proof for a class of genetic algorithms. Technical Report, University of Vienna (1990)

    Google Scholar 

  40. Hartmann, A.K., Barthel, W., Weigt, M.: Phase transition and finite-size scaling in the vertex-cover problem. Comput. Phys. Commn. 169, 234–237 (2005)

    Article  Google Scholar 

  41. He, J., Yao, X.: Drift analysis and average time complexity of evolutionary algorithms. Artifi. Intell. 127, 57–85 (2003)

    Article  Google Scholar 

  42. He, J., Yao, X.: Towards an analytic framework for analysing the computation time of evolutionary algorithms. Artif. Intell. 145, 59–97 (2003)

    Article  Google Scholar 

  43. He, J., Yao, X.: A study of drift analysis for estimating computation time of evolutionary algorithms. Nat. Comput. 3, 21–35 (2004)

    Article  Google Scholar 

  44. He, J., Yu, X.: Conditions for the convergence of evolutionary algorithms. J. Syst. Arch. 47, 601–612 (2001)

    Article  Google Scholar 

  45. Herroelen, W., De Reyck, B.: Phase transitions in project scheduling. J. Oper. Res. Soc. 50, 148–156 (1999)

    Google Scholar 

  46. Igel, C., Toussaint, M.: On classes of functions for which no free lunch results hold. Inf. Process. Lett. 86, 317–321 (2003)

    Article  Google Scholar 

  47. Igel, C., Toussaint, M.: A no-free-lunch theorem for non-uniform distributions of target functions. J. Math. Model. Algorithms 3, 313–322 (2004)

    Article  Google Scholar 

  48. Jacobson, S.H., Yücesan, E.: Analyzing the performance of generalized hill climbing algorithms. J. Heuristics 10, 387–405 (2004)

    Article  Google Scholar 

  49. Jansen, T., Wegener, I.: On the analysis of a dynamic evolutionary algorithm. J. Discrete Algorithms 4, 181–199 (2006)

    Article  Google Scholar 

  50. Jin, Y., Branke, J.: Evolutionary optimization in uncertain environments – a survey. IEEE Trans. Evol. Comput. 9, 303–317 (2005).

    Article  Google Scholar 

  51. Kennedy, J., Eberhart, R.C.: A discrete binary version of the particle swarm algorithm. Proceedings of the World Multiconference on Systemics, Cybernetics and Informatics, Orlando, FL, USA pp. 4104–4109 (1997)

    Google Scholar 

  52. Koehler, G.J.: Conditions that obviate the no-free-lunch theorems for optimization. Informs J. Comput. 19, 273–279 (2007)

    Article  Google Scholar 

  53. Ladret, V.: Asymptotic hitting time for a simple evolutionary model of protein folding. J. Appl. Probability 42, 39–51 (2005)

    Article  Google Scholar 

  54. Margolin, L.: On the convergence of the cross-entropy method. Ann. Oper. Res. 134, 201–214 (2005)

    Article  Google Scholar 

  55. Martin, O.C., Monasson, R., Zecchina, R.: Statistical mechanics methods and phase transitions in optimization problems. Theor. Compu. Sci. 265, 3–67 (2001)

    Article  Google Scholar 

  56. Merelo, J.-J., Cotta, C.: Building bridges: the role of subfields in metaheuristics. SIGEVOlution 1(4), 9–15 (2006)

    Article  Google Scholar 

  57. Mertens, S.: A physicist’s approach to number partitioning. Theor. Comput. Sci. 265, 79–108 (2001)

    Article  Google Scholar 

  58. Merz, P., Freisleben, B.: Fitness landscape analysis and memetic algorithms for the quadratic assignment problem. IEEE Trans. Evol Comput 4, 337–352 (2000)

    Article  Google Scholar 

  59. Monasson, R.: Introduction to phase transitions in random optimization problems. Technical Report, Laboratoire de Physique Theorique de l’ENS, Paris (2007)

    Google Scholar 

  60. Neumann, F., Wegener, I.: Randomized local search, evolutionary algorithms, and the minimum spanning tree problem. Theor Comp. Sci. 378, 32–40 (2007).

    Article  Google Scholar 

  61. Neumann, F., Witt, C.: Runtime analysis of a simple ant colony optimization algorithm. Proceedings of ISAAC ’06, Springer LNCS, Berlin Heidelberg vol. 4288, pp. 618–627 (2006)

    Google Scholar 

  62. Neumann, F., Witt, C.: Ant colony optimization and the minimum spanning tree problem. Proceedings of LION ’08, Learning and Intelligent Optimization, Theoretical Computer Science 411, 2406–2413 (2010)

    Google Scholar 

  63. Norman, F.: Markov Processes and Learning Models. Academic Press, New York (1972)

    Google Scholar 

  64. Oliveto, P.S., He, J., Yao, X.: Time complexity of evolutionary algorithms for combinatorial optimization: a decade of results. Int. J. Automat. and Comput. 4, 281–293 (2007)

    Article  Google Scholar 

  65. Oliveto, P.S., He, J., Yao, X.: Evolutionary algorithms and the vertex cover problem. Proceedings of the Congress on Evolutionary Computation CEC ’07, Singapore pp. 1870–1877 (2007)

    Google Scholar 

  66. Purkayastha, P., Baras, J.S.: Convergence results for ant routing algorithms via stochastic approximation and optimization. Proceedings of 46th IEEE Conference on Decision and Control, pp. 340–345 (2007)

    Google Scholar 

  67. Reidys C.M., Stadler, P.F.: Combinatorial landscapes. SIAM Rev. 44, 3–54 (2002)

    Article  Google Scholar 

  68. Rudolph, G.: Convergence Analysis of canonical genetic algorithms. IEEE Trans. Neural. Netw. 5, 96–101 (1994)

    Article  Google Scholar 

  69. Sasaki, G.H., Hajek, B.: The time complexity of maximum matching by simulated annealing. J. ACM 35, 67–89 (1988)

    Article  Google Scholar 

  70. Scharnow, J., Tinnefeld, K., Wegener, I.: Fitness landscapes based on sorting and shortest path problems. Proceedings of 7th Conference on Parallel Problem Solving from Nature, 54–63 (2002)

    Google Scholar 

  71. Schiavinotto, T., Stûtzle, T.: A review of metrics on permutations for search landscape analysis. Comput. Oper. Res. 34, 3143–3153 (2007)

    Article  Google Scholar 

  72. Sebastiani, G., Torrisi, G.L.: An extended ant colony algorithm and its convergence analysis. Methodol. Comput. Appl. Probability 7, 249–263 (2005)

    Article  Google Scholar 

  73. Storch, T.: How randomized search heuristics find maximum cliques in planar graphs. Proceedings of 8th Annual Conference on Genetic and Evolutionary Computation, Seattle, Washington, USA pp. 567–574 (2006)

    Google Scholar 

  74. Stützle, T., Hoos, H.H.: MAX-MIN Ant System. Future Gen. Comput. Sys. 16, 889–914 (2000)

    Article  Google Scholar 

  75. Stützle, T., Dorigo, M.: A short convergence proof for a class of ACO algorithms. IEEE Trans. Evol. Comput. 6, 358–365 (2002)

    Article  Google Scholar 

  76. Sudholt, D., Witt, C.: Runtime analysis of binary PSO. Proceedings of 10th Annual Conf. on Genetic and Evolutionary Computation, New York, USA pp. 135–142 (2008)

    Google Scholar 

  77. Teytaud, O., Gelly, S.: General lower bounds for evolutionary algorithms. Proceedings of 9th Conference on Parallel Problem Solving from Nature, pp. 21–31 (2006)

    Google Scholar 

  78. Trelea, I.C.: The particle swarm optimization algorithm: convergence analysis and parameter selection. Inf. Process. Lett. 85, 317–325 (2003)

    Article  Google Scholar 

  79. Vaughan, D.E., Jacobson, S.H., Kaul, H.: Analyzing the performance of simultaneous generalized hill climbing algorithms. Comput. Optim. Appl. 37, 103–119 (2007)

    Article  Google Scholar 

  80. Wegener, I.: Simulated annealing beats metropolis in combinatorial optimization. Proceedings of ICALP ’05, Springer LNCS, Berlin Heidelberg vol. 3580, pp. 589–601 (2005)

    Google Scholar 

  81. Wegener, I., Witt, C.: On the analysis of a simple evolutionary algorithm on quadratic pseudo-boolean functions. J. Discrete Algorithms 3, 61–78 (2005)

    Article  Google Scholar 

  82. Whitley, D., Watson, J.P.: Complexity theory and the no free lunch theorem. In: Burke, E.K., Kendall, G. (eds.) Search Methodologies: Introductory Tutorials in Optimization and Decision Support Techniques, Kluwer, Boston, 317–399 (2005)

    Google Scholar 

  83. Witt, C.: Worst-case and average-case approximations by simple randomized search heuristics. Proceedings of 22nd Annual Symposium on Theoretical Aspects of Computer Science, Springer LNCS, Berlin Heidelberg vol. 3404, pp. 44–56 (2005)

    Google Scholar 

  84. Witt, C.: Runtime analysis of the \((\mu+1)\) EA on simple pseudo-bolan functions. Proceedings of 8th Annual Conference on Genetic and Evolutionary Computation, Seattle, Washington, pp. 651–658 (2006)

    Google Scholar 

  85. Wolpert, D.H., Macready, W.G.: No free lunch theorems for optimization. IEEE Trans. Evol. Comput. 1, 67–82 (1997).

    Article  Google Scholar 

  86. Yao, A.C.: Probabilistic computations: towards a unified measure of complexity. Proceedings of 17th IEEE Symposium on the Foundations of Computer Science, 222–227 (1977)

    Google Scholar 

  87. Yu, Y., Zhou, Z.-H.: A new approach to estimating the expected first hitting time of evolutionary algorithms. Proceedings of 21th National Conference on Artificial Intelligence, Boston, MA, 555–560 (2006)

    Google Scholar 

  88. Zhang, W.: Phase transitions and backbones of the asymmetric travelling salesman problem. J. Artif. Intell. Res. 21, 471–497 (2004)

    Google Scholar 

  89. Zlochin, M., Birattari, M., Meuleau, N., Dorigo, M.: Model-based search for combinatorial optimization: a critical survey. Ann. Oper. Res. 131, 373–379 (2004)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Vienna, Universitaetsstrasse 5/9, A-1010, Vienna, Austria

    Walter J. Gutjahr

Authors
  1. Walter J. Gutjahr
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Walter J. Gutjahr .

Editor information

Editors and Affiliations

  1. , Département de mathématiques et génie in, École Polytechnique Montréal, Montreal, H3C 3A7, Québec, Canada

    Michel Gendreau

  2. Dépt. Informatique et Recherche, Opérationnelle, Université de Montreal, Pavillon André-Aisenstadt, Montreal, H3C 3J7, Québec, Canada

    Jean-Yves Potvin

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer Science+Business Media, LLC

About this chapter

Cite this chapter

Gutjahr, W.J. (2010). Stochastic Search in Metaheuristics. In: Gendreau, M., Potvin, JY. (eds) Handbook of Metaheuristics. International Series in Operations Research & Management Science, vol 146. Springer, Boston, MA. https://doi.org/10.1007/978-1-4419-1665-5_19

Download citation

Publish with us

Policies and ethics

Search

Navigation