Skip to main content
SpringerLink
Log in

Extremal Optimization: An Evolutionary Local-Search Algorithm

  • Chapter
Computational Modeling and Problem Solving in the Networked World

Part of the book series: Operations Research/Computer Science Interfaces Series ((ORCS,volume 21))

Abstract

A recently introduced general-purpose heuristic for finding high-quality solutions for many hard optimization problems is reviewed. The method is inspired by recent progress in understanding far-from-equilibrium phenomena in terms ofself-organized criticality, a concept introduced to describe emergent complexity in physical systems. This method, calledextremal optimization, successively replaces the value of extremely undesirable variables in a sub-optimal solution with new, random ones. Large, avalanche-like fluctuations in the cost function self-organize from this dynamics, effectively scaling barriers to explore local optima in distant neighborhoods of the configuration space while eliminating the need to tune parameters. Drawing upon models used to simulate the dynamics of granular media, evolution, or geology, extremal optimization complements approximation methods inspired by equilibrium statistical physics, such assimulated annealing. It may be but one example of applying new insights intonon-equilibrium phenomenasystematically to hard optimization problems. This method is widely applicable and so far has proved competitive with — and even superior to — more elaborate general-purpose heuristics on testbeds of constrained optimization problems with up to 105variables, such as bipartitioning, coloring, and satisfiability. Analysis of a suitable model predicts the only free parameter of the method in accordance with all experimental results.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. E. H. L. Aarts and P. J. M. van Laarhoven,Statistical Cooling: A general Approach to Combinatorial Optimization Problems, Philips J. Res.40, 193–226 (1985).

    MathSciNet  Google Scholar 

  2. E. H. L. Aarts and P. J. M. van Laarhoven,Simulated Annealing: Theory and Applications(Reidel, Dordrecht, 1987).

    MATH  Google Scholar 

  3. Local Search in Combinatorial Optimization, Eds. E. H. L. Aarts and J. K. Lenstra (Wiley, New York, 1997).

    MATH  Google Scholar 

  4. [4] SeeFrontiers in problem solving: Phase transitions and complexity, eds. T. Hogg, B. A. Huberman, and C. Williams, special issue of Artificial Intelligence81:1–2 (1996).

    Google Scholar 

  5. G. Ausiello et al.,Complexity and Approximation(Springer, Berlin, 1999).

    Book  MATH  Google Scholar 

  6. P. Bak and K. Sneppen,Punctuated Equilibrium and Criticality in a simple Model of Evolution, Phys. Rev. Lett.71, 4083–4086 (1993).

    Article  Google Scholar 

  7. P. Bak, C. Tang, and K. Wiesenfeld,Self-Organized Criticality, Phys. Rev. Lett.59, 381–384 (1987).

    Article  MathSciNet  Google Scholar 

  8. S. Boettcher,Extremal Optimization and Graph Partitioning at the Percolation Threshold, J. Math. Phys. A: Math. Gen.32, 5201–5211 (1999).

    Article  MathSciNet  MATH  Google Scholar 

  9. S. Boettcher,Extremal Optimization: Heuristics via Co-Evolutionary Avalanches, Computing in Science and Engineering2:6, 75 (2000).

    Article  Google Scholar 

  10. [10] S. Boettcher and M. Frank,Analysis of Extremal Optimization in Designed Search Spaces, Honors Thesis, Dept. of Physics, Emory University, (in preparation).

    Google Scholar 

  11. S. Boettcher and M. Grigni,Jamming model for the extremal optimization heuristic, J. Phys. A: Math. Gen.351109–1123 (2002).

    Article  MathSciNet  MATH  Google Scholar 

  12. [12] S. Boettcher, M. Grigni, G. Istrate, and A. G. Percus,Phase Transitions and Algorithmic Complexity in 3-Coloring, (in preparation).

    Google Scholar 

  13. S. Boettcher and A. G. Percus,Extremal Optimization: Methods derived from Co-Evolution,inGECCO-99: Proceedings of the Genetic and Evolutionary Computation Conference(Morgan Kaufmann, San Francisco, 1999), 825–832.

    Google Scholar 

  14. S. Boettcher and A. G. Percus,Nature’s Way of Optimizing, Artificial Intelligence119, 275–286 (2000).

    MATH  Google Scholar 

  15. S. Boettcher and A. G. Percus,Optimization with Extremal Dynamics, Phys. Rev. Lett.86, 5211–5214(2001).

    Article  Google Scholar 

  16. S. Boettcher and A. G. Percus,Extremal Optimization for Graph Partitioning, Phys. Rev. E64, 026114(2001).

    Article  Google Scholar 

  17. A. B. Bortz, M. H. Kalos, and J. L. Lebowitz, J. Comp. Phys.17, 10–18 (1975).

    Article  Google Scholar 

  18. P. Cheeseman, B. Kanefsky, and W. M. Taylor,Where the really hard Problems are, in Proc. of IJCAI-91, eds. J. Mylopoulos and R. Rediter (Morgan Kaufmann, San Mateo, CA, 1991), pp. 331–337.

    Google Scholar 

  19. H. Cohn and M. Fielding,Simulated Annealing: Searching for an optimal Temperature Schedule, SIAM J. Optim.9, 779–802 (1999).

    Article  MathSciNet  MATH  Google Scholar 

  20. S. A. Cook,The Complexity of Theorem-Proving Procedures, in: Proc. 3rd Annual ACM Symp. on Theory of Computing, 151–158 (Assoc, for Computing Machinery, New York, 1971).

    Chapter  Google Scholar 

  21. J. Culberson and I. P. Gent,Frozen Development in Graph Coloring, J. Theor. Comp. Sci.265, 227–264(2001).

    Article  MathSciNet  MATH  Google Scholar 

  22. J. Dall,Searching Complex State Spaces with Extremal Optimization and other Stochastics Techniques, Master Thesis, Fysisk Institut, Syddansk Universitet Odense, 2000 (in danish).

    Google Scholar 

  23. J. Dall and P. Sibani,Faster Monte Carlo Simulations at Low Temperatures: The Waiting Time Method, Computer Physics Communication141, 260–267 (2001).

    Article  MATH  Google Scholar 

  24. [24] W. Feller,An Introduction to Probability Theory and Its Applications, Vol. 1 (Wiley, New York, 1950).

    Google Scholar 

  25. M. Fielding,Simulated Annealing with an optimal fixed Temperature, SIAM J. Optim.11, 289–307 (2000).

    Article  MathSciNet  MATH  Google Scholar 

  26. M. R. Garey and D. S. Johnson,Computers and Intractability, A Guide to the Theory of NP-Completeness(W. H. Freeman, New York, 1979).

    MATH  Google Scholar 

  27. S. Geman and D. Geman,Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images, in Proc. 6th IEEE Pattern Analysis and Machine Intelligence, 721–741 (1984).

    Google Scholar 

  28. F. Glover,Future Paths for Integer Programming and Links to Artificial Intelligence, Computers & Ops. Res.5, 533–549 (1986).

    Article  Google Scholar 

  29. D. E. Goldberg,Genetic Algorithms in Search, Optimization, and Machine Learning, (Addison-Wesley, Reading, 1989).

    MATH  Google Scholar 

  30. S. J. Gould and N. Eldridge,Punctuated Equilibria: The Tempo and Mode of Evolution Reconsidered, Paleobiology3, 115–151 (1977).

    Google Scholar 

  31. J. W. Greene and K. J. Supowit,Simulated Annealing without rejecting moves, IEEE Trans. on Computer-Aided DesignCAD-5, 221–228 (1986).

    Article  Google Scholar 

  32. A. K. Hartmann,Evidence for existence of many pure ground states in 3d±J Spin Glasses, Europhys. Lett.40, 429 (1997).

    Article  Google Scholar 

  33. B. A. Hendrickson and R. Leland,A multilevel algorithm for partitioning graphs, in: Proceedings of Supercomputing’95, San Diego, CA (1995).

    Google Scholar 

  34. J. Houdayer and O. C. Martin,Renormalization for discrete optimization, Phys. Rev. Lett.83, 1030–1033 (1999).

    Article  Google Scholar 

  35. [ J. Holland,Adaptation in Natural and Artificial Systems(University of Michigan Press, Ann Arbor, 1975).

    Google Scholar 

  36. B. A. Huberman and T. Hogg,Phase transitions in artificial intelligence systems, Artificial Intelligence33, 155–171 (1987).

    Article  Google Scholar 

  37. M. Jerrum and A. Sinclair,The Markov chain Monte Carlo method: an approach to approximate counting and integration, in Approximation Algorithms for NP-hard Problems, ed. Dorit Hochbaum (PWS Publishers, 1996).

    Google Scholar 

  38. D. S. Johnson, C. R. Aragon, L. A. McGeoch, and C. Schevon,Optimization by Simulated Annealing — an Experimental Evaluation. I. Graph Partitioning, Operations Research37, 865–892 (1989).

    Article  MATH  Google Scholar 

  39. [39]7th DIMACS Implementation Challenge on Semidefinite and related Optimization Problems, eds. D. S. Johnson, G. Pataki, and F. Alizadeh (to appear, see http://dimacs.rutgers.edu/Challenges/Seventh/).

    Google Scholar 

  40. [40] M. Jünger and F. Liers (Cologne University), private communication.

    Google Scholar 

  41. [41] G. Karypis and V. Kumar,METIS, a Software Package for Partitioning Unstructured Graphs, see http://www-users.cs.umn.edu/~karypis/metis/main.shtml, (METIS is copyrighted by the Regents of the University of Minnisota).

    Google Scholar 

  42. S. Kirkpatrick, C. D. Gelatt, andM. P. Vecchi,Optimization by simulated annealing, Science220,671–680(1983).

    Article  MathSciNet  MATH  Google Scholar 

  43. S. Kirkpatrick and B. Selman,Critical Behavior in the Satisfiability of Random Boolean Expressions, Science264, 1297–1301 (1994).

    Article  MathSciNet  MATH  Google Scholar 

  44. M. Lundy and A. Mees,Convergence of an Annealing Algorithm, Math. Programming34, 111–124(1996).

    MathSciNet  Google Scholar 

  45. P. Merz and B. Freisleben,Memetic algorithms and the fitness landscape of the graph bi-partitioning problem, Lect. Notes Comput. Sc.1498, 165–11A (1998).

    Google Scholar 

  46. N. Metropolis, A.W. Rosenbluth, M.N. Rosenbluth, A.H. Teller and E. Teller,Equation of state calculations by fast computing machines, J. Chem. Phys.21(1953) 1087–1092.

    Article  Google Scholar 

  47. M. Mezard, G. Parisi, and M. A. Virasoro,Spin Glass Theory and Beyond(World Scientific, Singapore, 1987).

    MATH  Google Scholar 

  48. R. Monasson, R. Zecchina, S. Kirkpatrick, B. Selman, and L. Troyansky,Determining computational complexity from characteristic ‘phase transitions,’Nature 400, 133–137 (1999), and Random Struct. Alg15, 414–435 (1999).

    Article  MathSciNet  Google Scholar 

  49. Meta-Heuristics: Theory and Application, Eds. I. H. Osman and J. P. Kelly (Kluwer, Boston, 1996).

    Google Scholar 

  50. K. F. Pal,The ground state energy of the Edwards-Anders on Ising spin glass with a hybrid genetic algorithm, Physica A 223, 283–292 (1996).

    Google Scholar 

  51. R. G. Palmer, D. L. Stein, E. Abrahamvs, and P. W. Anderson,Models of Hierarchically Constrained Dynamics for Glassy Relaxation, Phys. Rev. Lett.53, 958–961 (1984).

    Article  Google Scholar 

  52. Modern Heuristic Search Methods, Eds. V. J. Rayward-Smith, I. H. Osman, and C. R. Reeves (Wiley, New York, 1996).

    MATH  Google Scholar 

  53. Modern Heuristic Techniques for Combinatorial Problems, Ed. C. R. Reeves (Wiley, New York, 1993).

    MATH  Google Scholar 

  54. G. B. Sorkin,Efficient Simulated Annealing on Fractal Energy Landscapes, Algorithmica6, 367–418(1991).

    Article  MathSciNet  MATH  Google Scholar 

  55. [55] D. E. Vaughan and S. H. Jacobson,Nonstationary Markov Chain Analysis of Simultaneous Generalized Hill Climbing Algorithms, (submitted), available at http://filebox.vt.edu/users/dvaughn/pdf_powerpoint/sghc2.pdf.

    Google Scholar 

  56. Meta-Heuristics: Advances and Trends in Local Search Paradigms for Optimization, Ed. S. Voss (Kluwer, Dordrecht, 1998).

    Google Scholar 

  57. I. Wegener,Theoretical aspects of evolutionary algorithms, Lecture Notes in Computer Science2076, 64–78(2001).

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Physics, Emory University, 30322, Atlanta, GA, USA

    Stefan Boettcher

  2. Computer and Computational Sciences Division, MS-B265, Los Alamos National Laboratory, Los Alamos, 87545, NM, USA

    Allon G. Percus

Authors
  1. Stefan Boettcher
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Allon G. Percus
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. The Pennsylvania State University, USA

    Hemant K. Bhargava

  2. Arizona State University, USA

    Nong Ye

Rights and permissions

Reprints and permissions

Copyright information

© 2003 Springer Science+Business Media New York

About this chapter

Cite this chapter

Boettcher, S., Percus, A.G. (2003). Extremal Optimization: An Evolutionary Local-Search Algorithm. In: Bhargava, H.K., Ye, N. (eds) Computational Modeling and Problem Solving in the Networked World. Operations Research/Computer Science Interfaces Series, vol 21. Springer, Boston, MA. https://doi.org/10.1007/978-1-4615-1043-7_3

Download citation

  • DOI: https://doi.org/10.1007/978-1-4615-1043-7_3

  • Publisher Name: Springer, Boston, MA

  • Print ISBN: 978-1-4613-5366-9

  • Online ISBN: 978-1-4615-1043-7

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation