An Introduction to Fitness Landscape Analysis and Cost Models for Local Search

  • Jean-Paul Watson
Part of the International Series in Operations Research & Management Science book series (ISOR, volume 146)


Despite their empirical effectiveness, our theoretical understanding of metaheuristic algorithms based on local search (and all other paradigms) is very limited, leading to significant problems for both researchers and practitioners. Specifically, the lack of a theory of local search impedes the development of more effective metaheuristic algorithms, prevents practitioners from identifying the metaheuristic most appropriate for a given problem, and permits widespread conjecture and misinformation regarding the benefits and/or drawbacks of particular metaheuristics. Local search metaheuristic performance is closely linked to the structure of the fitness landscape, i.e., the nature of the underlying search space. Consequently, understanding such structure is a first step toward understanding local search behavior, which can ultimately lead to a more general theory of local search. In this chapter, we introduce and survey the literature on fitness landscape analysis for local search, placing the research in the context of a broader, critical classification scheme delineating methodologies by their potential to account for local search metaheuristic performance.


Local Search Tabu Search Problem Instance Cost Model Travel Salesman Problem 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



Sandia is a multipurpose laboratory operated by Sandia Corporation, a Lockheed-Martin Company, for the United States Department of Energy under contract DE-AC04-94AL85000.


  1. 1.
    Achlioptas, D., Gomes, C., Kautz, H., Selman, B.: Generating satisfiable problem instances. In: Ford, K. (ed.) Proceedings of the 17th National Conference on Artificial Intelligence (AAAI-00), pp. 256–261. AAAI/MIT Press (2000)Google Scholar
  2. 2.
    Beveridge, J., Graves, C., Steinborn, J.: Comparing random starts local search with key feature matching. In: Proceedings of the 15th International Joint Conference on Artificial Intelligence (IJCAI-97) (1997)Google Scholar
  3. 3.
    Cheeseman, P., Kanefsky, B., Taylor, W.: Where the Really hard problems are. In: Proceedings of the 12th International Joint Conference on Artificial Intelligence (IJCAI-91), pp. 331–337 (1991)Google Scholar
  4. 4.
    Clark, D., Frank, J., Gent, I., MacIntyre, E., Tomov, N., Walsh, T.: Local search and the number of solutions. In: Freuder, E.C. (ed.) Proceedings of the 2nd International Conference on Principles and Practices of Constraint Programming (CP-96), pp. 119–133. Springer (1996)Google Scholar
  5. 5.
    Cohen, P.: Empirical Methods for Artificial Intelligence. MIT Press (1995)Google Scholar
  6. 6.
    Fournier, N.G.: Modelling the dynamics of stochastic local search on k-SAT. J. Heuristics 13, 587–639 (2007)CrossRefGoogle Scholar
  7. 7.
    Frank, J., Cheeseman, P., Stutz, J.: When gravity fails: local search topology. J. Artif. Intell. Res. 7, 249–281 (1997)Google Scholar
  8. 8.
    Glover, F., Laguna, M.: Tabu Search. Kluwer, Boston, MA (1997)CrossRefGoogle Scholar
  9. 9.
    Hogg, T., Huberman, B., Williams, C.: Special issue on frontiers in problem solving: phase transitions and complexity. Artif. Intell. 81(1–2) (1996)Google Scholar
  10. 10.
    Hooker, J.: Testing heuristics: we have it all wrong. J. Heuristics 1, 33–42 (1995)CrossRefGoogle Scholar
  11. 11.
    Hoos, H.: A mixture-model for the behaviour of SLS algorithms for SAT. In: Proceedings of the 18th National Conference on Artificial Intelligence (AAAI-02), pp. 661–667. AAAI Press/MIT Press (2002)Google Scholar
  12. 12.
    Hoos, H., Stüzle, T.: Stochastic Local Search: Foundations and Applications. Morgan Kaufmann (2005)Google Scholar
  13. 13.
    Johnson, D., McGeoch, L.A.: The traveling salesman problem: a case study in local optimization. In: Local Search in Optimization, pp. 215–310. John Wiley (1997)Google Scholar
  14. 14.
    Jones, T.: Evolutionary algorithms, fitness landscapes, and search. Ph.D. thesis, Department of Computer Science, University of New Mexico (1995)Google Scholar
  15. 15.
    Jones, T., Forrest, S.: Fitness distance correlation as a measure of problem difficulty. In: Eschelman L. (ed.) Proceedings of the 6th International Conference on Genetic Algorithms, pp. 184–192. Morgan Kaufmann (1995)Google Scholar
  16. 16.
    Kauffman, S.: The Origins of Order. Oxford University Press (1993)Google Scholar
  17. 17.
    Kirkpatrick, S., Selman, B.: Critical behavior in the satisfiability of random boolean expressions. Science 264, 1297–1301 (1994)CrossRefGoogle Scholar
  18. 18.
    Kirkpatrick, S., Toulouse, G.: Configuration space analysis of traveling salesman problems. J. Phys. 46, 1277–1292 (1985)CrossRefGoogle Scholar
  19. 19.
    Lin, S., Kernighan, B.: An effective heuristic algorithm for the traveling salesman problem. Oper. Res. 21, 498–516 (1973)CrossRefGoogle Scholar
  20. 20.
    Lourenço, H., Martin, O., Stuützle, T.: Iterated local search. In: Glover, F., Kochenberger, G. (eds.) Handbook of Metaheuristics. Kluwer (2003)Google Scholar
  21. 21.
    Mattfeld, D., Bierwirth, C., Kopfer, H.: A search space analysis of the job shop scheduling problem. Ann. Oper. Res. 86, 441–453 (199)Google Scholar
  22. 22.
    Mezard, M., Parisi, G.: A replica analysis of the traveling salesman problem. J. Phys. 47, 1285–1296 (1986)CrossRefGoogle Scholar
  23. 23.
    Monasson, R., Zecchina, R., Kirkpatrick, S., Selman, B., Troyansky, L.: Determining computational complexity for characteristic ‘phase transitions’. Nature 400, 133–137 (1998)Google Scholar
  24. 24.
    Mühlenbein, H., Georges-Schleuter, M., Krämer, O.: Evolution algorithms in combinatorial optimization. Parallel Comput. 7, 65–85 (1988)CrossRefGoogle Scholar
  25. 25.
    Nowicki, E., Smutnicki, C.: A fast taboo search algorithm for the job shop problem. Manage. Sci. 42(6), 797–813 (1996)CrossRefGoogle Scholar
  26. 26.
    Papadimitriou, C.: On selecting a satisfying truth assignment. In: Proceedings of the 33rd Annual IEEE Symposium on Foundations of Computer Science (FOCS 91) (1991)Google Scholar
  27. 27.
    Parkes, A.: Clustering at the phase transition. In: Proceedings of the 14th National Conference on Artificial Intelligence (AAAI-97), pp. 340–345. AAAI/MIT Press (1997)Google Scholar
  28. 28.
    Prosser, P.: Binary constraint satisfaction problems: Some are harder than others. In: Proceedings of the 11th European Conference on Artificial Intelligence (ECAI-94), pp. 95–99 (1994)Google Scholar
  29. 29.
    Rana, S.: Local optima and genetic algorithms. Ph.D. thesis, Department of Computer Science, Colorado State University (1999)Google Scholar
  30. 30.
    Rana, S., Whitley, L.: Representation, search, and genetic algorithms. In: Proceedings of the 14th National Conference on Artificial Intelligence (AAAI-97). AAAI Press/MIT Press (1997)Google Scholar
  31. 31.
    Reeves, C.: Landscapes, operators and heuristic search. Ann. Oper. Res. 86, 473–490 (1998)CrossRefGoogle Scholar
  32. 32.
    Riedys, C., Stadler, P.: Combinatorial landscapes. Technical Report 01-03-014, Santa Fe Institute (2001)Google Scholar
  33. 33.
    Schneider, J., Froschhammer, C., Morgernstern, I., Husslein, T., Singer, J.: Searching for backbones–-an efficient parallel algorithms for the traveling salesman problem. Comput. Phys. Comm. 96, 173–188 (1996)CrossRefGoogle Scholar
  34. 34.
    Schrag, R., Crawford, J.M.: Implicates and prime implications in random 3SAT. Artif. Intell. 88, 199–222 (1996)CrossRefGoogle Scholar
  35. 35.
    Singer, J., Gent, I., Smaill, A.: Backbone fragility and the local search cost peak. J. Artif. Intell. Res. 12, 235–270 (2000)Google Scholar
  36. 36.
    Slaney, J., Walsh, T.: Backbones in optimization and approximation. In: Nebel, B. (ed.) Proceedings of the 17th International Joint Conference on Artificial Intelligence (IJCAI-01), pp. 254–259. Morgan Kaufmann (2001)Google Scholar
  37. 37.
    Sourlas, N.: Statistical mechanics and the traveling salesman problem. Europhys. Lett. 2, 919–923 (1986)CrossRefGoogle Scholar
  38. 38.
    Stadler, P.: Landscapes and their correlation functions. J. Math. Chem. 20, 1–45 (1996)CrossRefGoogle Scholar
  39. 39.
    Stuützle, T.: Personal communication (2001)Google Scholar
  40. 40.
    Watson, J.P.: Problem difficulty for local search in job-shop scheduling. Ph.D. thesis, Department of Computer Science, Colorado State University (2003)Google Scholar
  41. 41.
    Watson, J.P. Beck, J.C., Howe, A., Whitley, L.: Problem difficulty for tabu search in job-shop scheduling. Artif. Intell. 143(2), 189–217 (2003)CrossRefGoogle Scholar
  42. 42.
    Watson, J.P., Whitley, L.D., Howe, A.E.: Linking search space structure, run-time dynamics, and problem difficulty: a step toward demystifying tabu search. J. Artif. Intell. Res. 24, 221–261 (2005)CrossRefGoogle Scholar
  43. 43.
    Weinberger, E.D.: Correlated and uncorrelated fitness landscapes and how to tell the differences. Biol. Cybern. 63, 325–336 (1989)CrossRefGoogle Scholar
  44. 44.
    Wright, S.: The roles of mutation, inbreeding, crossbreeding and selection in evolution. In: Jones, D. (ed.) International Proceedings of the 6th International Congress on Genetics, vol. 1, pp. 356–366 (1932)Google Scholar
  45. 45.
    Yokoo, M.: Why adding more constraints makes a problem easier for hill-climbing algorithms: Analyzing landscapes of CSPs. In: Proceedings of the 3rd International Conference on the Principles and Practice of Constraint Programming (CP-97), pp. 356–370. Springer (1997)Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  1. 1.Discrete Math and Complex Systems DepartmentSandia National LaboratoriesAlbuquerqueUSA

Personalised recommendations