Soft Computing

, Volume 21, Issue 7, pp 1735–1752 | Cite as

Analysing and characterising optimization problems using length scale

  • Rachael MorganEmail author
  • Marcus Gallagher
Methodologies and Application


Analysis of optimization problem landscapes is fundamental in the understanding and characterisation of problems and the subsequent practical performance of algorithms. In this paper, a general framework is developed for characterising black-box optimization problems based on length scale, which measures the change in objective function with respect to the distance between candidate solution pairs. Both discrete and continuous problems can be analysed using the framework, however, in this paper, we focus on continuous optimization. Length scale analysis aims to efficiently and effectively utilise the information available in black-box optimization. Analytical properties regarding length scale are discussed and illustrated using simple example problems. A rigorous sampling methodology is developed and demonstrated to improve upon current practice. The framework is applied to the black-box optimization benchmarking problem set, and shows greater ability to discriminate between the problems in comparison to seven well-known landscape analysis techniques. Dimensionality reduction and clustering techniques are applied comparatively to an ensemble of the seven techniques and the length scale information to gain insight into the relationships between problems. A fundamental summary of length scale information is an estimate of its probability density function, which is shown to capture salient structural characteristics of the benchmark problems.


Length scale Fitness landscape analysis Black-box optimization Search space diagnostics 


Compliance with ethical standards

Conflicts of interest



  1. Aggarwal CC, Hinneburg A, Keim DA (2001) On the surprising behavior of distance metrics in high dimensional spaces. In: Proceedings of the 8th international conference on database theory. Springer, London, pp 420–434Google Scholar
  2. Bartz-Beielstein T, Chiarandini M, Paquete L, Preuss M (2010) Experimental methods for the analysis of optimization algorithms. Springer, New YorkCrossRefzbMATHGoogle Scholar
  3. Beliakov G (2006) Interpolation of lipschitz functions. J Comput Appl Math 196(1):20–44MathSciNetCrossRefzbMATHGoogle Scholar
  4. Beyer KS, Goldstein J, Ramakrishnan R, Shaft U (1999) When is ”nearest neighbor” meaningful? In: Proceedings of the 7th international conference on database theory. Springer, London, pp 217–235Google Scholar
  5. Borenstein Y, Poli R (2006) Kolmogorov complexity, optimization and hardness. In: IEEE congress on evolutionary computation (CEC 2006), pp 112–119Google Scholar
  6. Cheeseman P, Kanefsky B, Taylor W (1991) Where the really hard problems are. In: Proceedings of 12th international joint conference on AI, Morgan Kauffman, pp 331–337Google Scholar
  7. Collard P, Vérel S, Clergue M (2004) Local search heuristics: fitness cloud versus fitness landscape. In: The 2004 European conference on artificial intelligence, IOS Press, pp 973–974Google Scholar
  8. Cover TM, Thomas JA (1991) Elements of information theory. Wiley, New YorkCrossRefzbMATHGoogle Scholar
  9. Fletcher R (1987) Practical methods of optimization, 2nd edn. Wiley, HobokenzbMATHGoogle Scholar
  10. Forrester A, Sóbester A, Keane A (2008) Engineering design via surrogate modelling: a practical guide. Wiley, HobokenCrossRefGoogle Scholar
  11. Gallagher M (2000) Multi-layer perceptron error surfaces: visualization, structure and modelling. PhD thesis, Department of Computer Science and Electrical Engineering, University of QueenslandGoogle Scholar
  12. Gallagher M (2001) Fitness distance correlation of neural network error surfaces: a scalable, continuous optimization problem. In: Raedt LD, Flach P (eds) European conference on machine learning, Singapore, Lecture notes in artificial intelligence, vol 2167, pp 157–166Google Scholar
  13. Gallagher M, Downs T, Wood I (2002) Empirical evidence for ultrametric structure in multi-layer perceptron error surfaces. Neural Process Lett 16(2):177–186CrossRefzbMATHGoogle Scholar
  14. Gent I, Walsh T (1996) The TSP phase transition. Artif Intell 88(1–2):349–358MathSciNetCrossRefzbMATHGoogle Scholar
  15. Grinstead CM, Snell JL (2012) Introduction to probability. American Mathematical Society, ProvidencezbMATHGoogle Scholar
  16. Guyon I, Elisseeff A (2003) An introduction to variable and feature selection. J Mach Learn Res 3:1157–1182zbMATHGoogle Scholar
  17. Hansen N (2000) Invariance, self-adaptation and correlated mutations in evolution strategies. In: Schoenauer et al M (ed) Parallel problem solving from nature—PPSN VI. Lecture notes in computer science, vol 1917, Springer, pp 355–364Google Scholar
  18. Hansen N, Finck S, Ros R, Auger A (2010) Real-parameter black-box optimization benchmarking 2010: noiseless functions definitions. Technical Report, RR-6829, INRIAGoogle Scholar
  19. Horst R, Tuy H (1996) Global optimization: deterministic approaches. Springer, New YotkCrossRefzbMATHGoogle Scholar
  20. Hutter F, Hamadi Y, Hoos H, Leyton-Brown K (2006) Performance prediction and automated tuning of randomized and parametric algorithms. In: Benhamou F (ed) Principles and practice of constraint programming. Lecture notes in computer science, vol 4204, Springer, pp 213–228Google Scholar
  21. Jin Y (2005) A comprehensive survey of fitness approximation in evolutionary computation. Soft Comput 9(1):3–12CrossRefGoogle Scholar
  22. Jones T, Forrest S (1995) Fitness distance correlation as a measure of problem difficulty for genetic algorithms. In: Proceedings of the 6th international conference on genetic algorithms. Morgan Kaufmann, San Francisco, CA, pp 184–192Google Scholar
  23. Kullback S, Leibler RA (1951) On information and sufficiency. Ann Math Stat 22(1):79–86MathSciNetCrossRefzbMATHGoogle Scholar
  24. Lee JA, Verleysen M (2007) Nonlinear dimensionality reduction. Springer, New YorkCrossRefzbMATHGoogle Scholar
  25. Lunacek M, Whitley D (2006) The dispersion metric and the CMA evolution strategy. In: Proceedings of the 8th annual conference on genetic and evolutionary computation. ACM, New York, pp 477–484Google Scholar
  26. Macready W, Wolpert D (1996) What makes an optimization problem hard. Complexity 5:40–46MathSciNetCrossRefGoogle Scholar
  27. Malan K, Engelbrecht A (2009) Quantifying ruggedness of continuous landscapes using entropy. In: IEEE congress on evolutionary computation, pp 1440–1447Google Scholar
  28. Mersmann O, Bischl B, Trautmann H, Preuss M, Weihs C, Rudolph G (2011) Exploratory landscape analysis. In: Proceedings of the 13th annual conference on genetic and evolutionary computation. ACM, New York, pp 829–836Google Scholar
  29. Morgan R, Gallagher M (2012) Length scale for characterising continuous optimization problems. In: Coello et al CAC (ed) Parallel problem solving from nature—PPSN XII. Lecture notes in computer science, vol 7491, Springer, pp 407–416Google Scholar
  30. Morgan R, Gallagher M (2014) Sampling techniques and distance metrics in high dimensional continuous landscape analysis: limitations and improvements. IEEE Trans Evol Comput 18(3):456–461CrossRefGoogle Scholar
  31. Muñoz MA, Kirley M, Halgamuge S (2012a) A meta-learning prediction model of algorithm performance for continuous optimization problems. In: Coello et al CAC (ed) Parallel problem solving from nature—PPSN XII. Lecture notes in computer science, vol 7491, Springer, pp 226–235Google Scholar
  32. Muñoz MA, Kirley M, Halgamuge SK (2012b) Landscape characterization of numerical optimization problems using biased scattered data. In: IEEE congress on evolutionary computation, pp 1180–1187Google Scholar
  33. Müller C, Baumgartner B, Sbalzarini I (2009) Particle swarm CMA evolution strategy for the optimization of multi-funnel landscapes. In: IEEE congress on evolutionary computation, pp 2685–2692Google Scholar
  34. Müller CL, Sbalzarini IF (2011) Global characterization of the CEC 2005 fitness landscapes using fitness-distance analysis. In: Proceedings of the 2011 international conference on applications of evolutionary computation. vol Part I. Springer, Berlin, Heidelberg, pp 294–303Google Scholar
  35. Overton M (2001) Numerical computing with IEEE floating point arithmetic. Cambridge University Press, CambridgeCrossRefzbMATHGoogle Scholar
  36. Pitzer E, Affenzeller M (2012) A comprehensive survey on fitness landscape analysis. In: Fodor J, Klempous R, Suárez Araujo C (eds) Recent advances in intelligent engineering systems, studies in computational intelligence. Springer, New York, pp 161–191Google Scholar
  37. Pitzer E, Affenzeller M, Beham A, Wagner S (2012) Comprehensive and automatic fitness landscape analysis using heuristiclab. In: Moreno-Díaz R, Pichler F, Quesada-Arencibia A (eds) Computer aided systems theory–EUROCAST 2011. Lecture notes in computer science, vol 6927, Springer, pp 424–431Google Scholar
  38. Reidys C, Stadler P (2002) Combinatorial landscapes. SIAM Rev 44(1):3–54MathSciNetCrossRefzbMATHGoogle Scholar
  39. Rice JR (1976) The algorithm selection problem. Adv Comput 15:65–118CrossRefGoogle Scholar
  40. Ridge E, Kudenko D (2007) An analysis of problem difficulty for a class of optimisation heuristics. In: Proceedings of the 7th european conference on evolutionary computation in combinatorial optimization, Springer, pp 198–209Google Scholar
  41. Rosé H, Ebeling W, Asselmeyer T (1996) The density of states—a measure of the difficulty of optimisation problems. In: Voigt et al HM (ed) Parallel problem solving from nature PPSN IV. Lecture notes in computer science, vol 1141, Springer, pp 208–217Google Scholar
  42. Rosen K (1999) Handbook of discrete and combinatorial mathematics, 2nd edn., Discrete mathematics and its applicationsTaylor & Francis, RoutledgeCrossRefGoogle Scholar
  43. Sergeyev YD, Kvasov DE (2010) Lipschitz global optimization. Wiley Encycl Oper Res Manag Sci 4:2812–2828Google Scholar
  44. Sheather S, Jones M (1991) A reliable data-based bandwidth selection method for kernel density estimation. J R Stat Soc Ser B (Methodological) 53:683–690MathSciNetzbMATHGoogle Scholar
  45. Shlesinger MF, West BJ, Klafter J (1987) Lévy dynamics of enhanced diffusion: application to turbulence. Phys Rev Lett 58:1100–1103MathSciNetCrossRefGoogle Scholar
  46. Smith T, Husbands P, O’Shea M (2002) Fitness landscapes and evolvability. Evol Comput 10(1):1–34CrossRefGoogle Scholar
  47. Smith-Miles K (2008) Cross-disciplinary perspectives on meta-learning for algorithm selection. ACM Comput Surv 41(1):1–25CrossRefGoogle Scholar
  48. Smith-Miles K, Lopes L (2011) Measuring instance difficulty for combinatorial optimization problems. Comput Oper Res 39(5):875–889MathSciNetCrossRefzbMATHGoogle Scholar
  49. Smith-Miles K, Tan TT (2012) Measuring algorithm footprints in instance space. In: IEEE congress on evolutionary computation (CEC), pp 1–8Google Scholar
  50. Solla SA, Sorkin GB, White SR (1986) Configuration space analysis for optimization problems. In: et al EB (ed) Disordered systems and biological organization, NATO ASI Series, vol F20, Springer, Berlin, New York, pp 283–293Google Scholar
  51. Stadler PF (1996) Landscapes and their correlation functions. J Math Chem 20:1–45MathSciNetCrossRefzbMATHGoogle Scholar
  52. Stadler PF (2002) Fitness landscapes. In: Lässig M, Valleriani A (eds) Biological evolution and statistical physics. Lecture notes in physics, vol 585, Springer, pp 183–204Google Scholar
  53. Stadler PF, Schnabl W (1992) The landscape of the traveling salesman problem. Phys Lett A 161(4):337–344MathSciNetCrossRefzbMATHGoogle Scholar
  54. Steer K, Wirth A, Halgamuge S (2008) Information theoretic classification of problems for metaheuristics. In: Li et al X (ed) Simulated evolution and learning, Lecture notes in computer science, vol 5361, Springer, pp 319–328Google Scholar
  55. Strongin R (1973) On the convergence of an algorithm for finding a global extremum. Eng Cybern 11:549–555MathSciNetGoogle Scholar
  56. Talbi E (2009) Metaheuristics: from design to implementation., Wiley series on parallel and distributed computingWiley, HobokenCrossRefzbMATHGoogle Scholar
  57. Van der Maaten L, Hinton G (2008) Visualizing data using t-SNE. J Mach Learn Res 9(11):2579–2605zbMATHGoogle Scholar
  58. van Hemert J (2005) Property analysis of symmetric travelling salesman problem instances acquired through evolution. Evol Comput Comb Optim 3448:122–131zbMATHGoogle Scholar
  59. Vassilev VK, Fogarty TC, Miller JF (2000) Information characteristics and the structure of landscapes. Evol Comput 8:31–60CrossRefGoogle Scholar
  60. Wang Y, Li B (2008) Understand behavior and performance of real coded optimization algorithms via nk-linkage model. In: IEEE world congress on computational intelligence, pp 801–808Google Scholar
  61. Weinberger E (1990) Correlated and uncorrelated fitness landscapes and how to tell the difference. Biol Cybern 63:325–336CrossRefzbMATHGoogle Scholar
  62. Whitley D, Watson JP (2005) Complexity theory and the no free lunch theorem. In: Search methodologies, Springer, pp 317–339Google Scholar
  63. Whitley D, Lunacek M, Sokolov A (2006) Comparing the niches of CMA-ES, CHC and pattern search using diverse benchmarks. In: Runarsson et al TP (ed) Parallel problem solving from nature—PPSN IX. Lecture notes in computer science, vol 4193, Springer, pp 988–997Google Scholar
  64. Wood GR, Zhang BP (1996) Estimation of the Lipschitz constant of a function. J Glob Optim 8:91–103MathSciNetCrossRefzbMATHGoogle Scholar
  65. Zhang W (2004) Phase transitions and backbones of the asymmetric traveling salesman problem. J Artif Intell Res 21(1):471–497MathSciNetzbMATHGoogle Scholar
  66. Zhang W, Korf RE (1996) A study of complexity transitions on the asymmetric traveling salesman problem. Artif Intell 81(1–2):223–239MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  1. 1.School of Information Technology and Electrical EngineeringUniversity of QueenslandBrisbaneAustralia

Personalised recommendations