Soft Computing

, Volume 16, Issue 12, pp 2115–2133 | Cite as

Arbitrary function optimisation with metaheuristics

No free lunch and real-world problems
  • Carlos García-MartínezEmail author
  • Francisco J. Rodriguez
  • Manuel Lozano
Original Paper


No free lunch theorems for optimisation suggest that empirical studies on benchmarking problems are pointless, or even cast negative doubts, when algorithms are being applied to other problems not clearly related to the previous ones. Roughly speaking, reported empirical results are not just the result of algorithms’ performances, but the benchmark used therein as well; and consequently, recommending one algorithm over another for solving a new problem might be always disputable. In this work, we propose an empirical framework, arbitrary function optimisation framework, that allows researchers to formulate conclusions independent of the benchmark problems that were actually addressed, as long as the context of the problem class is mentioned. Experiments on sufficiently general scenarios are reported with the aim of assessing this independence. Additionally, this article presents, to the best of our knowledge, the first thorough empirical study on the no free lunch theorems, which is possible thanks to the application of the proposed methodology, and whose main result is that no free lunch theorems unlikely hold on the set of binary real-world problems. In particular, it is shown that exploiting reasonable heuristics becomes more beneficial than random search when dealing with binary real-world applications.


Empirical studies No free lunch theorems Real-world problems General-purpose algorithms Unbiased results 



Beliefs usually need to be critically analysed before becoming real knowledge. Being loyal to this idea, the authors would like to express that this study would not have been initiated without the fact that, their journal submissions proposing new approaches, and analysed on many different kinds of problems, were sometimes rejected on the claim that '’according to the NFL, if your proposal wins, then it loses on the rest of problems that have not been analysed”. Therefore and being honest with ourselves, this study, we are really glad of having developed, is in part thanks to the corresponding reviewers and deciding editors’ comments that put us on the way.


  1. Auger A, Teytaud O (2007) Continuous lunches are free. In: Proceedings of the genetic and evolutionary computation conference, ACM Press, New York, pp 916–922Google Scholar
  2. Auger A, Teytaud O (2008) Continuous lunches are free plus the design of optimal optimization algorithms. Algorithmica 57(1):121–146. doi: 10.1007/s00453-008-9244-5 Google Scholar
  3. Barr RS, Golden BL, Kelly JP, Resende MG, Stewart WRJ (1995) Designing and reporting on computational experiments with heuristic methods. J Heuristics 1:9–32Google Scholar
  4. Beasley J (1998) Heuristic algorithms for the unconstrained binary quadratic programming problem. Techical report, The Management School, Imperial CollegeGoogle Scholar
  5. Blancke S, Boudry M, Braeckman J (2010) Simulation of biological evolution under attack, but not really: a response to Meester. Biol Philos 26(1):113–118. doi: 10.1007/s10539-009-9192-8 Google Scholar
  6. Blum C, Puchinger J, Raidl GR, Roli A (2011) Hybrid metaheuristics in combinatorial optimization: a survey. Appl Soft Comput 11(6):4135–4151CrossRefGoogle Scholar
  7. Chen DS, Batson R, Dang Y (2010) Applied integer programming: modeling and solution. Wiley, ChichesterGoogle Scholar
  8. Corne DW, Knowles JD (2003) No free lunch and free leftovers theorems for multiobjective optimisation problems. Evol Multi Crit Optim LNCS 2632:327–341. doi: 10.1007/3-540-36970-8_23
  9. Dembski WA, Marks II RJ (2009) Conservation of information in aearch: measuring the cost of auccess. IEEE Trans Syst Man Cybernet Part A 39(5):1051–1061. doi: 10.1109/TSMCA.2009.2025027 Google Scholar
  10. Dembski WA, Marks II RJ (2010) The Search for a aearch: measuring the information cost of higher level search. J Advan Comput Intell Intell Informatics 14(5):475–486Google Scholar
  11. Derrac J, García S, Molina D, Herrera F (2011) A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm Evol Comput 1(1):3–18CrossRefGoogle Scholar
  12. Droste S, Jansen T, Wegener I (1999) Perhaps not a free lunch but at least a free appetizer. In: Proceedings of the genetic and evolutionary computation conference (GECCO’99), Morgan Kaufmann, pp 833–839Google Scholar
  13. Droste S, Jansen T, Wegener I (2002) Optimization with randomized search heuristics—the (A)NFL theorem, realistic scenarios, and difficult functions. Theor Comput Sci 287(1):131–144MathSciNetzbMATHCrossRefGoogle Scholar
  14. Eshelman L, Schaffer J (1991) Preventing premature convergence in genetic algorithms by preventing incest. In: Belew R, Booker L (eds) International conference on genetic algorithms. Morgan Kaufmann, San Mateo, pp 115–122Google Scholar
  15. Forrest S, Mitchell M (1993) Relative building block fitness and the building block hypothesis. In: Whitley L (ed) Foundations of genetic algorithms 2. Morgan Kaufmann, San Mateo, pp 109–126Google Scholar
  16. Friedman M (1940) A comparison of alternative tests of significance for the problem of m rankings. Ann Math Stat 11(1):86–92zbMATHCrossRefGoogle Scholar
  17. Garcia S, Fernández A, Luengo J, Herrera F (2009a) A study of statistical techniques and performance measures for genetics-based machine learning: accuracy and interpretability. Soft Comput 13(10):959–977CrossRefGoogle Scholar
  18. Garcia S, Molina D, Lozano M, Herrera F (2009b) A Study on the use of non-parametric tests for analyzing the evolutionary algorithms’ behaviour: a case study on the CEC’2005 special session on real parameter optimization. J Heuristics 15(6):617–644zbMATHCrossRefGoogle Scholar
  19. García-Martínez C, Lozano M (2010) Evaluating a local genetic algorithm as context-independent local search operator for metaheuristics. Soft Comput 14(10):1117–1139CrossRefGoogle Scholar
  20. García-Martínez C, Lozano M, Rodriguez FJ (2011a) Arbitrary function optimization. No free lunch and real-world problems.
  21. García-Martínez C, Rodríguez-Díaz FJ, Lozano M (2011b) Role differentiation and malleable mating for differential evolution: an analysis on large-scale optimisation. Soft Comput 15(11):2109–2126. doi: 10.1007/ s00500-010-0641-8 Google Scholar
  22. García-Martínez C, Lozano M, Rodríguez-Dìaz FJ (2012) A simulated annealing method based on a specialised evolutionary algorithm. Appl Soft Comput 12(2):573–588CrossRefGoogle Scholar
  23. Glover F, Laguna M (1997) Tabu search. Kluwer Academic Publishers, NorwellGoogle Scholar
  24. Goldberg D, Korb B, Deb K (1989) Messy genetic algorithms: motivation, analysis, and first results. Complex Syst 3:493–530MathSciNetzbMATHGoogle Scholar
  25. Gortázar F, Duarte A, Laguna M, Martí R (2010) Black box scatter search for general classes of binary optimization problems. Comput Operat Res 37(11):1977–1986. doi: 10.1016/j.cor.2010.01.013 Google Scholar
  26. Hansen N (2005) Compilation of results on the CEC benchmark function set. Technical report, Institute of Computational Science, ETH Zurich, SwitzerlandGoogle Scholar
  27. Herrera F, Lozano M, Verdegay J (1998) Tackling realcoded genetic algorithms: operators and tools for behavioral analysis. Artif Intell Rev 12(4):265–319zbMATHCrossRefGoogle Scholar
  28. Holm S (1979) A simple sequentially rejective multiple test procedure. Scand J Stat 6:65–70MathSciNetzbMATHGoogle Scholar
  29. Hooker JN (1995) Testing heuristics: We have it all wrong. J Heuristics 1(1):33–42. doi: 10.1007/ BF02430364 Google Scholar
  30. Igel C, Toussaint M (2003) On classes of functions for which no free lunch results hold. Inform Process Lett 86(6):317–321MathSciNetzbMATHCrossRefGoogle Scholar
  31. Igel C, Toussaint M (2004) A no-free-lunch theorem for non-uniform distributions of target functions. J Math Modell Algorithms 3(4):313–322MathSciNetzbMATHCrossRefGoogle Scholar
  32. Iman R, Davenport J (1980) Approximations of the critical region of the Friedman statistic. Commun Stat 9:571–595Google Scholar
  33. Jiang P, Chen Y (2010) Free lunches on the discrete Lipschitz class. Theor Comput Sci 412(17):1614–1628. doi: 10.1016/j.tcs.2010.12.028 Google Scholar
  34. Jünger M, Liebling T, Naddef D, Nemhauser G, Pulleyblank W et al (eds) (2009) 50 Years of integer programming 1958–2008: from the early years to the state-of-the-art. Springer, BerlinGoogle Scholar
  35. Karp R (1972) Reducibility among combinatorial problems. In: Miller R, Thatcher J (eds) Complexity of computer computations. Plenum Press, New York, pp 85–103Google Scholar
  36. Kauffman S (1989) Adaptation on rugged fitness landscapes. Lect Sci Complex 1:527–618Google Scholar
  37. Kirkpatrick S, Gelatt Jr C, Vecchi M (1983) Optimization by simulated annealing. Science 220(4598):671–680MathSciNetzbMATHCrossRefGoogle Scholar
  38. Koehler GJ (2007) Conditions that obviate the no-free- lunch theorems for optimization. INFORMS J Comput 19(2):273–279. doi: 10.1287/ijoc.1060.0194
  39. Laplace PS (1814) Essai philosophique sur les probabilités. Technical report, Paris, CourcierGoogle Scholar
  40. Lozano M, García-Martínez C (2010) Hybrid metaheuristics with evolutionary algorithms specializing in intensification and diversification: overview and progress report. Comput Operat Res 37:481–497zbMATHCrossRefGoogle Scholar
  41. Lozano M, Herrera F, Molina D (eds) (2011) Scalability of evolutionary algorithms and other metaheuristics for large scale continuous optimization problems, vol 15. Soft ComputingGoogle Scholar
  42. Marshall JAR, Hinton TG (2010) Beyond no free lunch: realistic algorithms for arbitrary problem classes. In: IEEE Congr Evol Comput 1:18–23Google Scholar
  43. Pelikan M, Goldberg D, Cantú-Paz E (2000) Linkage problem, distribution estimation, and bayesian networks. Evol Comput 8(3):311–340CrossRefGoogle Scholar
  44. Rodriguez FJ, García-Martínez C, Lozano M (2012) Hybrid metaheuristics based on evolutionary algorithms and simulated annealing: taxonomy, comparison, and synergy test. IEEE Trans Evol Comput. doi: 10.1109/TEVC.2012.2182773
  45. Schaffer J, Eshelman L (1991) On crossover as an evolutionary viable strategy. In: Belew R, Booker L (eds) Proceedings of the international conference on genetic algorithms. Morgan Kaufmann, San Mateo, pp 61–68Google Scholar
  46. Schumacher C, Vose MD, Whitley LD (2001) The No Free Lunch and Problem Description Length. In: Proc. of the Genetic and Evolutionary Computation Conference, pp 565–570Google Scholar
  47. Service TC (2010) A no free lunch theorem for multiobjective optimization. Inform Process Lett 110(21):917–923. doi: 10.1016/j.ipl.2010.07.026 Google Scholar
  48. Smith K, Hoos H, Stützle T (2003) Iterated robust tabu search for MAX-SAT. In: Carbonell J, Siekmann J (eds) Proceedings of the Canadian society for computational studies of intelligence Conference, vol LNCS 2671. Springer, Berlin, pp 129–144Google Scholar
  49. Talbi E (2002) A taxonomy of hybrid metaheuristics. J Heuristics 8(5):541–564CrossRefGoogle Scholar
  50. Thierens D (2002) Adaptive mutation rate control schemes in genetic algorithms. In: Proceedings of the congress on evolutionary computation, pp 980–985Google Scholar
  51. Thierens D (2004) Population-based iterated local search: restricting neighborhood search by crossover. In: Proceedings of the genetic and evolutionary computation conference, vol LNCS 3103. Springer, Berlin, pp 234–245Google Scholar
  52. Watson R, Pollack J (1999) Hierarchically consistent test problems for genetic algorithms. Proc Congr Evol Comput 2:1406–1413Google Scholar
  53. Whitley D, Rowe J (2008) Focused no free lunch theorems. In: Proceedings of the genetic and evolutionary computation conference. ACM Press, New York, pp 811–818. doi: 10.1145/1389095.1389254
  54. Whitley D, Watson JP (2005) Complexity theory and the no free lunch theorem. Search Methodologies, Springer, Berlin, pp 317–339. doi: 10.1007/0-387-28356-0_11
  55. Whitley D, Rana S, Dzubera J, Mathias E (1996) Evaluating evolutionary algorithms. Artif Intell 85:245–276CrossRefGoogle Scholar
  56. Wolpert D, Macready W (1997) No free lunch theorems for optimization. IEEE Trans Evol Comput 1(1):67–82CrossRefGoogle Scholar
  57. Zar J (1999) Biostatistical analysis. Prentice Hall, Upper Saddle RiverGoogle Scholar

Copyright information

© Springer-Verlag 2012

Authors and Affiliations

  • Carlos García-Martínez
    • 1
    Email author
  • Francisco J. Rodriguez
    • 2
  • Manuel Lozano
    • 2
  1. 1.Department of Computing and Numerical AnalysisUniversity of CórdobaCórdobaSpain
  2. 2.Department of Computer Sciences and Artificial Intelligence CITIC-UGRUniversity of GranadaGranadaSpain

Personalised recommendations