Advertisement

A No-Free-Lunch theorem for non-uniform distributions of target functions

  • Christian Igel
  • Marc Toussaint
Article

Abstract

The sharpened No-Free-Lunch-theorem (NFL-theorem) states that, regardless of the performance measure, the performance of all optimization algorithms averaged uniformly over any finite set F of functions is equal if and only if F is closed under permutation (c.u.p.). In this paper, we first summarize some consequences of this theorem, which have been proven recently: The number of subsets c.u.p. can be neglected compared to the total number of possible subsets. In particular, problem classes relevant in practice are not likely to be c.u.p. The average number of evaluations needed to find a desirable (e.g., optimal) solution can be calculated independent of the optimization algorithm in certain scenarios. Second, as the main result, the NFL-theorem is extended. Necessary and sufficient conditions for NFL-results to hold are given for arbitrary distributions of target functions. This yields the most general NFL-theorem for optimization presented so far.

Key words

evolutionary computation No-Free-Lunch theorem 

Mathematics Subject Classifications (2000)

90C27 68T20 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Droste, S., Jansen, T., Wegener, I. 2002Optimization with randomized search heuristics - The (A)NFL theorem, realistic scenarios, and difficult functionsTheoret. Comput. Sci.287131144Google Scholar
  2. 2.
    English, T.M.: Evaluation of evolutionary and genetic optimizers: No free lunch, in L. J. Fogel, P. J. Angeline and T. Bäck (eds), Proceedings of the Fifth Annual Conference on Evolutionary Programming (EP V), 1996, pp. 163–169.Google Scholar
  3. 3.
    English, T. M.: Optimization is easy and learning is hard in the typical function, in A. Zalzala, C. Fonseca, J.-H. Kim and A. Smith (eds), Proceedings of the 2000 Congress on Evolutionary Computation (CEC 2000), 2000, pp. 924–931.Google Scholar
  4. 4.
    Igel, C. and Stagge, P.: Graph isomorphisms effect structure optimization of neural networks, in International Joint Conference on Neural Networks (IJCNN 2002), 2002, pp. 142–147.Google Scholar
  5. 5.
    Igel, C., Toussaint, M. 2003Neutrality and self-adaptationNatural Computing2117132Google Scholar
  6. 6.
    Igel, C., Toussaint, M. 2003On classes of functions for which no free lunch results holdInform. Process. Lett.86317321Google Scholar
  7. 7.
    Igel, C. and Toussaint, M.: Recent results on no-free-lunch theorems for optimization, arXiv preprint cs.NE/0303032, http://arxiv.org/abs/cs.NE/0303032, 2003.Google Scholar
  8. 8.
    Köppen, M., Wolpert, D. H., Macready, W. G. 1995Remarks on a recent paper on the “No Free Lunch” theoremsIEEE Trans. Evolut. Comput.5295296Google Scholar
  9. 9.
    Radcliffe, N. J., Surry, P. D. 1995Fundamental limitations on search algorithms: Evolutionary computing in perspectiveLecture Notes in Comput. Sci.1000275291Google Scholar
  10. 10.
    Schumacher, C.: Fundamental limitations of search, Ph.D. thesis, University of Tennessee, 2000.Google Scholar
  11. 11.
    Schumacher, C., Vose, M. D. and Whitley, L. D.: The no free lunch and description length, in L. Spector, E. Goodman, A. Wu, W. Langdon, H.-M. Voigt, M. Gen, S. Sen, M. Dorigo, S. Pezeshk, M. Garzon and E. Burke (eds), Genetic and Evolutionary Computation Conference (GECCO 2001), 2001, pp. 565–570.Google Scholar
  12. 12.
    Streeter, M. J.: Two broad classes of functions for which a no free lunch result does not hold, in E. Cantú-Paz, J. A. Foster, K. Deb, D. Davis, R. Roy, U.-M. O’Reilly, H.-G. Beyer, R. Standish, G. Kendall, S. Wilson, M. Harman, J. Wegener, D. Dasgupta, M. A. Potter, A. C. Schultz, K. Dowsland, N. Jonoska and J. Miller (eds), Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2003), 2003, pp. 1418–1430.Google Scholar
  13. 13.
    Whitley, D. 1999A free lunch proof for gray versus binary encodingsProceedings of the Genetic and Evolutionary Computation Conference (GECCO 1999)1726733Google Scholar
  14. 14.
    Wolpert, D. H., Macready, W. G. 1995No free lunch theorems for search, Technical Report SFI-TR-05-010Santa Fe InstituteSanta Fe, NM, USAGoogle Scholar
  15. 15.
    Wolpert, D. H., Macready, W. G. 1997No free lunch theorems for optimizationIEEE Trans. Evolut. Comput.16782Google Scholar

Copyright information

© Kluwer Academic Publishers 2004

Authors and Affiliations

  • Christian Igel
    • 1
  • Marc Toussaint
    • 1
  1. 1.Chair of Theoretical Biology, Institut für NeuroinformatikRuhr-Universität BochumBochumGermany

Personalised recommendations