Advertisement

Journal of Optimization Theory and Applications

, Volume 115, Issue 3, pp 549–570 | Cite as

Simple Explanation of the No-Free-Lunch Theorem and Its Implications

  • Y.C. Ho
  • D.L. Pepyne
Article

Abstract

The no-free-lunch theorem of optimization (NFLT) is an impossibility theorem telling us that a general-purpose, universal optimization strategy is impossible. The only way one strategy can outperform another is if it is specialized to the structure of the specific problem under consideration. Since optimization is a central human activity, an appreciation of the NFLT and its consequences is essential. In this paper, we present a framework for conceptualizing optimization that leads to a simple but rigorous explanation of the NFLT and its implications.

No-free-lunch theorem optimization learning decision making search strategy selection impossibility theorem representation and encoding robustness sensitivity complexity 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    CULBERSON, J. C., On the Futility of Blind Search, Technical Report TR 96–18, Department of Computing Science, University of Alberta, Edmonton, Alberta, Canada, 1996.Google Scholar
  2. 2.
    CULBERSON, J. C., On the Futility of Blind Search: An Algorithmic View of 'No Free Lunch', Evolutionary Computation, Vol. 6, pp. 109–127, 1998.Google Scholar
  3. 3.
    DUDA, R. O., HART, P. E., and STORK, D. G., Pattern Classification, 2nd Edition, John Wiley and Sons, New York, NY, 2001.Google Scholar
  4. 4.
    HO, Y. C., The No-Free-Lunch Theorem and the Human Machine Interface, IEEE Control Systems Magazine, Vol. 19, pp. 88–90, 1999.Google Scholar
  5. 5.
    KOPPEN, M., WOLPERT, D. H., and MACREADY, W. G., Remarks on a Recent Paper on the No-Free-Lunch Theorems, IEEE Transactions on Evolutionary Computation, Vol. 5, pp. 295–296, 2001.Google Scholar
  6. 6.
    RADCLIFF, N.J., and SURRY, P. D., Fundamental Limitations on Search Algorithms: Evolutionary Computing in Perspective, Lecture Notes in Computer Science, Springer Verlag, New York, NY, Vol. 1000, pp. 275–291, 1995.Google Scholar
  7. 7.
    SHARPE, O., Beyond NFL: A Few Tentatiûe Steps (available online at http://www.cogs.susx.ac.uk/users/olivers/).Google Scholar
  8. 8.
    SHARPE, O., Continuing Beyond NFL: Dissecting Real World Problems (available online at http://www.cogs.susx.ac.uk/users/olivers/).Google Scholar
  9. 9.
    WOLPERT, D. H., and MACREADY, W. G., No-Free-Lunch Theorems for Optimization, IEEE Transactions on Evolutionary Computation, Vol. 1, pp. 67–82, 1997.Google Scholar
  10. 10.
    WOLPERT, D. H., and MACREADY, W. G., No-Free-Lunch Theorems for Search, Technical Report SFI-TR–95-02-010, Santa Fe Institute, Santa Fe, New Mexico, 1995.Google Scholar
  11. 11.
    HO, Y. C., An Explanation of Ordinal Optimization: Soft Computing for Hard Problems, Information Sciences, Vol. 113, pp. 169–192, 1999.Google Scholar
  12. 12.
    LIN, X. C., A New Framework for Discrete Stochastic Optimization, Doctoral Dissertation, Harvard University, 2000.Google Scholar
  13. 13.
    DAI, L., Convergence Properties of Ordinal Comparisons in the Simulation of DEDS, Journal of Optimization Theory and Applications, Vol. 91, pp. 363–388, 1996.Google Scholar
  14. 14.
    DU, D. Z and KO, K. I., Theory of Computational Complexity, Wiley-Interscience, New York, NY, 2000.Google Scholar

Copyright information

© Plenum Publishing Corporation 2002

Authors and Affiliations

  • Y.C. Ho
    • 1
  • D.L. Pepyne
    • 2
  1. 1.Division of Engineering and Applied SciencesHarvard UniversityCambridge
  2. 2.Division of Engineering and Applied SciencesHarvard UniversityCambridge

Personalised recommendations