Advertisement

A Priori Advantages of Meta-Induction and the No Free Lunch Theorem: A Contradiction?

  • Gerhard Schurz
  • Paul Thorn
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10505)

Abstract

Recently a new account to the problem of induction has been developed [1], based on a priori advantages of regret-weighted meta-induction (RW) in online learning [2]. The claimed a priori advantages seem to contradict the no free lunch (NFL) theorem, which asserts that relative to a state-uniform prior distribution (SUPD) over possible worlds all (non-clairvoyant) prediction methods have the same expected predictive success. In this paper we propose a solution to this problem based on four novel results:
  • RW enjoys free lunches, i.e., its predictive long-run success dominates that of other prediction strategies.

  • Yet the NFL theorem applies to online prediction tasks provided the prior distribution is a SUPD.

  • The SUPD is maximally induction-hostile and assigns a probability of zero to all possible worlds in which RW enjoys free lunches. This dissolves the apparent conflict with the NFL.

  • The a priori advantages of RW can be demonstrated even under the assumption of a SUPD. Further advantages become apparent when a frequency-uniform distribution is considered.

Keywords

Problem of induction No free lunch theorem Online prediction under expert advice Regret-weighted meta-induction 

References

  1. 1.
    Schurz, G.: The meta-inductivist’s winning strategy in the prediction game: a new approach to Hume’s problem. Philos. Sci. 75, 278–305 (2008)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Cesa-Bianchi, N., Lugosi, G.: Prediction, Learning, and Games. Cambridge University Press, Cambridge (2006)CrossRefzbMATHGoogle Scholar
  3. 3.
    Wolpert, D., Macready, W.: No free lunch theorems for search. Technical report SFI-TR-95-02-010, Santa Fe Institute (1995)Google Scholar
  4. 4.
    Wolpert, D.: On the connection between in-sample testing and generalization error. Complex Syst. 6, 47–94 (1992)MathSciNetzbMATHGoogle Scholar
  5. 5.
    Schaffer, C.: A conservation law for generalization performance. In: Machine Learning (Proceedings of ICML 1994), pp. 259–265. Morgan Kaufmann, Burlington (1994)Google Scholar
  6. 6.
    Rao, R., Gordon, D., Spears, W.: For every generalization action, is there really an equal and opposite reaction? In: Machine Learning (Proceedings of ICML 1995), pp. 471–479. Morgan Kaufmann, Burlington (1994)Google Scholar
  7. 7.
    Giraud-Carrier, C., Provost, F.: Toward a justification of meta-learning: is the no free lunch theorem a show-stopper? In: Proceedings of the ICML-2005 Workshop on Meta-learning, pp. 12–19 (2006)Google Scholar
  8. 8.
    Wolpert, D.: The lack of a priori distinctions between learning algorithms. Neural Comput. 8, 1341–1390 (1996)CrossRefGoogle Scholar
  9. 9.
    Schurz, G., Thorn, P.: The revenge of ecological rationality: strategy-selection by meta-induction. Minds Mach. 26(1), 31–59 (2016)CrossRefGoogle Scholar
  10. 10.
    Schurz, G.: Meta-induction is an optimal prediction strategy. In: Proceedings of the 18th Annual Belgian-Dutch Conference on Machine Learning, pp. 66–74. University of Tilburg (2009)Google Scholar
  11. 11.
    Gigerenzer, G., Todd, P.M., the ABC Research Group. (eds.): Simple Heuristics That Make Us Smart. Oxford University Press, Dordrecht (1999)Google Scholar
  12. 12.
    Schurz, G.: The Optimality of Meta-induction. A New Approach to Hume’s Problem. Book manuscript (2017)Google Scholar
  13. 13.
    Carnap, R.: Logical Foundations of Probability, 2nd edn. University of Chicago Press, Chicago (1962)zbMATHGoogle Scholar
  14. 14.
    Howson, C., Urbach, P.: Scientific Reasoning: The Bayesian Approach, 2nd edn. Open Court, Chicago (1996)Google Scholar
  15. 15.
    Solomonoff, R.: A formal theory of inductive inference. Inf. Control 7(1–22), 224–254 (1964)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.DCLPSHeinrich Heine University DüesseldorfDüesseldorfGermany

Personalised recommendations