A Priori Advantages of Meta-Induction and the No Free Lunch Theorem: A Contradiction?
RW enjoys free lunches, i.e., its predictive long-run success dominates that of other prediction strategies.
Yet the NFL theorem applies to online prediction tasks provided the prior distribution is a SUPD.
The SUPD is maximally induction-hostile and assigns a probability of zero to all possible worlds in which RW enjoys free lunches. This dissolves the apparent conflict with the NFL.
The a priori advantages of RW can be demonstrated even under the assumption of a SUPD. Further advantages become apparent when a frequency-uniform distribution is considered.
KeywordsProblem of induction No free lunch theorem Online prediction under expert advice Regret-weighted meta-induction
- 3.Wolpert, D., Macready, W.: No free lunch theorems for search. Technical report SFI-TR-95-02-010, Santa Fe Institute (1995)Google Scholar
- 5.Schaffer, C.: A conservation law for generalization performance. In: Machine Learning (Proceedings of ICML 1994), pp. 259–265. Morgan Kaufmann, Burlington (1994)Google Scholar
- 6.Rao, R., Gordon, D., Spears, W.: For every generalization action, is there really an equal and opposite reaction? In: Machine Learning (Proceedings of ICML 1995), pp. 471–479. Morgan Kaufmann, Burlington (1994)Google Scholar
- 7.Giraud-Carrier, C., Provost, F.: Toward a justification of meta-learning: is the no free lunch theorem a show-stopper? In: Proceedings of the ICML-2005 Workshop on Meta-learning, pp. 12–19 (2006)Google Scholar
- 10.Schurz, G.: Meta-induction is an optimal prediction strategy. In: Proceedings of the 18th Annual Belgian-Dutch Conference on Machine Learning, pp. 66–74. University of Tilburg (2009)Google Scholar
- 11.Gigerenzer, G., Todd, P.M., the ABC Research Group. (eds.): Simple Heuristics That Make Us Smart. Oxford University Press, Dordrecht (1999)Google Scholar
- 12.Schurz, G.: The Optimality of Meta-induction. A New Approach to Hume’s Problem. Book manuscript (2017)Google Scholar
- 14.Howson, C., Urbach, P.: Scientific Reasoning: The Bayesian Approach, 2nd edn. Open Court, Chicago (1996)Google Scholar