Falsification and Future Performance

  • David Balduzzi
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7070)


We information-theoretically reformulate two measures of capacity from statistical learning theory: empirical VC-entropy and empirical Rademacher complexity. We show these capacity measures count the number of hypotheses about a dataset that a learning algorithm falsifies when it finds the classifier in its repertoire minimizing empirical risk. It then follows from that the future performance of predictors on unseen data is controlled in part by how many hypotheses the learner falsifies. As a corollary we show that empirical VC-entropy quantifies the message length of the true hypothesis in the optimal code of a particular probability distribution, the so-called actual repertoire.


Unlabeled Data Future Performance Optimal Code Empirical Risk Hypothesis Space 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Balduzzi, D., Tononi, G.: Integrated Information in Discrete Dynamical Systems: Motivation and Theoretical Framework. PLoS Comput. Biol. 4(6), e1000091 (2008)Google Scholar
  2. 2.
    Balduzzi, D., Tononi, G.: Qualia: the geometry of integrated information. PLoS Comput. Biol. 5(8), e1000462 (2009)Google Scholar
  3. 3.
    Boucheron, S., Lugosi, G., Massart, P.: A Sharp Concentration Inequality with Applications. Random Structures and Algorithms 16(3), 277–292 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Bousquet, O., Boucheron, S., Lugosi, G.: Introduction to Statistical Learning Theory. In: Bousquet, O., von Luxburg, U., Rätsch, G. (eds.) Machine Learning 2003. LNCS (LNAI), vol. 3176, pp. 169–207. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  5. 5.
    Corfield, D., Schölkopf, B., Vapnik, V.: Falsification and Statistical Learning Theory: Comparing the Popper and Vapnik-Chervonenkis Dimensions. Journal for General Philosophy of Science 40(1), 51–58 (2009)CrossRefGoogle Scholar
  6. 6.
    Dowe, D.L.: MML, hybrid Bayesian network graphical models, statistical consistency, invariance and uniqueness. In: Handbook of the Philosophy of Science. Philosophy of Statistics, vol. 7, pp. 901–982. Elsevier (2011)Google Scholar
  7. 7.
    Harman, G., Kulkarni, S.: Reliable Reasoning: Induction and Learning Theory. MIT Press (2007)Google Scholar
  8. 8.
    Koltchinskii, V.: Rademacher penalties and structural risk minimization. IEEE Trans. Inf. Theory 47, 1902–1914 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Lewis, D.: On the Plurality of Worlds. Basil Blackwell, Oxford (1986)Google Scholar
  10. 10.
    Maynard Smith, J.: The Concept of Information in Biology. Philosophy of Science 67, 177–194 (2000)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Popper, K.: The Logic of Scientific Discovery. Hutchinson (1959)Google Scholar
  12. 12.
    Solomonoff, R.J.: A formal theory of inductive inference I, II. Inform. Control 7, 1–22, 224-254 (1964)Google Scholar
  13. 13.
    Vapnik, V.: Estimation of Dependencies Based on Empirical Data. Springer (1982)Google Scholar
  14. 14.
    Vapnik, V.: Statistical Learning Theory. John Wiley & Sons (1998)Google Scholar
  15. 15.
    Wallace, C.S.: Statistical and Inductive Inference by Minimum Message Length. Springer (2005)Google Scholar
  16. 16.
    Wallace, C.S., Boulton, D.M.: An information measure for classification. The Computer Journal 11, 185–194 (1968)CrossRefzbMATHGoogle Scholar
  17. 17.
    Wallace, C.S., Dowe, D.L.: Minimum Message Length and Kolmogorov Complexity. The Computer Journal 42(4), 270–283 (1999)CrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • David Balduzzi
    • 1
  1. 1.MPI for Intelligent SystemsTuebingenGermany

Personalised recommendations