Advertisement

Discrete & Computational Geometry

, Volume 44, Issue 3, pp 589–598 | Cite as

VC Dimensions of Principal Component Analysis

  • Yohji Akama
  • Kei Irie
  • Akitoshi Kawamura
  • Yasutaka Uwano
Article

Abstract

Motivated by statistical learning theoretic treatment of principal component analysis, we are concerned with the set of points in ℝ d that are within a certain distance from a k-dimensional affine subspace. We prove that the VC dimension of the class of such sets is within a constant factor of (k+1)(dk+1), and then discuss the distribution of eigenvalues of a data covariance matrix by using our bounds of the VC dimensions and Vapnik’s statistical learning theory. In the course of the upper bound proof, we provide a simple proof of Warren’s bound of the number of sign sequences of real polynomials.

VC dimensions Principal component analysis Warren’s bound 

References

  1. 1.
    Basu, S., Pollack, R., Roy, M.-F.: On the Betti numbers of sign conditions. Proc. Am. Math. Soc. 133(4), 965–974 (2005) (electronic) MATHCrossRefMathSciNetGoogle Scholar
  2. 2.
    Basu, S., Pollack, R., Roy, M.-F.: An asymptotically tight bound on the number of connected components of realizable sign conditions. Combinatorica (to appear) Google Scholar
  3. 3.
    Blumer, A., Ehrenfeucht, A., Haussler, D., Warmuth, M.K.: Learnability and the Vapnik–Chervonenkis dimension. J. Assoc. Comput. Mach. 36, 929–965 (1989) MATHMathSciNetGoogle Scholar
  4. 4.
    Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. Wiley-Interscience, New York (2001) MATHGoogle Scholar
  5. 5.
    Dudley, R.M.: Balls in R k do not cut all subsets of k+2 points. Adv. Math. 31(3), 306–308 (1979) MATHCrossRefMathSciNetGoogle Scholar
  6. 6.
    Johnstone, I.M.: On the distribution of the largest eigenvalue in principal components analysis. Ann. Stat. 29(2), 295–327 (2001) MATHCrossRefMathSciNetGoogle Scholar
  7. 7.
    Massart, P.: Concentration Inequalities and Model Selection. Lecture Notes in Mathematics, vol. 1896. Springer, Berlin (2007). Lectures from the 33rd Summer School on Probability Theory Held in Saint-Flour, 6–23 July 2003, With a Foreword by Jean Picard MATHGoogle Scholar
  8. 8.
    Matoušek, J.: Lectures on Discrete Geometry. Graduate Texts in Mathematics, vol. 212. Springer, New York (2002) MATHGoogle Scholar
  9. 9.
    Mendelson, S., Vershynin, R.: Entropy and the combinatorial dimension. Invent. Math. 152(1), 37–55 (2003) MATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    Milnor, J.W.: On the Betti numbers of real varieties. Proc. Am. Math. Soc. 15, 275–280 (1964) MATHMathSciNetGoogle Scholar
  11. 11.
    Milnor, J.W.: Topology from the Differentiable Viewpoint. Princeton Landmarks in Mathematics. Princeton University Press, Princeton (1997) MATHGoogle Scholar
  12. 12.
    Vapnik, V.N.: Statistical Learning Theory. Adaptive and Learning Systems for Signal Processing, Communications, and Control. Wiley, New York (1998) MATHGoogle Scholar
  13. 13.
    Warren, H.E.: Lower bounds for approximation by nonlinear manifolds. Trans. Am. Math. Soc. 133, 167–178 (1968) MATHMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  • Yohji Akama
    • 1
  • Kei Irie
    • 2
  • Akitoshi Kawamura
    • 3
  • Yasutaka Uwano
    • 1
  1. 1.Mathematical InstituteTohoku UniversitySendaiJapan
  2. 2.Department of MathematicsKyoto UniversityKyotoJapan
  3. 3.Department of Computer ScienceUniversity of TorontoTorontoCanada

Personalised recommendations