Transductive Rademacher Complexity and Its Applications

  • Ran El-Yaniv
  • Dmitry Pechyony
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4539)

Abstract

We present data-dependent error bounds for transductive learning based on transductive Rademacher complexity. For specific algorithms we provide bounds on their Rademacher complexity based on their “unlabeled-labeled” decomposition. This decomposition technique applies to many current and practical graph-based algorithms. Finally, we present a new PAC-Bayesian bound for mixtures of transductive algorithms based on our Rademacher bounds.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Balcan, M.F., Blum, A.: An Augmented PAC Model for Semi-Supervised Learning (chapter 22). In: Chapelle, O., Schölkopf, B., Zien, A. (eds.) Semi-Supervised Learning, pp. 383–404. MIT Press, Cambridge (2006)Google Scholar
  2. 2.
    Bartlett, P., Bousquet, O., Mendelson, S.: Local Rademacher complexities. Annals of Probability 33(4), 1497–1537 (2005)MATHMathSciNetGoogle Scholar
  3. 3.
    Bartlett, P., Mendelson, S.: Rademacher and Gaussian complexities: risk bounds and structural results. Journal of Machine Learning Research 3, 463–482 (2002)CrossRefMathSciNetGoogle Scholar
  4. 4.
    Belkin, M., Matveeva, I., Niyogi, P.: Regularization and semi-supervised learning on large graphs. In: Shawe-Taylor, J., Singer, Y. (eds.) COLT 2004. LNCS (LNAI), vol. 3120, pp. 624–638. Springer, Heidelberg (2004)Google Scholar
  5. 5.
    Belkin, M., Niyogi, P.: Semi-supervised learning on Riemannian manifolds. Machine Learning 56, 209–239 (2004)MATHCrossRefGoogle Scholar
  6. 6.
    Blum, A., Langford, J.: PAC-MDL Bounds. In: COLT, pp. 344–357. Springer, Heidelberg (2003)Google Scholar
  7. 7.
    Bousquet, O., Elisseeff, A.: Stability and generalization. Journal of Machine Learning Research 2, 499–526 (2002)MATHCrossRefMathSciNetGoogle Scholar
  8. 8.
    Chapelle, O., Schölkopf, B., Zien, A.: Semi-Supervised Learning. MIT Press, Cambridge, MA (2006), http://www.kyb.tuebingen.mpg.de/ssl-book Google Scholar
  9. 9.
    Derbeko, P., El-Yaniv, R., Meir, R.: Explicit learning curves for transduction and application to clustering and compression algorithms. Journal of Artificial Intelligence Research 22, 117–142 (2004)MATHMathSciNetGoogle Scholar
  10. 10.
    Devroye, L., Gyorfi, L., Lugosi, G.: A Probabilistic Theory of Pattern Recognition. Springer, Heidelberg (1996)MATHGoogle Scholar
  11. 11.
    El-Yaniv, R., Gerzon, L.: Effective transductive learning via objective model selection. Pattern Recognition Letters 26, 2104–2115 (2005)CrossRefGoogle Scholar
  12. 12.
    El-Yaniv, R., Pechyony, D.: Stable transductive learning. In: Lugosi, G., Simon, H.U. (eds.) Proceedings of the 19th Annual Conference on Learning Theory, pp. 35–49 (2006)Google Scholar
  13. 13.
    Hanneke, S.: An analysis of graph cut size for transductive learning. In: ICML, pp. 393–399 (2006)Google Scholar
  14. 14.
    Herbster, M., Pontil, M., Wainer, L.: Online learning over graphs. In: ICML, pp. 305–312 (2005)Google Scholar
  15. 15.
    Joachims, T.: Transductive learning via spectral graph partitioning. In: Proceedings of the 20th International Conference on Machine Learning, pp. 290–297 (2003)Google Scholar
  16. 16.
    Lanckriet, G., Cristianini, N., Bartlett, P., Ghaoui, L.E., Jordan, M.: Learning the Kernel Matrix with Semidefinite Programming. Journal of Machine Learning Research 5, 27–72 (2004)Google Scholar
  17. 17.
    Meir, R., Zhang, T.: Generalization error bounds for Bayesian Mixture Algorithms. Journal of Machine Learning Research 4, 839–860 (2003)CrossRefMathSciNetGoogle Scholar
  18. 18.
    Scholkopf, B., Herbrich, R., Smola, A.: A generalized representer theorem. In: Helmbold, D., Williamson, B. (eds.) COLT 2001 and EuroCOLT 2001. LNCS (LNAI), vol. 2111, pp. 416–426. Springer, Heidelberg (2001)Google Scholar
  19. 19.
    Vapnik, V., Chervonenkis, A.: The theory of pattern recognition. Moscow: Nauka (1974)Google Scholar
  20. 20.
    Vapnik, V.N.: Estimation of Dependences Based on Empirical Data. Springer, Heidelberg (1982)MATHGoogle Scholar
  21. 21.
    Zhang, T., Ando, R.: Analysis of spectral kernel design based semi-supervised learning. In: NIPS, pp. 1601–1608 (2005)Google Scholar
  22. 22.
    Zhou, D., Bousquet, O., Lal, T.N., Weston, J., Scholkopf, B.: Learning with local and global consistency. In: NIPS, pp. 321–328 (2003)Google Scholar
  23. 23.
    Zhu, X., Ghahramani, Z., Lafferty, J.D.: Semi-supervised learning using gaussian fields and harmonic functions. In: ICML, pp. 912–919 (2003)Google Scholar

Copyright information

© Springer Berlin Heidelberg 2007

Authors and Affiliations

  • Ran El-Yaniv
    • 1
  • Dmitry Pechyony
    • 1
  1. 1.Computer Science Department, Technion - Israel Institute of Technology 

Personalised recommendations