Advertisement

Regularized Semi-supervised Classification on Manifold

  • Lianwei Zhao
  • Siwei Luo
  • Yanchang Zhao
  • Lingzhi Liao
  • Zhihai Wang
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3918)

Abstract

Semi-supervised learning gets estimated marginal distribution P X with a large number of unlabeled examples and then constrains the conditional probability p(y | x) with a few labeled examples. In this paper, we focus on a regularization approach for semi-supervised classification. The label information graph is first defined to keep the pairwise label relationship and can be incorporated with neighborhood graph which reflects the intrinsic geometry structure of P X . Then we propose a novel regularized semi-supervised classification algorithm, in which the regularization term is based on the modified Graph Laplacian. By redefining the Graph Laplacian, we can adjust and optimize the decision boundary using the labeled examples. The new algorithm combines the benefits of both unsupervised and supervised learning and can use unlabeled and labeled examples effectively. Encouraging experimental results are presented on both synthetic and real world datasets.

Keywords

Marginal Distribution Regularization Term Decision Boundary Real World Dataset Neighborhood Graph 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Belkin, M., Niyogi, P., Sindhwani, V.: Manifold Regularization: A Geometric Framework for Learning from Examples. Department of Computer Science, University of Chicago, TR-2004-06Google Scholar
  2. 2.
    Belkin, M., Niyogi, P., Sindhwani, V.: On Manifold Regularization. Department of Computer Science, University of Chicago, TR-2004-05Google Scholar
  3. 3.
    Belkin, M., Matveeva, I., Niyogi, P.: Regression and Regularization on Large Graphs. In: Proceedings of the Conference on Computational Learning Theory (2004)Google Scholar
  4. 4.
    Belkin, M., Niyogi, P.: Using Manifold Structure for Partially Labeled Classification. In: NIPS 2002, vol. 15 (2002)Google Scholar
  5. 5.
    Belkin, M., Niyogi, P.: Laplacian Eigenmaps for Dimensionality Reduction and Data Representation, Neural Computation (June 2003)Google Scholar
  6. 6.
    Blum, A., Mitchell, T.: Combining Labeled and Unlabeled Data with Co-training. In: Proceedings of the Conference on Computational Learning Theory (1998)Google Scholar
  7. 7.
    Szummer, M., Jaakkola, T.: Partially Labeled Classification with Markov Random Walks. In: NIPS 2001, vol. 14 (2001)Google Scholar
  8. 8.
    Zhu, X., Ghahramani, Z., Lafferty, J.: Semi-supervised Learning Using Gaussian Fields and Harmonic Functions. In: ICML 2003 (2003)Google Scholar
  9. 9.
    Blum, A., Chawla, S.: Learning from Labeled and Unlabeled Data Using Graph Mincuts. In: ICML 2001 (2001)Google Scholar
  10. 10.
    Zhou, D., Bousquet, O., Lal, T.N., Weston, J., Schoelkopf, B.: Learning with Local and Global Consistency. In: NIPS 2003, vol. 16 (2003)Google Scholar
  11. 11.
    De Vito, E., Rosasco, L., Caponnetto, A., De Giovannini, U., Odone, F.: Learning from Examples as an Inverse Problem. Journal of Machine Learning Research 6, 883–904 (2005)MathSciNetMATHGoogle Scholar
  12. 12.
    Bousquet, O., Chapelle, O., Hein, M.: Measure Based Regularization. In: NIPS 2003, vol. 16 (2003)Google Scholar
  13. 13.
    Szummer, M., Jaakkola, T.: Information Regularization with Partially Labeled Data. In: NIPS 2002, vol. 15 (2002)Google Scholar
  14. 14.
    Krishnapuram, B., Williams, D., Ya, X., Hartemink, A., Carin, L., Figueiredo, M.A.T.: On Semi-Supervised Classification. In: NIPS 2004, vol. 17 (2004)Google Scholar
  15. 15.
    Kegl, B., Ligen, W.: Boosting on Manifolds: Adaptive Regularization of Base Classifiers. In: NIPS 2004, vol. 17 (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Lianwei Zhao
    • 1
  • Siwei Luo
    • 1
  • Yanchang Zhao
    • 2
  • Lingzhi Liao
    • 1
  • Zhihai Wang
    • 1
  1. 1.School of Computer and Information TechnologyBeijing Jiaotong UniversityBeijingChina
  2. 2.Faculty of Information TechnologyUniversity of TechnologySydneyAustralia

Personalised recommendations