Skip to main content

Manifold Coarse Graining for Online Semi-supervised Learning

  • Conference paper

Part of the Lecture Notes in Computer Science book series (LNAI,volume 6911)

Abstract

When the number of labeled data is not sufficient, Semi-Supervised Learning (SSL) methods utilize unlabeled data to enhance classification. Recently, many SSL methods have been developed based on the manifold assumption in a batch mode. However, when data arrive sequentially and in large quantities, both computation and storage limitations become a bottleneck. In this paper, we present a new semi-supervised coarse graining (CG) algorithm to reduce the required number of data points for preserving the manifold structure. First, an equivalent formulation of Label Propagation (LP) is derived. Then a novel spectral view of the Harmonic Solution (HS) is proposed. Finally an algorithm to reduce the number of data points while preserving the manifold structure is provided and a theoretical analysis on preservation of the LP properties is presented. Experimental results on real world datasets show that the proposed method outperforms the state of the art coarse graining algorithm in different settings.

Keywords

  • Semi-Supervised Learning
  • Manifold Assumption
  • Harmonic Solution
  • Label Propagation
  • Spectral Coarse Graining
  • Online Classification

References

  1. Zhu, X.: Semi-Supervised Learning Literature Survey. Technical Report 1530, Department of Computer Sciences, University of Wisconsin Madison (2005)

    Google Scholar 

  2. Chapelle, O., Scholkopf, B., Zien, A.: Semi-supervised Learning. MIT Press, Cambridge (2006)

    CrossRef  Google Scholar 

  3. Belkin, M., Niyogi, P., Sindhwani, V.: Manifold Regularization: a Geometric Framework for Learning from Labeled and Unlabeled Examples. Journal of Machine Learning Research 7, 2399–2434 (2006)

    MathSciNet  MATH  Google Scholar 

  4. Duchenne, O., Audibert, J., Keriven, R., Ponce, J., Segonne, F.: Segmentation by Transduction. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2008, pp. 1–8 (2008)

    Google Scholar 

  5. Belkin, M., Niyogi, P.: Using Manifold Structure for Partially Labeled Classification. Advances in Neural Information Processing Systems 15, 929–936 (2003)

    Google Scholar 

  6. Grabner, H., Leistner, C., Bischof, H.: Semi-supervised On-Line Boosting for Robust Tracking. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 234–247. Springer, Heidelberg (2008)

    CrossRef  Google Scholar 

  7. He., X.: Incremental Semi-Supervised Subspace Learning for Image Retrieval. In: Proceedings of the ACM Conference on Multimedia (2004)

    Google Scholar 

  8. Moh, Y., Buhmann, J.M.: Manifold Regularization for Semi-Supervised Sequential Learning. In: ICASSP (2009)

    Google Scholar 

  9. Goldberg, A., Li, M., Zhu, X.: Online Manifold Regularization: A New Learning Setting and Empirical Study. In: Proceeding of ECML (2008)

    Google Scholar 

  10. Dasgupta, S., Freund, Y.: Random Projection Trees and Low Dimensional Manifolds. Technical Report CS2007-0890, University of California, San Diego (2007)

    Google Scholar 

  11. Valko, M., Kveton, B., Ting, D., Huang, L.: Online Semi-Supervised Learning on Quantized Graphs. In: Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence, UAI (2010)

    Google Scholar 

  12. Lafon, S., Lee, A.B.: Diffusion Maps and Coarse-Graining: A Unified Framework for Dimensionality Reduction, Graph Partitioning, and Data Set Parameterization. IEEE Transactions on Pattern Analysis and Machine Intelligence 28(9), 1393–1403 (2006)

    CrossRef  Google Scholar 

  13. Zhou, D., Bousquet, O., Lal, T., Weston, J., Scholkopf, B.: Learning with local and global consistency. Neural Information Processing Systems (2004)

    Google Scholar 

  14. Zhu, X., Ghahramani, Z., Lafferty, J.: Semi-Supervised Learning Using Gaussian Fields and Harmonic Functions. In: ICML (2003)

    Google Scholar 

  15. Zhu, X., Ghahramani, Z.: Learning from Labeled and Unlabeled Data with Label Propagation. Technical Report CMU-CALD-02-107, Carnegie Mellon University (2002)

    Google Scholar 

  16. Press, W.H., Teukolsky, S.A., Vetterling, W.T., Flannery, B.P.: Numerical Recipes, The Art of Scientific Computing, 3rd edn. Cambridge University Press, Cambridge (2007)

    MATH  Google Scholar 

  17. Gfeller, D., De Los Rios, P.: Spectral Coarse Graining of Complex Networks. Physical Review Letters 99, 3 (2007)

    CrossRef  Google Scholar 

  18. Frank, A., Asuncion, A.: UCI Machine Learning Repository (2010)

    Google Scholar 

  19. Fei, L., Fergus, R., Perona, P.: Learning Generative Visual Models From Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories. In: IEEE CVPR 2004, Workshop on Generative Model Based Vision (2004)

    Google Scholar 

  20. Chatzichristofis, S.A., Boutalis, Y.S.: CEDD: Color and Edge Directivity Descriptor: A Compact Descriptor for Image Indexing and Retrieval. In: ICVS, pp. 312–322 (2008)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2011 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Farajtabar, M., Shaban, A., Rabiee, H.R., Rohban, M.H. (2011). Manifold Coarse Graining for Online Semi-supervised Learning. In: Gunopulos, D., Hofmann, T., Malerba, D., Vazirgiannis, M. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2011. Lecture Notes in Computer Science(), vol 6911. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-23780-5_35

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-23780-5_35

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-23779-9

  • Online ISBN: 978-3-642-23780-5

  • eBook Packages: Computer ScienceComputer Science (R0)