Abstract
When the number of labeled data is not sufficient, Semi-Supervised Learning (SSL) methods utilize unlabeled data to enhance classification. Recently, many SSL methods have been developed based on the manifold assumption in a batch mode. However, when data arrive sequentially and in large quantities, both computation and storage limitations become a bottleneck. In this paper, we present a new semi-supervised coarse graining (CG) algorithm to reduce the required number of data points for preserving the manifold structure. First, an equivalent formulation of Label Propagation (LP) is derived. Then a novel spectral view of the Harmonic Solution (HS) is proposed. Finally an algorithm to reduce the number of data points while preserving the manifold structure is provided and a theoretical analysis on preservation of the LP properties is presented. Experimental results on real world datasets show that the proposed method outperforms the state of the art coarse graining algorithm in different settings.
Chapter PDF
Similar content being viewed by others
Keywords
References
Zhu, X.: Semi-Supervised Learning Literature Survey. Technical Report 1530, Department of Computer Sciences, University of Wisconsin Madison (2005)
Chapelle, O., Scholkopf, B., Zien, A.: Semi-supervised Learning. MIT Press, Cambridge (2006)
Belkin, M., Niyogi, P., Sindhwani, V.: Manifold Regularization: a Geometric Framework for Learning from Labeled and Unlabeled Examples. Journal of Machine Learning Research 7, 2399–2434 (2006)
Duchenne, O., Audibert, J., Keriven, R., Ponce, J., Segonne, F.: Segmentation by Transduction. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2008, pp. 1–8 (2008)
Belkin, M., Niyogi, P.: Using Manifold Structure for Partially Labeled Classification. Advances in Neural Information Processing Systems 15, 929–936 (2003)
Grabner, H., Leistner, C., Bischof, H.: Semi-supervised On-Line Boosting for Robust Tracking. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 234–247. Springer, Heidelberg (2008)
He., X.: Incremental Semi-Supervised Subspace Learning for Image Retrieval. In: Proceedings of the ACM Conference on Multimedia (2004)
Moh, Y., Buhmann, J.M.: Manifold Regularization for Semi-Supervised Sequential Learning. In: ICASSP (2009)
Goldberg, A., Li, M., Zhu, X.: Online Manifold Regularization: A New Learning Setting and Empirical Study. In: Proceeding of ECML (2008)
Dasgupta, S., Freund, Y.: Random Projection Trees and Low Dimensional Manifolds. Technical Report CS2007-0890, University of California, San Diego (2007)
Valko, M., Kveton, B., Ting, D., Huang, L.: Online Semi-Supervised Learning on Quantized Graphs. In: Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence, UAI (2010)
Lafon, S., Lee, A.B.: Diffusion Maps and Coarse-Graining: A Unified Framework for Dimensionality Reduction, Graph Partitioning, and Data Set Parameterization. IEEE Transactions on Pattern Analysis and Machine Intelligence 28(9), 1393–1403 (2006)
Zhou, D., Bousquet, O., Lal, T., Weston, J., Scholkopf, B.: Learning with local and global consistency. Neural Information Processing Systems (2004)
Zhu, X., Ghahramani, Z., Lafferty, J.: Semi-Supervised Learning Using Gaussian Fields and Harmonic Functions. In: ICML (2003)
Zhu, X., Ghahramani, Z.: Learning from Labeled and Unlabeled Data with Label Propagation. Technical Report CMU-CALD-02-107, Carnegie Mellon University (2002)
Press, W.H., Teukolsky, S.A., Vetterling, W.T., Flannery, B.P.: Numerical Recipes, The Art of Scientific Computing, 3rd edn. Cambridge University Press, Cambridge (2007)
Gfeller, D., De Los Rios, P.: Spectral Coarse Graining of Complex Networks. Physical Review Letters 99, 3 (2007)
Frank, A., Asuncion, A.: UCI Machine Learning Repository (2010)
Fei, L., Fergus, R., Perona, P.: Learning Generative Visual Models From Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories. In: IEEE CVPR 2004, Workshop on Generative Model Based Vision (2004)
Chatzichristofis, S.A., Boutalis, Y.S.: CEDD: Color and Edge Directivity Descriptor: A Compact Descriptor for Image Indexing and Retrieval. In: ICVS, pp. 312–322 (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Farajtabar, M., Shaban, A., Rabiee, H.R., Rohban, M.H. (2011). Manifold Coarse Graining for Online Semi-supervised Learning. In: Gunopulos, D., Hofmann, T., Malerba, D., Vazirgiannis, M. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2011. Lecture Notes in Computer Science(), vol 6911. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-23780-5_35
Download citation
DOI: https://doi.org/10.1007/978-3-642-23780-5_35
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-23779-9
Online ISBN: 978-3-642-23780-5
eBook Packages: Computer ScienceComputer Science (R0)