Efficient and Robust Semi-supervised Learning Over a Sparse-Regularized Graph

  • Hang SuEmail author
  • Jun Zhu
  • Zhaozheng Yin
  • Yinpeng Dong
  • Bo Zhang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9912)


Graph-based Semi-Supervised Learning (GSSL) has limitations in widespread applicability due to its computationally prohibitive large-scale inference, sensitivity to data incompleteness, and incapability on handling time-evolving characteristics in an open set. To address these issues, we propose a novel GSSL based on a batch of informative beacons with sparsity appropriately harnessed, rather than constructing the pairwise affinity graph between the entire original samples. Specifically, (1) beacons are placed automatically by unifying the consistence of both data features and labels, which subsequentially act as indicators during the inference; (2) leveraging the information carried by beacons, the sample labels are interpreted as the weighted combination of a subset of characteristics-specified beacons; (3) if unfamiliar samples are encountered in an open set, we seek to expand the beacon set incrementally and update their parameters by incorporating additional human interventions if necessary. Experimental results on real datasets validate that our algorithm is effective and efficient to implement scalable inference, robust to sample corruptions, and capable to boost the performance incrementally in an open set by updating the beacon-related parameters.


Semi-supervised learning Beacon Sparse representation Online learning 


  1. 1.
    Chapelle, O., Schlkopf, B., Zien, A.: Semi-Supervised Learning, 1st edn. The MIT Press, Cambrdige (2010)Google Scholar
  2. 2.
    Zhu, X., Goldberg, A.B., Brachman, R., Dietterich, T.: Introduction to Semi-Supervised Learning. Morgan and Claypool Publishers, Cambrdige (2009)Google Scholar
  3. 3.
    Zhu, X.: Semi-supervised learning literature survey. Technical report 1530, Computer Sciences, Carnegie Mellon University (2005)Google Scholar
  4. 4.
    Subramanya, A., Talukdar, P.: Graph-Based Semi-Supervised Learning. Synthesis Lectures on Artificial Intelligence and Machine Learning. Morgan & Claypool Publishers, San Rafael (2014)zbMATHGoogle Scholar
  5. 5.
    Blum, A., Chawla, S.: Learning from labeled and unlabeled data using graph mincuts. In: Proceedings of the Eighteenth International Conference on Machine Learning (ICML), pp. 19–26 (2001)Google Scholar
  6. 6.
    Grady, L.: Random walks for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 28(11), 1768–1783 (2006)CrossRefGoogle Scholar
  7. 7.
    Couprie, C., Grady, L., Najman, L., Talbot, H.: Power watershed: a unifying graph-based optimization framework. IEEE Trans. Pattern Anal. Mach. Intell. 33(7), 1384–1399 (2011)CrossRefGoogle Scholar
  8. 8.
    Belkin, M., Niyogi, P., Sindhwani, V.: Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J. Mach. Learn. Res. 7, 2399–2434 (2006)zbMATHMathSciNetGoogle Scholar
  9. 9.
    Zhu, X., Ghahramani, Z., Lafferty, J.: Semi-supervised learning using Gaussian fields and harmonic functions. In: Twentieth International Conference on Machine Learning (ICML), pp. 912–919 (2003)Google Scholar
  10. 10.
    Zhou, D., Bousquet, O., Lal, T.N., Weston, J., Schölkopf, B.: Learning with local and global consistency. In: Advances in Neural Information Processing Systems (NIPS), pp. 321–328. MIT Press (2004)Google Scholar
  11. 11.
    Cheng, B., Yang, J., Yan, S., Fu, Y., Huang, T.S.: Learning with \(\ell _1\)-graph for image analysis. IEEE Trans. Image Process. 19(4), 858–866 (2010)CrossRefMathSciNetGoogle Scholar
  12. 12.
    Yang, Y., Wang, Z., Yang, J., Wang, J., Chang, S., Huang, T.S.: Data clustering by Laplacian regularized \(\ell _1\)-graph. In: Twenty-Eighth AAAI Conference on Artificial Intelligence (AAAI), pp. 3148–3149 (2014)Google Scholar
  13. 13.
    Talwalkar, A., Kumar, S., Rowley, H.: IEEE Conference on Large-scale manifold learning. In: Computer Vision and Pattern Recognition (CVPR), pp. 1–8. IEEE (2008)Google Scholar
  14. 14.
    Fergus, R., Weiss, Y., Torralba, A.: Semi-supervised learning in gigantic image collections. In: Advances in Neural Information Processing Systems (NIPS), pp. 522–530 (2009)Google Scholar
  15. 15.
    Dai, D., Van Gool, L.: Ensemble projection for semi-supervised image classification. In: 2013 IEEE International Conference on Computer Vision (CVPR), pp. 2072–2079. IEEE (2013)Google Scholar
  16. 16.
    Liu, W., He, J., Chang, S.F.: Large graph construction for scalable semi-supervised learning. In: Proceedings of the 27th International Conference on Machine Learning (ICML), pp. 679–686 (2010)Google Scholar
  17. 17.
    Chen, X., Cai, D.: Large scale spectral clustering with landmark-based representation. In: The 25th AAAI Conference on Artificial Intelligence (AAAI), pp. 313–318 (2011)Google Scholar
  18. 18.
    Kleinberg, J., Slivkins, A., Wexler, T.: Triangulation and embedding using small sets of beacons. In: Proceedings of the 45th Annual IEEE Symposium on Foundations of Computer Science, pp. 444–453, October 2004Google Scholar
  19. 19.
    Zhang, Z., Ladicky, L., Torr, P., Saffari, A.: Learning anchor planes for classification. In: Advances in Neural Information Processing Systems (NIPS), pp. 1611–1619 (2011)Google Scholar
  20. 20.
    Globerson, A., Roweis, S.: Nightmare at test time: robust learning by feature deletion. In: Proceedings of the 23rd international conference on Machine learning (ICML), pp. 353–360 (2006)Google Scholar
  21. 21.
    Kim, S.J., Koh, K., Lustig, M., Boyd, S., Gorinevsky, D.: An interior-point method for large-scale \(\ell _1\)-1-regularized least squares. IEEE J. Sel. Top. Sig. Process. 1(4), 606–617 (2007)CrossRefGoogle Scholar
  22. 22.
    Mairal, J., Bach, F., Ponce, J., Sapiro, G.: Online dictionary learning for sparse coding. In: Proceedings of the 26th Annual International Conference on Machine Learning (ICML), pp. 689–696 (2009)Google Scholar
  23. 23.
    Su, H., Yin, Z., Huh, S., Kanade, T.: Cell segmentation in phase contrast microscopy images via semi-supervised classification over optics-related features. Med. Image Anal. 17, 746–765 (2013)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Hang Su
    • 1
    Email author
  • Jun Zhu
    • 1
  • Zhaozheng Yin
    • 2
  • Yinpeng Dong
    • 1
  • Bo Zhang
    • 1
  1. 1.State Key Lab of Intelligent Technology and Systems, Tsinghua National Lab for Information Science and Technology, Department of Computer Science and Technology, Center for Bio-Inspired Computing ResearchTsinghua UniversityBeijingChina
  2. 2.Department of Computer ScienceMissouri University of Science and TechnologyRollaUSA

Personalised recommendations