Abstract
The global smoothness and the local label fitting are two key issues for estimating the function on the graph in graph based semi-supervised learning (GSSL). The unsupervised normalized cut method can provide a more reasonable criterion for learning the global smoothness of the data than classic GSSL methods. However, the semi-supervised norm of the normalized cut, which is a NP-hard problem, has not been studied well. In this paper, a new GSSL framework is proposed by extending normalized cut to its semi-supervised norm. The NP-hard semi-supervised normalized cut problem is innovatively solved by effective algorithms. In addition, we can design more reasonable local label fitting terms than conventional GSSL methods. Other graph cut methods are also investigated to extend the proposed semi-supervised learning algorithms. Furthermore, we incorporate the nonnegative matrix factorization with the proposed learning algorithms to solve the out-of-sample problem in semi-supervised learning. Solutions obtained by the proposed algorithms are sparse, nonnegative and congruent with unit matrix. Experiment results on several real benchmark datasets indicate that the proposed algorithms achieve good results compared with state-of-art methods.
Similar content being viewed by others
References
Chawla NV, Karakoulas GI (2005) Learning from labeled and unlabeled data: an empirical study across techniques and domains. J Artif Intell Res (JAIR) 23:331–366
Chapelle O, Schölkopf B, Zien A (eds) (2006) Semi-supervised learning. MIT Press, Cambridge, MA
Zhu X, Goldberg AB (2009) Introduction to semi-supervised learning. Synth Lect Artif Intell Mach Learn 3(1):1–130
Nie F, Xu D, Tsang IWH, Zhang C (2010) Flexible manifold embedding: a framework for semi-supervised and unsupervised dimension reduction. Image Process IEEE Trans 19(7):1921–1932
Gao Q, Huang Y, Gao X, Shen W, Zhang H (2015) A novel semi-supervised learning for face recognition. Neurocomputing 152:69–76
Cai D, He X, Han J (2007) Semi-supervised discriminant analysis. In: Computer Vision, 2007. ICCV 2007. IEEE 11th international conference on IEEE, pp 1–7
Zhou D, Bousquet O, Lal TN, Weston J, Schölkopf B (2004) Learning with local and global consistency. Adv Neural Inf Process Syst 16(16):321–328
Wang J, Jebara T, Chang SF (2013) Semi-supervised learning using greedy max-cut. J Mach Learn Res 14(1):771–800
Zhu X, Ghahramani Z, Lafferty J (2003) Semi-supervised learning using gaussian fields and harmonic functions. In: ICML, vol 3, pp 912–919
Tang J, Hua XS, Qi GJ, Wang M, Mei T, Wu X (2007) Structure-sensitive manifold ranking for video concept detection. In: Proceedings of the 15th international conference on Multimedia, ACM, pp 852–861
Tang J, Hua XS, Qi GJ, Song Y, Wu X (2008) Video annotation based on kernel linear neighborhood propagation. Multimed IEEE Trans 10(4):620–628
Wang M, Mei T, Yuan X, Song Y, Dai LR (2007) Video annotation by graph-based learning with neighborhood similarity. In: Proceedings of the 15th international conference on multimedia, ACM, pp 325–328
Zhao M, Chow TW, Zhang Z, Li B (2015) Automatic image annotation via compact graph based semi-supervised learning. Knowl-Based Syst 76:148–165
Huang L, Wang Y, Liu X, Lang B (2013) Efficient semi-supervised annotation with proxy-based local consistency propagation. In: Multimedia and Expo (ICME), 2013 IEEE international conference on, IEEE, pp 1–6
Liu S, Yan S, Zhang T, Xu C, Liu J, Lu H (2012) Weakly supervised graph propagation towards collective image parsing. Multimed IEEE Trans 14(2):361–373
Zhao M, Chan RH, Chow TW, Tang P (2014) Compact graph based semi-supervised learning for medical diagnosis in Alzheimer’s disease. Signal Process Lett IEEE 21(10):1192–1196
Wang F, Zhang C (2008) Label propagation through linear neighborhoods. Knowl Data Eng IEEE Trans 20(1):55–67
Zhuang L, Gao H, Lin Z, Ma Y, Zhang X, Yu N (2012) Non-negative low rank and sparse graph for semi-supervised learning. In: Computer vision and pattern recognition (CVPR), 2012 IEEE conference on, IEEE, pp 2328–2335
Ni B, Yan S, Kassim A (2012) Learning a propagable graph for semisupervised learning: classification and regression. Knowl Data Eng IEEE Trans 24(1):114–126
Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res 7:2399–2434
Shi J, Malik J (2000) Normalized cuts and image segmentation. Pattern Anal Mach Intell IEEE Trans 22(8):888–905
Hagen L, Kahng AB (1992) New spectral methods for ratio cut partitioning and clustering. Comput Aid Des Integr Circuit Syst IEEE Trans 11(9):1074–1085
Sarkar S, Soundararajan P (2000) Supervised learning of large perceptual organization: graph spectral partitioning and learning automata. Pattern Anal Mach Intell IEEE Trans 22(5):504–525
Wu Z, Leahy R (1993) An optimal graph theoretic approach to data clustering: theory and its application to image segmentation. Pattern Anal Mach Intell IEEE Trans 15(11):1101–1113
Ding CH, He X, Zha H, Gu M, Simon HD (2001) A min–max cut algorithm for graph partitioning and data clustering. In: Data mining, 2001. ICDM 2001, proceedings IEEE international conference on, IEEE, pp 107–114
Xu L, Li W, Schuurmans D (2009) Fast normalized cut with linear constraints. In: Computer vision and pattern recognition, IEEE conference on, IEEE, pp 2866–2873
Hu H, Feng J, Yu C, Zhou J (2013) Multi-class constrained normalized cut with hard, soft, unary and pairwise priors and its applications to object segmentation. Image Process IEEE Trans 22(11):4328–4340
Yang YT, Fishbain B, Hochbaum DS, Norman EB, Swanberg E (2013) The supervised normalized cut method for detecting, classifying, and identifying special nuclear materials. INFORMS J Comput 26(1):45–58
Kulis B, Basu S, Dhillon I, Mooney R (2009) Semi-supervised graph clustering: a kernel approach. Mach Learn 74(1):1–22
Ren W, Li G, Tu D, Jia L (2014) Nonnegative matrix factorization with regularizations. Emerg Select Topn Circuit Syst IEEE J 4(1):153–164
Von Luxburg U (2007) A tutorial on spectral clustering. Stat Comput 17(4):395–416
Ding CH, He X, Simon HD (2005) On the equivalence of nonnegative matrix factorization and spectral clustering. In: SDM, vol 5, pp 606–610
Ren WY, Li GH, Tu D (2015) Graph clustering by congruency approximation. IET Comput Vis. doi:10.1049/iet-cvi.2014.0131
Kulis B (2012) Metric learning: a survey. Found Trend Mach Learn 5(4):287–364
James W, Stein C (1961) Estimation with quadratic loss. In: Proceedings of the fourth Berkeley symposium on mathematical statistics and probability, vol 1 pp 361–379
Kjeldsen TH (2000) A contextualized historical analysis of the Kuhn–Tucker Theorem in nonlinear programming: the impact of World War II. Hist Math 27(4):331–361
Chapelle O, Zien A (2005) Semi-supervised classification by low density separation. In: Proceedings of the 10th international workshop on artificial intelligence and statistics, vol 1, pp 57–64
Niyogi X (2004) Locality preserving projections. In: Neural information processing systems, vol 16, p 153. MIT
Liu G, Lin Z, Yan S, Sun J, Yu Y, Ma Y (2013) Robust recovery of subspace structures by low-rank representation. Pattern Anal Mach Intell IEEE Trans 35(1):171–184
Hoyer PO (2004) Non-negative matrix factorization with sparseness constraints. J Mach Learn Res 5:1457–1469
Fujiwara Y, Irie G (2014) Efficient label propagation. In: Proceedings of the 31st international conference on machine learning (ICML-14), pp 784–792
Lee DD, Seung HS (1999) Learning the parts of objects by non-negative matrix factorization. Nature 401(6755):788–791
Lee DD, Seung HS (2001) Algorithms for non-negative matrix factorization. In: Advances in neural information processing systems, pp 556–562
Cai D, He X, Wu X, Han J (2008) Non-negative matrix factorization on manifold. In: Data mining, 2008. ICDM’08, 8th IEEE international conference on (pp 63–72), IEEE
Cai D, He X, Han J, Huang TS (2011) Graph regularized nonnegative matrix factorization for data representation. Pattern Anal Mach Intell IEEE Trans 33(8):1548–1560
Zadeh LA (1968) Probability measures of fuzzy events. J Math Anal Appl 23(2):421–427
Wang XZ, Xing HJ, Li Y, Hua Q, Dong CR, Pedrycz W (2015) A study on relationship between generalization abilities and fuzziness of base classifiers in ensemble learning. IEEE Trans Fuzzy Syst 23(5):1638–1654
Wang XZ, Aamir Raza Ashfaq R, Fu AM (2015) Fuzziness based sample categorization for classifier performance improvement. J Intell Fuzzy Syst 29(3):1185–1196
Wang XZ, Dong LC, Yan JH (2012) Maximum ambiguity-based sample selection in fuzzy decision tree induction. Knowl Data Eng IEEE Trans 24(8):1491–1505
Shannon CE (1948) A mathematical theory of communication. Bell Syst Tech J 27:379–423, 623–656
Hartley RVL (1949) Transmission of information. Bell Syst Tech J 7:535–563
Acknowledgments
This paper is supported by College of Information System and Management, National University of Defense Technology and subsidized by National Natural Science Foundation of China Grant No. 61170158.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Ren, W., Li, G. Graph based semi-supervised learning via label fitting. Int. J. Mach. Learn. & Cyber. 8, 877–889 (2017). https://doi.org/10.1007/s13042-015-0458-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13042-015-0458-y