Abstract
As a class of semi-supervised learning methods, manifold regularization learning has recently attracted a lot of attention, due to their great success in exploiting the underlying geometric structures among data. This paper presents a novel semi-supervised approach by combining manifold regularization learning with the idea of multiple kernels, named after ensemble multiple-kernel manifold regularization learning. In our approach, multiple kernels we introduced are not only used to add the flexibility and diversity of the candidate space for the learning problem, but also act as a similarity measure to search for an optimal graph Laplacian in some sense. In other words, the proposed method allows us to learn an ’ideal’ kernel and an optimal graph Laplacian simultaneously, which is of significant difference from existing methods. The associated optimization problem is solved efficiently by an alternating iteration procedure. We implement experiments over four real world data sets to demonstrate the benefits of the proposed method.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res 7:2399–2434
Bucak S, Jin R, Jain A (2014) Multiple kernel learning for visual object recognition: a review. IEEE Trans Pattern Anal Mach Intell 39(7):1354–1369
Chen L, Tsan IW, Xu D (2012) Laplacian embedded regression for scalable manifold regularization. IEEE Trans Neural Netw Learn Syst 33(6):902–915
Chapelle O, Weston J, Schokopf B (2003) Cluster kernels for semi-supervised learning. In: Proceedings of the advances in neural information processing system 15
Chen S, Chris H, Ding Q, Luo B (2015) Similarity learning of manifold data. IEEE Trans Cybern 45(9):1744–1756
Dornaika F, El Traboulsi Y (2016) Learning flexible graph-based semi-supervised embedding. IEEE Trans Neural Netw Learn Syst 46(1):206–218
Fu D, Yang T (2013) Manifold regularization multiple kernel learning machine for classification. In: Proceedings of the 2013 international conference on machine learning and cybernetics
Geng B, Tao DC, Xu C, Yang LJ, Hua S (2012) Ensemble manifold regularization. IEEE Trans Pattern Anal Mach Intell 34(6):1227–1233
Gnen M, Alpaydin E (2011) Multiple kernel learning algorithms. J Mach Learn Res 12:2211–2268
Han Y, Yang K, Ma YL, Liu GZ (2014) Localized multiple kernel learning via sample-wise alternating optimization. IEEE Trans Cybern 44(1):137–138
Lanckriet G, Cristianini N, Bartlett P, Ghaoui LE, Jordan MI (2004) Learning the kernel matrix with semidefinite programming. J Mach Learn Res 5:27–72
Luo Y, Tao DC, Geng B, Xu C, Maybank SJ (2013) Manifold regularized multitask learning for semi-supervised multilabel image classification. IEEE Trans Image Process 22(2):523–536
Liu W, Wang J, Chang SF (2012) Robust and scalable graph-based semisupervised learning. Proc IEEE 100(9):2624–2638
Nie F, Xu D, Tsang IW, Zhang CS (2010) Flexible manifold embedding a framework for semi-supervised and unsupervised dimension reduction. IEEE Trans Image Process 19(7):1921–1932
Rakotomamonjy A, Bach F, Canu S, Grandvalet Y (2008) SimpleMKL. J Mach Learn Res 9:2491–2521
Scholkopf B, Smola AJ (2002) Learning with kernel. MIT press, Cambrige
Sindhwani V, Niyogi P, Belkin M (2005) Linear manifold regularization for large scale semi-supervised learning. In: Proceedings of the 22nd international conference on machine learning
Wang G, Wang F, Chen T, Yeung D, Lochovsky F (2011) Solution path for manifold regularized semisupervised classification. IEEE Trans Syst Man Cybern B Cybern 42(2):308–319
Xu J, Paiva ARC, Park Il (Memming), Principe JC (2008) A reproducing kernel Hilbert space framework for information-theoretic learning. IEEE Trans Signal Process 56(12):5891–5902
Xu Y, Chen DR, Li HX, Liu L (2013) Least square regularized regression in sum space. IEEE Trans Neural Netw Learn Syst 24(4):635–646
Xu Z, Jin R, Zhu SH, Lyu M, King I (2010) Smooth optimization for effective multiple kernel learning. In: Proceedings of the 24nd AAAI conference on artificial intelligence
Yu J, Rui Y, TaoJ DC, Yu Y Rui, Tao DC (2014) High-order distance based multiview stochastic learning in image classification. IEEE Trans Cybern 44(12):2431–2442
Yu J, Rui Y, Tao DC (2014) Click prediction for web image reranking using multimodal sparse coding. IEEE Trans Image Process 23(5):2019–2032
Yu J, Rui Y, Chen B (2014) Exploiting click constraints and multiview features for image reranking. IEEE Trans Multimed 16(1):159–168
Acknowledgments
We thank the editor and two anonymous reviewers for their helpful comments. This work is supported in part by the Guangdong Provincial Science and Technology Major Projects of China under Grant 67000-42020009. The coauthor Lv’s research is supported partially by National Natural Science Foundation of China (Grant No.11301421), and Fundamental Research Funds for the Central Universities of China (Grants No. JBK120509 and JBK140507), as well as KLAS-130026507.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Niu, G., Ma, Z. & Lv, S. Ensemble Multiple-Kernel Based Manifold Regularization. Neural Process Lett 45, 539–552 (2017). https://doi.org/10.1007/s11063-016-9543-9
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11063-016-9543-9