Semi-supervised multi-view learning machine is developed to process the corresponding semi-supervised multi-view data sets which consist of labeled and unlabeled instances. But in real-world applications, for a multi-view data set, only few instances are labeled with the limitation of manpower and cost. As a result, few prior knowledge which is necessary for the designing of a learning machine is provided. Moreover, in practice, different views and features play diverse discriminant roles while traditional learning machines treat these roles equally and assign the same weight just for convenience. In order to solve these problems, we introduce Universum learning to obtain more prior knowledge and assign different weights for views and features to reflect their diverse discriminant roles. The proposed learning machine is named as weight-and-Universum-based semi-supervised multi-view learning machine (WUSM). In WUSM, we first obtain weights of views and features. Then, we construct Universum set to obtain more prior knowledge on the basis of these weights. Different from traditional construction ways, the used construction way makes full use of the information of all labeled and unlabeled instances rather than only a pair of positive and negative training instances. Finally, we design the machine with the usage of the Universum set along with original data set. Our contributions are given as follows. (1) With the usage of all (labeled, unlabeled) instances of the data set, the Universum set provides more useful prior knowledge. (2) WUSM considers the diversities of views and features. (3) WUSM advances the development of semi-supervised multi-view learning machines. Experiments on bipartite ranking, feature selection, dimensionality reduction, classification, clustering, etc. validate the advantages of WUSM and draw a conclusion that with the introduction of Universum learning, view weights, and feature weights, the performance of a semi-supervised multi-view learning machine is boosted.
This is a preview of subscription content, log in to check access.
This work is supported by Project funded by China Postdoctoral Science Foundation under Grant Number 2019M651576, National Natural Science Foundation of China (Grant Nos. 61602296, 61673301), Natural Science Foundation of Shanghai (Grant Nos. 16ZR1414500), National Key R&D Program of China (Grant No. 213), Major Project of Ministry of Public Security (Grant No. 20170004). Furthermore, this work is also sponsored by ‘Chenguang Program’ supported by Shanghai Education Development Foundation and Shanghai Municipal Education Commission under Grant Number 18CG54. The authors would like to thank their supports.
Compliance with ethical standards
Conflict of interest
The authors declare that they have no conflict of interest.
This article does not contain any studies with human participants or animals performed by any of the authors.
Informed consent was obtained from all individual participants included in the study.
Amini MR, Usunier N, Goutte C (2009) Learning from multiple partially observed views-an application to multilingual text categorization. In: Neural information processing systems (NIPS), pp 28–36Google Scholar
Deng MQ, Wang C, Chen QF (2016) Human gait recognition based on deterministic learning through multiple views fusion. Pattern Recognit Lett 78(C):56–63CrossRefGoogle Scholar
Dhar S (2014) Analysis and extensions of Universum learning. Dissertations and Theses, Grad works, University of Minnesota, ProQuest Dissertations PublishingGoogle Scholar
Du YT, Li Q, Cai ZM, Guan XH (2013) Multi-view semi-supervised web image classification via co-graph. Neurocomputing 122:430–440CrossRefGoogle Scholar
Epshteyn A, DeJong G (2006) Generative prior knowledge for discriminative classification. AI Access Foundation 27(27):25–53MathSciNetzbMATHGoogle Scholar
Han C, Chen J, Wu QY, Mu S, Min HQ (2015) Sparse Markov chain-based semi-supervised multi-instance multi-label method for protein function prediction. J Bioinf Comput Biol 13(5):1543001CrossRefGoogle Scholar
Hou CP, Zhang CS, Wu Y, Nie FP (2010) Multiple view semi-supervised dimensionality reduction. Pattern Recognit 43(3):720–730zbMATHCrossRefGoogle Scholar
Nie F, Xu D, Li X, Xiang S (2011) Semi-supervised dimensionality reduction and classification through virtual label regression. IEEE Trans Syst Man Cybern B Cybern 41(3):675–685CrossRefGoogle Scholar
Peng B, Qian G, Ma YQ (2008) View-invariant pose recognition using multilinear analysis and the Universum. Adv Vis Comput 5359:581–591CrossRefGoogle Scholar
Schölkopf B, Shawe-Taylor J, Smola AJ, Gmd BS, Smola EJ, Williamson RC (1999) Generalization bounds via eigenvalues of the gram matrix. Technical report 99-035, NeuroColtGoogle Scholar
Schölkopf B, Simard P, Smola AJ, Vapnik V (1997) Prior knowledge in support vector kernels. In: Neural information processing systems (NIPS), pp 640–646Google Scholar
Seliya N, Khoshgoftaar TM (2007) Software quality analysis of unlabeled program modules with semisupervised clustering. IEEE Trans Syst Man Cybern A Syst Hum 37(2):201–211CrossRefGoogle Scholar
Sheikhpour R, Sarram MA, Gharaghani S, Chahooki MAZ (2017) A survey on semi-supervised feature selection methods. Pattern Recognit 64:141–158CrossRefGoogle Scholar
Shen C, Wang P, Shen F, Wang H (2012) UBoost: boosting with the Universum. IEEE Trans Pattern Anal Mach Intell 34(4):825–832CrossRefGoogle Scholar
Shi CJ, Ruan QQ, An GY, Ge C (2015) Semi-supervised sparse feature selection based on multi-view Laplacian regularization. Image Vis Comput 41:1–10CrossRefGoogle Scholar
Sinz F, Chapelle O, Agarwal A, Schölkopf B (2008) An analysis of inference with the Universum. In: Advances in neural information processing systems (NIPS2008), pp 1369–1376Google Scholar
Sun SL, Zhang QJ (2011) Multiple-view multiple-learner semi-supervised learning. Neural Process Lett 34:229–240CrossRefGoogle Scholar
Tao H, Hou CP, Nie FP, Zhu JB, Yi DY (2017) Scalable multi-view semi-supervised classification via adaptive regression. IEEE Trans Image Process 26(9):4283–4296MathSciNetzbMATHCrossRefGoogle Scholar
Tzortzis G, Likas A (2012) Kernel-based weighted multi-view clustering. In: 2012 IEEE 12th international conference on data mining, pp 675–684Google Scholar
Usunier N, Amini MR, Goutte C (2011) Multiview semi-supervised learning for ranking multilingual documents. In: Gunopulos D, Hofmann T, Malerba D, Vazirgiannis M (eds) Machine learning and knowledge discovery in databases. ECML PKDD 2011. Lecture notes in computer science, vol 6913. Springer, Berlin, Heidelberg, pp 443–458CrossRefGoogle Scholar
Vapnik V, Chervonenkis A (1971) On the uniform convergence of relative frequencies of events to their probabilities. Theory Probab Appl 16(2):264–280zbMATHCrossRefGoogle Scholar
Vapnik V, Kotz S (1982) Estimation of dependences based on empirical data. Springer, BerlinzbMATHGoogle Scholar
Wang F (2011) Semisupervised metric learning by maximizing constraint margin. IEEE Trans Syst Man Cybern B Cybern 41(4):931–939CrossRefGoogle Scholar
Wang Z, Xu J, Chen SC, Gao DQ (2012) Regularized multi-view machine based on response surface technique. Neurocomputing 97:201–213CrossRefGoogle Scholar
Wang HY, Wang X, Zheng J, Deller JR, Peng HY, Zhu LQ, Chen WG, Li XL, Liu RJ, Bao HJ (2014) Video object matching across multiple non-overlapping camera views based on multi-feature fusion and incremental learning. Pattern Recognit 47(12):3841–3851CrossRefGoogle Scholar
Wang Z, Zhu YJ, Liu WW, Chen ZH, Gao DQ (2014) Multi-view learning with Universum. Knowl Based Syst 70:376–391CrossRefGoogle Scholar
Weston J, Collobert R, Sinz F, Bottou L, Vapnik V (2006) Inference with the Universum. In: The 23rd international conference on machine learning, pp 1009–1016Google Scholar
Wu F, Jing XY, You XG, Yue D, Hu RM, Yang JY (2016) Multi-view low-rank dictionary learning for image classification. Pattern Recognit 50:143–154zbMATHCrossRefGoogle Scholar
Xu XX, Li W, Xu D, Tsang IW (2016) Co-labeling for multi-view weakly labeled learning. IEEE Trans Pattern Anal Mach Intell 38(6):1113–1125CrossRefGoogle Scholar
Xu YM, Wang CD, Lai JH (2016) Weighted multi-view clustering with feature selection. Pattern Recognit 53:25–35CrossRefGoogle Scholar
Xu YT, Chen M, Yang ZJ, Li GH (2016) v-twin support vector machine with Universum data for classification. Appl Intell 44(4):956–968CrossRefGoogle Scholar
Yang ZK, Liu Z, Liu SY, Min L, Meng WT (2014) Adaptive multi-view selection for semi-supervised emotion recognition of posts in online student community. Neurocomputing 144:138–150CrossRefGoogle Scholar
Yu H, Wang XC, Wang GY (2017) A semi-supervised three-way clustering framework for multi-view data. In: International joint conference on rough sets, pp 313–325CrossRefGoogle Scholar