Abstract
Multiple Empirical Kernel Learning (MEKL) explicitly maps the samples to empirical feature spaces, in which the feature vectors of the mapped samples are explicitly presented. Thus with the explicit representation of the samples, almost all algorithms can be kernelized directly, which is much easier in processing and analyzing the structure of the empirical feature spaces. However, in conventional MEKL, samples are treated to belong to one exact class, and contribute the same importance to the decision surface. However, in many real-world applications, input samples may not be fully assigned to one class. MEKL suffers the instinct drawbacks in representing these samples. To overcome this problem, we assign a fuzzy membership to each mapped sample in each feature space and reformulate MEKL to the Fuzzy MEKL (FMEKL) in which different samples in different feature spaces can make different contributions to the learning of decision surface. Moreover, we propose a novel fuzzy membership evaluation approach named locality density-based fuzzy membership evaluation, which assign larger fuzzy membership to the samples with higher local density. Thus, FMEKL by adopting the locality density-based fuzzy membership evaluation is named as Locality Density-based Fuzzy Multiple Empirical Kernel Learning (LD-FMEKL). Experimental results on both balanced and imbalanced real-world datasets validate that LD-FMEKL outperforms the compared algorithms. The contributions of this work are: (i) reformulating traditional MEKL to fuzzy multiple empirical kernel learning; (ii) introducing an alternative locality density-based fuzzy membership evaluation approach; (iii) proposing the locality density-based FMEKL.
Similar content being viewed by others
Notes
The value can be computed in the website http://graphpad.com/quickcalcs/PValue1.cfm.
Yale Face database is publicly available at http://vision.ucsd.edu/datasets/yale_face_dataset_original/yalefaces.zip.
Letter digit dataset is publicly available at http://www.cam-orl.co.uk.
Coil-20 database is publicly available at http://www.cs.columbia.edu/CAVE/coil-20.html.
References
Alcalá-Fdez J, Fernandez A, Luengo J, Derrac J, Garcıa S, Sánchez L, Herrera F (2011) KEEL: A Software Tool to Assess Evolutionary Algorithms for Data Mining Problems. Journal of Multiple-Valued Logic and Soft Computing 17:2–3
Alcalá-Fdez J, Sanchez L, Garcia S, del Jesus MJ, Ventura S, Garrell JM, Otero J, Romero C, Bacardit J, Rivas VM (2009) KEEL: A Software Tool to Assess Evolutionary Algorithms for Data Mining Problems. Soft Computing 13(3):307–318
Althloothi S, Mahoor MH, Zhang X, Voyles RM (2014) Human Activity Recognition Using Multi-features and Multiple Kernel Learning. Pattern Recognition 47:1800–1812
Bach FR, Lanckriet GR, Jordan MI (2004) Multiple Kernel Learning, Conic Duality, and the SMO Algorithm. In: International Conference on Machine Learning, pp 6–13 ACM
Bache K, Lichman M (2013) UCI Machine Learning Repository, University of California, Irvine, School of Information and Computer Sciences, http://archive.ics.uci.edu/ml
Barandela R, Sánchez JS, Garcıa V, Rangel E (2003) Strategies for Learning in Class Imbalance Problems. Pattern Recognition 36(3):849–851
Berline N, Getzler E, Vergne M (2004) Heat Kernels and Dirac Operators. Springer-Verlag, Berlin, New York
Bucak SS, Jin R, Jain AK (2014) Multiple Kernel Learning for Visual Object Eecognition: A Review. IEEE Transactions on Pattern Analysis and Machine Intelligence 36(7):1354–1369
Chen D, He Q, Wang X (2010) FRSVMs: Fuzzy Rough Set based Support Vector Machines. Fuzzy Sets and Systems 161(4):596–607
Cortes C, Vapnik V (1995) Support Vector Machine. Machine Learning 20(3):273–297
Demšar J (2006) Statistical Comparisons of Classifiers over Multiple Data Sets. Journal of Machine Learning Research 7:1–30
Deng Z, Choi K, Jiang Y, Wang S (2014) Generalized Hidden-Mapping Ridge Regression, Knowledge-Leveraged Inductive Transfer Learning for Neural Networks, Fuzzy Systems and Kernel Methods. IEEE Transactions on Cybernetics 44(12):2585–2599
Galar M, Fernandez A, Barrenechea E, Bustince H, Herrera F (2012) A Review on Ensembles for the Class Imbalance Problem: Bagging-, Boosting-, and Hybrid-based Approaches. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 42(4):463–484
Gönen M, Alpaydın E (2013) Localized Algorithms for Multiple Kernel Learning. Pattern Recognition 46(3):795–807
He H, Garcia EA (2009) Learning from Imbalanced Data. IEEE Transactions on Knowledge and Data Engineering 21(9):1263–1284
Huang J, Ling CX (2005) Using AUC and Accuracy in Evaluating Learning Algorithms. IEEE Transactions on Knowledge and Data Engineering 17(3):299–310
Jian L, Xia Z, Liang X, Gao C (2011) Design of a Multiple Kernel Learning Algorithm for LS-SVM by Convex Programming. Neural Networks 24(5):476–483
Jiang X, Yi Z, Lv JC (2006) Fuzzy SVM with a new Fuzzy Membership Function. Neural Computing and Applications 15(3–4):268–276
Khalilia MA, Bezdek J, Popescu M, Keller JM (2014) Improvements to the Relational Fuzzy C-means Clustering Algorithm. Pattern Recognition 47(12):3920–3930
Lam HK (2011) Polynomial Fuzzy-Model-Based Control Systems: Stability Analysis Via Piecewise-Linear Membership Functions. IEEE Transactions on Fuzzy Systems 19(3):588–593
Lam HK, Narimani M (2010) Quadratic-Stability Analysis of Fuzzy-Model-Based Control Systems Using Staircase Membership Functions. IEEE Transactions on Fuzzy Systems 18(1):125–137
Leski J (2004) Kernel Ho-Kashyap Classifier with Generalization Control. International Journal of Applied Mathematics and Computer Science 14(1):53–62
Leski JM (2004) An \(\varepsilon \)-margin Nonlinear Classifier based on Fuzzy If-then Rules. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 34(1):68–76
Li K, Lu X (2011) A Double Margin based Fuzzy Support Vector Machine Algorithm. Journal of Computers 6(9):1962–1970
Li X, Yang G (2014) Fault Detection for TCS Fuzzy Systems With Unknown Membership Functions. IEEE Transactions on Fuzzy Systems 22(1):139–152
Liang Z, Xia S, Zhou Y, Zhang L (2013) Training Lp-norm Multiple Kernel Learning in the Primal. Neural Networks 46:172–182
Lin C, Wang S (2002) Fuzzy Support Vector Machines. IEEE Transactions on Neural Networks 13(2):464–471
Lin KP (2014) A Novel Evolutionary Kernel Intuitionistic Fuzzy \(C\)-means Clustering Algorithm. IEEE Transactions on Fuzzy Systems 22(5):1074–1087
Łkeski J (2003) Ho-Kashyap Classifier with Generalization Control. Pattern Recognition Letters 24(14):2281–2290
Lu J, Plataniotis KN, Venetsanopoulos AN (2003) Face Recognition Using Kernel Direct Discriminant Analysis Algorithms. IEEE Transactions on Neural Networks 14(1):117–126
Maldonado S, López J (2014) Imbalanced Data Classification Using Second-order Cone Programming Support Vector Machines. Pattern Recognition 47(5):2070–2079
Nazarpour A, Adibi P (2014) Two-stage Multiple Kernel Learning for Supervised Dimensionality Reduction. Pattern Recognition 48(5):1854–1862
Sonnenburg S, Rätsch G, Schäfer C (2006) A General and Efficient Multiple Kernel Learning Algorithm. Neural Information Processing Systems 18:1273–1280
Tsang ECC, Yeung DS, Chan PPK (2003) Fuzzy Support Vector Machines for Solving Two-class Problems. In: International Conference on Machine Learning and Cybernetics, volume 2, pp 1080–1083. IEEE, (2003)
Vapnik V (2000) The Nature of Statistical Learning Theory. Springer Science and Business Media,
Wang Y, Wang S, Lai KK (2005) A new Fuzzy Support Vector Machine to Evaluate Credit Risk. IEEE Transactions on Fuzzy Systems 13(6):820–831
Wang Z, Chen SC, Sun TK (2008) MultiK-MHKS: A Novel Multiple Kernel Learning Algorithm. IEEE Transactions on Pattern Analysis and Machine Intelligence 30(2):348–353
Wong T (2015) Performance Evaluation of Classification Algorithms by K-fold and Leave-one-out Cross Validation. Pattern Recognition 48(9):2839–2846
Wu Z, Zhang H, Liu J (2014) A Fuzzy Support Vector Machine Algorithm for Classification based on a Novel PIM Fuzzy Clustering Method. Neurocomputing 125:119–124
Xiong H, Swamy MNS, Ahmad MO (2005) Optimizing the Kernel in the Empirical Feature Space. IEEE Transactions on Neural Networks 16(2):460–474
Yang X, Zhang G, Lu J, Ma J (2011) A Kernel Fuzzy C-means Clustering-based Fuzzy Support Vector Machine Algorithm for Classification Problems with Outliers or Noises. IEEE Transactions on Fuzzy Systems 19(1):105–115
Ye J (2005) Characterization of a Family of Algorithms for Generalized Discriminant Analysis on Undersampled Problems. Journal of Machine Learning Research 6:483–502
Ye J, Li T, Xiong T, Janardan R (2004) Using Uncorrelated Discriminant Analysis for Tissue Classification With Gene Expression Data. IEEE Transactions on Computational Biology and Bioinformatics 1(4):181–190
Zhang H, Liu D (2006) Fuzzy Modeling and Fuzzy Control. Springer Science and Business Media, Boston, Birkhäuser
Zhang H, Quan Y (2001) Modeling, Identification, and Control of a Class of Nonlinear Systems. IEEE Transactions on Fuzzy Systems 9(2):349–354
Acknowledgements
This work is supported by Natural Science Foundations of China under Grant No. 61672227, Shuguang Program supported by Shanghai Education Development Foundation and Shanghai Municipal Education Commission, and Action Plan for Innovation on Science and Technology Projects of Shanghai under Grant No. 16511101000.
Author information
Authors and Affiliations
Corresponding authors
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Fan, Q., Wang, Z. & Gao, D. Locality Density-Based Fuzzy Multiple Empirical Kernel Learning. Neural Process Lett 49, 1485–1509 (2019). https://doi.org/10.1007/s11063-018-9881-x
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11063-018-9881-x