Abstract
In discriminant analysis, trace ratio is an important criterion for minimizing the between-class similarity and maximizing the within-class similarity, simultaneously. In brief, we address the trace ratio problem associated with many semi-supervised discriminant analysis algorithms as they use the normal Euclidean distances between training data samples. Based on this problem, we propose a new semi-supervised orthogonal discriminant analysis technique with relative distance constraints called SSODARD. Different from the existing semi-supervised dimensionality reduction algorithms, our algorithm is more consistent in propagating the label information from the labeled data to the unlabeled data because of the use of relative distance function instead of normal Euclidean distance function. For finding this appropriate relative distance function, we use pairwise constraints generated from labeled data and satisfy them using Bregman projection. Since the projection is not orthogonal, we require an appropriate subset of constraints. In order to select such a subset of constraints, we further develop a framework called MO-SSODARD, which uses evolutionary algorithm while optimizing various validity indices simultaneously. The experimental results on various datasets show that our proposed approaches are superior than the state-of-the-art discriminant algorithms with respect to various validity indices.
Similar content being viewed by others
References
Baldi P, Long AD (2001) A Bayesian framework for the analysis of microarray expression data: regularized t-test and statistical inferences of gene changes. Bioinformatics 17(6):509–519
Belhumeur PN, Hespanha JP, Kriegman DJ (1997) Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720
Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res 7(Nov):2399–2434
Blum A, Mitchell T (1998) Combining labeled and unlabeled data with co-training. In: Proceedings of the eleventh annual conference on computational learning theory, pp 92–100. ACM
Cai D, He X, Han J (2007) Semi-supervised discriminant analysis. In: IEEE 11th international conference on computer vision, 2007. ICCV 2007, pp 1–7. IEEE
Chen LF, Liao HYM, Ko MT, Lin JC, Yu GJ (2000) A new IDA-based face recognition system which can solve the small sample size problem. Pattern Recognit 33(10):1713–1726
Chung MK (2012) Gaussian kernel smoothing. Lecture notes, pp 1–10
Ciro GC, Dugardin F, Yalaoui F, Kelly R (2016) A nsga-ii and nsga-iii comparison for solving an open shop scheduling problem with resource constraints. IFAC-PapersOnLine 49(12):1272–1277
Corne DW, Knowles JD, Oates MJ (2000) The pareto envelope-based selection algorithm for multiobjective optimization. In: International conference on parallel problem solving from nature, pp 839–848. Springer
Das I, Dennis JE (1998) Normal-boundary intersection: a new method for generating the pareto surface in nonlinear multicriteria optimization problems. SIAM J Optim 8(3):631–657
Deb K (2014) Multi-objective optimization. In: Search methodologies, pp 403–449. Springer
Deb K, Jain H (2014) An evolutionary many-objective optimization algorithm using reference-point-based nondominated sorting approach, part i: solving problems with box constraints. IEEE Trans Evol Comput 18(4):577–601
Deb K, Pratap A, Agarwal S, Meyarivan TAMT (2002) A fast and elitist multiobjective genetic algorithm: Nsga-ii. IEEE Trans Evol Comput 6(2):182–197
Duchene J, Leclercq S (1988) An optimal transformation for discriminant and principal component analysis. IEEE Trans Pattern Anal Mach Intell 10(6):978–983
Fisher Ronald A (1936) The use of multiple measurements in taxonomic problems. Annals Hum Genet 7(2):179–188
Hand DJ (1982) Kernel discriminant analysis. JOHN WILEY & SONS, INC., ONE WILEY DR., SOMERSET, N. J. 08873, 1982, 264
He X, Niyogi P (2004) Locality preserving projections. In: Advances in neural information processing systems, pp 153–160
Huang S, Elgammal A, Huangfu L, Yang D, Zhang X (2014) Globality-locality preserving projections for biometric data dimensionality reduction. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 15–20
Huang Yi, Xu Dong, Nie Feiping (2012) Semi-supervised dimension reduction using trace ratio criterion. IEEE Trans Neural Netw Learn Syst 23(3):519–526
Izenman AJ (2013) Linear discriminant analysis. In: Modern multivariate statistical techniques, pp 237–280. Springer
Jain H, Deb K (2014) An evolutionary many-objective optimization algorithm using reference-point based nondominated sorting approach, part ii: handling constraints and extending to an adaptive approach. IEEE Trans Evol Comput 18(4):602–622
Jin Z, Yang JY, Hu ZS, Lou Z (2001) Face recognition based on the uncorrelated discriminant transformation. Pattern Recognit 34(7):1405–1416
Joachims T (1999) Transductive inference for text classification using support vector machines. In: ICML, vol 99, pp 200–209
Krzanowski WJ, Jonathan P, McCarthy WV, Thomas MR (1995) Discriminant analysis with singular covariance matrices: methods and applications to spectroscopic data. Appl Stat 44(1):101–115
Kulis B, Sustik MA, Dhillon IS (2009) Low-rank kernel learning with bregman matrix divergences. J Mach Learn Res 10(Feb):341–376
Li Ming, Yuan Baozong (2005) 2d-lda: a statistical linear discriminant analysis for image matrix. Pattern Recognit Lett 26(5):527–532
Lu J, Zhou X, Tan YP, Shang Y, Zhou Jie (2012) Cost-sensitive semi-supervised discriminant analysis for face recognition. IEEE Trans Inf Forensics Secur 7(3):944–953
Mao-Guo G, Li-Cheng J, Dong-Dong Y, Wen-Ping M (2009) Evolutionary multi-objective optimization algorithms
Mishra S, Saha S, Mondal S (2016) Divide and conquer based non-dominated sorting for parallel environment. In: 2016 IEEE congress on evolutionary computation (CEC), pp 4297–4304. IEEE
Murata T, Ishibuchi H (1995) Moga: multi-objective genetic algorithms. In: IEEE international conference on evolutionary computation, 1995. vol 1, p 289. IEEE
Nie F, Xiang S, Jia Y, Zhang C (2009) Semi-supervised orthogonal discriminant analysis via label propagation. Pattern Recognit 42(11):2615–2627
Nie F, Xu D, Tsang IWH, Zhang C (2010) Flexible manifold embedding: a framework for semi-supervised and unsupervised dimension reduction. IEEE Trans Image Process 19(7):1921–1932
Rao CR (1948) The utilization of multiple measurements in problems of biological classification. J R Stat Soc Ser B (Methodological) 10(2):159–203
Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326
Seada H, Deb K (2015) U-nsga-iii: a unified evolutionary optimization procedure for single, multiple, and many objectives: proof-of-principle results. In: International conference on evolutionary multi-criterion optimization, pp 34–49. Springer
Si S, Tao D, Geng B (2010) Bregman divergence-based regularization for transfer subspace learning. IEEE Trans Knowl Data Eng 22(7):929–942
Srinivas N, Deb K (1994) Multiobjective optimization using nondominated sorting in genetic algorithms. Evolu Comput 2(3):221–248
Sugiyama M, Idé T, Nakajima S, Sese J (2010) Semi-supervised local fisher discriminant analysis for dimensionality reduction. Mach Learn 78(1–2):35
Wang H, Yan S, Xu D, Tang X, Huang T (2007) Trace ratio vs. ratio trace for dimensionality reduction. In: IEEE Conference on computer vision and pattern recognition, 2007. CVPR’07. pp 1–8. IEEE
Wang Sheng, Lu Jianfeng, Gu Xingjian, Du Haishun, Yang Jingyu (2016) Semi-supervised linear discriminant analysis for dimension reduction and classification. Pattern Recognit 57:179–189
Wang Xiaogang, Tang Xiaoou (2004) Dual-space linear discriminant analysis for face recognition. In: Proceedings of the 2004 IEEE computer society conference on computer vision and pattern recognition, 2004. CVPR 2004. vol 2, pp II–II. IEEE
Wold S, Esbensen K, Geladi P (1987) Principal component analysis. Chemom Intell Lab Syst 2(1–3):37–52
Xu Y, Pan SJ, Xiong H, Wu Q, Luo Ronghua, Min Huaqing, Song Hengjie (2017) A unified framework for metric transfer learning. IEEE Trans Knowl Data Eng 29(6):1158–1171
Ye J, Li Q (2005) A two-stage linear discriminant analysis via qr-decomposition. IEEE Trans Pattern Anal Mach Intell 27(6):929–941
Yu W, Teng X, Liu C (2006) Face recognition using discriminant locality preserving projections. Image Vision Comput 24(3):239–248
Zhang Y, Yeung DY (2008a) Semi-supervised discriminant analysis using robust path-based similarity. In: IEEE conference on computer vision and pattern recognition, 2008. CVPR 2008. pp 1–8. IEEE
Zhang Y, Yeung DY (2008b) Semi-supervised discriminant analysis via cccp. In: Joint European conference on machine learning and knowledge discovery in databases, pp 644–659. Springer
Zhou D, Bousquet O, Lal TN, Weston J, Schölkopf B (2004) Learning with local and global consistency. In: Advances in neural information processing systems, pp 321–328
Zitzler E, Laumanns M, Thiele L (2001) Spea2: improving the strength pareto evolutionary algorithm. TIK-report, 103
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Human participants
This study does not contain any studies with human participants or animals performed by any of the authors.
Additional information
Communicated by V. Loia.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Sanodiya, R.K., Saha, S. & Mathew, J. Semi-supervised orthogonal discriminant analysis with relative distance : integration with a MOO approach. Soft Comput 24, 1599–1618 (2020). https://doi.org/10.1007/s00500-019-03990-9
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-019-03990-9