Abstract
Least square regression (LSR) and principal component analysis (PCA) are two representative dimensionality reduction algorithms in the fields of machine learning. In this paper, we propose a novel method to jointly learn projections from the subspaces derived from the modified LSR and PCA. To implement simultaneous feature learning, we design a novel joint regression learning model by imposing two orthogonal constraints. Therefore, the learned projections can preserve the minimum reconstruction error and the discriminative information in the low-dimensional subspaces. Besides, since the traditional LSR and PCA are sensitive to the outliers, we utilize the robust L2,1-norm as the metric of loss function to improve the model’s robustness. A simple iterative algorithm is proposed to solve the proposed framework. Experiments on face databases show the promising performance of our method.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abdi, H., Williams, L.J.: Principal component analysis. Wiley Interdiscip. Rev. Comput. Stat. 2, 433–459 (2010)
Jolliffe, I.T.: Principal Component Analysis, vol. 87, pp. 41–64. Springer, Berlin (2010). https://doi.org/10.1007/b98835
Zhou, Z., Jin, Z.: Double nuclear norm-based robust principal component analysis for image disocclusion and object detection. Neurocomputing 205, 481–489 (2016)
Lai, Z., Xu, Y., Chen, Q., Yang, J., Zhang, D.: Multilinear sparse principal component analysis. IEEE Trans. Neural Netw. Learn. Syst. 25, 1942–1950 (2014)
Zhang, F., Yang, J., Qian, J., Xu, Y.: Nuclear norm-based 2-DPCA for extracting features from images. IEEE Trans. Neural Netw. Learn. Syst. 26, 2247–2260 (2015)
Brooks, J.P., Boone, E.L.: A pure L1-norm principal component analysis. Comput. Stat. Data Anal. 61, 83 (2013)
Nie, F., Huang, H., Ding, C.H.Q., Luo, D., Wang, H.: Robust principal component analysis with non-greedy l1-norm maximization. Presented at the IJCAI Proceedings-International Joint Conference on Artificial Intelligence (2011)
Kwak, N.: Principal component analysis based on L1-norm maximization. IEEE Trans. Pattern Anal. Mach. Intell. 30, 1672–1680 (2008)
Ding, C., Zhou, D., He, X., Zha, H.: R1-PCA: rotational invariant L1-norm principal component analysis for robust subspace factorization. In: Proceedings of the 23rd International Conference on Machine Learning, pp. 281–288 (2006)
Shi, X., Nie, F., Lai, Z., Guo, Z.: Robust principal component analysis via optimal mean by joint ℓ2,1 and Schatten p-norms minimization. Neurocomputing 283, 205–213 (2018)
Nie, F., Yuan, J., Huang, H.: Optimal mean robust principal component analysis. In: Proceedings of the 31st International Conference on Machine Learning, pp. 1062–1070 (2014)
Yi, S., Lai, Z., He, Z., Cheung, Y.M., Liu, Y.: Joint sparse principal component analysis. Pattern Recognit. 61, 524–536 (2016)
Nie, F., Huang, H., Cai, X., Ding, C.: Efficient and robust feature selection via joint L2,1-norms minimization. In: Advances in Neural Information Processing Systems, pp. 1813–1821 (2010)
Cai, X., Ding, C., Nie, F., Huang, H.: On the equivalent of low-rank linear regressions and linear discriminant analysis based regressions. In: Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1124–1132. ACM (2013)
Zou, H., Hastie, T., Tibshirani, R., Url, S.: Sparse principal component analysis. J. Comput. Graph. Stat. 15, 265–286 (2006)
Yang, J., Yang, J.Y.: Why can LDA be performed in PCA transformed space? Pattern Recognit. 36, 563–566 (2003)
Zheng, W.S., Lai, J.H., Li, S.Z.: 1D-LDA vs. 2D-LDA: when is vector-based linear discriminant analysis better than matrix-based? Pattern Recognit. 41, 2156–2172 (2008)
Fang, X., Xu, Y., Li, X., Lai, Z., Wong, W.K., Fang, B.: Regularized label relaxation linear regression. IEEE Trans. Neural Netw. Learn. Syst. PP, 1–13 (2017)
Martinez, A.M.: The AR face database. CVC Technical Report 24 (1998)
Jonathon Phillips, P., Moon, H., Rizvi, S.A., Rauss, P.J.: The FERET evaluation methodology for face-recognition algorithms. IEEE Trans. Pattern Anal. Mach. Intell. 22, 1090–1104 (2000)
Sim, T., Baker, S., Bsat, M.: The CMU pose, illumination, and expression database. IEEE Trans. Pattern Anal. Mach. Intell. 25, 1615–1618 (2003)
Acknowledgments
This work was supported in part by the Natural Science Foundation of China (Grant 61573248, Grant 61773328, Grant 61773328 and Grant 61703283), Research Grant of The Hong Kong Polytechnic University (Project Code:G-UA2B), China Postdoctoral Science Foundation (Project 2016M590812 and Project 2017T100645), the Guangdong Natural Science Foundation (Project 2017A030313367 and Project 2017A030310067), the Guangdong medical scientific and technological research funding under grant A2017251, Shenzhen Municipal Science and Technology Innovation Council (No. JCYJ20170302153434048, No. JCYJ20160429182058044 and No. JCYJ20160429182058044).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Xu, X., Lai, Z., Chen, Y., Kong, H. (2018). Robust Discriminative Principal Component Analysis. In: Zhou, J., et al. Biometric Recognition. CCBR 2018. Lecture Notes in Computer Science(), vol 10996. Springer, Cham. https://doi.org/10.1007/978-3-319-97909-0_25
Download citation
DOI: https://doi.org/10.1007/978-3-319-97909-0_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-97908-3
Online ISBN: 978-3-319-97909-0
eBook Packages: Computer ScienceComputer Science (R0)