Skip to main content
SpringerLink
Log in

A general robust low–rank multinomial logistic regression for corrupted matrix data classification

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Multi-classification of corrupted matrix data is a significant problem in machine learning and pattern recognition. However, most of the existing methods can only handle the clean data or the corrupted data with the know statistical information of noises. Besides, they usually reshape the matrix data into a vector as the input, very likely to destroy the structure of the raw data, thereby reducing the model performance. In order to address the above issues, a general robust low–rank multinomial logistic regression is proposed for corrupted matrix data. The proposed approach has three outstanding merits as follows: (1) by using the multinomial logistic regression combined with three regularization terms corresponding to matrix structure, low–rank and sparsity, the clean data recovery and the classification are simultaneously fulfilled; (2) the proposed method can adapt to more general corrupted matrix data since it does not require strong statistical assumptions about noise, and (3) the theoretical analysis is provided to show the convergence of the proposed multi-block ADMM algorithm, and such a convergence can be rigidly guaranteed by introducing two auxiliary variables such that the coefficients of the equality constraints are orthogonal. Finally, extensive experimental results have demonstrated the effectiveness and robustness of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Algorithm 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Tomioka R, Aihara K, Müller KR (2006) Logistic regression for single trial EEG classification[J]. Adv Neural Inf Process Syst, vol 19

  2. Yan S, Xu D, Yang Q et al (2006) Multilinear discriminant analysis for face recognition[J]. IEEE Trans Image Process 16(1):212–220

    Article  MathSciNet  Google Scholar 

  3. Koo JJ, Evans AC, Gross WJ (2009) 3-D brain MRI tissue classification on FPGAs[J]. IEEE Trans Image Process 18(12):2735–2746

    Article  MathSciNet  MATH  Google Scholar 

  4. Shakhnarovich G, Darrell T, Indyk P (2008) Nearest-neighbor methods in learning and vision[J]. IEEE Trans Neural Netw 19(2):377

    Article  Google Scholar 

  5. Kwak C, Clayton-Matthews A (2002) Multinomial logistic regression[J]. Nurs Res 51(6):404–410

    Article  Google Scholar 

  6. Böhning D (1992) Multinomial logistic regression algorithm[J]. Ann Inst Stat Math 44(1):197–200

    Article  MATH  Google Scholar 

  7. Erb RJ (1993) Introduction to backpropagation neural network computation[J]. Pharm Res 10 (2):165–170

    Article  Google Scholar 

  8. Neelapu R, Devi GL, Rao KS (2018) Deep learning based conventional neural network architecture for medical image classification[J]. Traitement du Signal 35(2):169

    Article  Google Scholar 

  9. Song K, Nie F, Han J et al (2017) Parameter free large margin nearest neighbor for distance metric learning. In: Proceedings of the AAAI conference on artificial intelligence, vol 31

  10. Cai D, He X, Hu Y et al (2007) Learning a spatially smooth subspace for face recognition. In: 2007 IEEE conference on computer vision and pattern recognition. IEEE, pp 1–7

  11. Gabriel KR (1998) Generalised bilinear regression[J]. Biometrika 85(3):689–700

    Article  MathSciNet  MATH  Google Scholar 

  12. Hou C, Nie F, Yi D et al (2012) Efficient image classification via multiple rank regression[J]. IEEE Trans Image Process 22(1):340–352

    MathSciNet  MATH  Google Scholar 

  13. Hou C, Jiao Y, Nie F et al (2017) 2D feature selection by sparse matrix regression[J]. IEEE Trans Image Process 26(9):4255–4268

    Article  MathSciNet  MATH  Google Scholar 

  14. Yuan H, Li J, Lai LL et al (2020) Low–rank matrix regression for image feature extraction and feature selection[J]. Inf Sci 522:214–226

    Article  MathSciNet  Google Scholar 

  15. Fan Y, Qin G, Zhu ZY (2014) Robust variable selection in linear mixed models[J]. Commun Statistics-Theory Methods 43(21):4566–4581

    Article  MathSciNet  MATH  Google Scholar 

  16. Fan Y, Qin G, Zhu Z (2012) Variable selection in robust regression models for longitudinal data[J]. Multivar Anal 109:156–167

    Article  MathSciNet  MATH  Google Scholar 

  17. Copas JB (1988) Binary regression models for contaminated data[J]. J Roy Stat Soc: Ser B (Methodol) 50(2):225–253

    MathSciNet  Google Scholar 

  18. Xu G, Hu B G, Principe JC (2016) Robust bounded logistic regression in the class imbalance problem. In: 2016 international joint conference on neural networks (IJCNN). IEEE, pp 1434–1441

  19. Gillard JW (2006) An historical overview of linear regression with errors in both variables[J]. Cardiff University School of Mathematics Technical Report

  20. Kummell CH (1879) Reduction of observation equations which contain more than one observed quantity[J]. The Analyst 6(4):97–105

    Article  MATH  Google Scholar 

  21. Wald A (1940) The fitting of straight lines if both variables are subject to error[J]. Ann Math Stat 11(3):284–300

    Article  MathSciNet  MATH  Google Scholar 

  22. Gillard JW, Iles TC (2005) Method of moments estimation in linear regression with errors in both variables[J]. Cardiff University School of Mathematics Technical Paper:3–43

  23. Lindley DV (1947) Regression lines and the linear functional relationship[J]. Suppl J R Stat Soc 9(2):218–244

    Article  MathSciNet  MATH  Google Scholar 

  24. Yin M, Zeng D, Gao J et al (2018) Robust multinomial logistic regression based on RPCA[J]. IEEE J Sel Top Signal Process 12(6):1144–1154

    Article  Google Scholar 

  25. Chen L, Sun D, Toh KC (2017) A note on the convergence of ADMM for linearly constrained convex optimization problems[J]. Comput Optim Appl 66(2):327–343

    Article  MathSciNet  MATH  Google Scholar 

  26. Liu Q, Shen X, Gu Y (2019) Linearized ADMM for nonconvex nonsmooth optimization with convergence analysis[J]. IEEE Access 7:76131–76144

    Article  Google Scholar 

  27. Candès EJ, Li X, Ma Y et al (2011) Robust principal component analysis[J]. Journal of the ACM (JACM) 58(3):1–37

    Article  MathSciNet  MATH  Google Scholar 

  28. Wright J, Ganesh A, Rao S et al (2009) Robust principal component analysis: exact recovery of corrupted low–rank matrices via convex optimization[J]. Adv Neural Inf Process Syst, vol 22

  29. Cai JF, Candès EJ, Shen Z (2010) A singular value thresholding algorithm for matrix completion[J]. SIAM J Optim 20(4):1956–1982

    Article  MathSciNet  MATH  Google Scholar 

  30. Boyd S, Parikh N, Chu E et al (2011) Distributed optimization and statistical learning via the alternating direction method of multipliers[J]. Foundations and Trends® in Machine Learning 3(1):1–122

    Article  MATH  Google Scholar 

  31. Chen C, He B, Ye Y et al (2016) The direct extension of ADMM for multi-block convex minimization problems is not necessarily convergent[J]. Math Program 155(1):57–79

    Article  MathSciNet  MATH  Google Scholar 

  32. Wang Z (2020) A new clustering method based on morphological operations[J]. Expert Syst Appl 145:113102

    Article  Google Scholar 

  33. Chen L, Jiang X, Liu X et al (2020) Robust low–rank tensor recovery via nonconvex singular value minimization[J]. IEEE Trans Image Process 29:9044–9059

    Article  MathSciNet  MATH  Google Scholar 

  34. Wang S, Chen Y, Cen Y et al (2022) Nonconvex low–rank and sparse tensor representation for multi-view subspace clustering[J]. Appl Intell:1–14

  35. Qin W, Wang H, Zhang F et al (2022) Low–rank high-order tensor completion with applications in visual data[J]. IEEE Trans Image Process 31:2433–2448

    Article  Google Scholar 

  36. Halder B, Ahmed MM, Amagasa T et al (2022) Missing information in imbalanced data stream: fuzzy adaptive imputation approach[J]. Appl Intell 52(5):5561–5583

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Grant Nos.11401383 and 62073223). The authors thank the editor and three anonymous referees for their insightful comments and helpful suggestions.

Author information

Authors and Affiliations

  1. College of Science, University of Shanghai for Science and Technology, Shanghai, 200093, China

    Yuyu Hu & Yali Fan

  2. Department of Control Science and Engineering, University of Shanghai for Science and Technology, Shanghai, 200093, China

    Yan Song & Ming Li

Authors
  1. Yuyu Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yali Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yan Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ming Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yali Fan.

Ethics declarations

Competing interests

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hu, Y., Fan, Y., Song, Y. et al. A general robust low–rank multinomial logistic regression for corrupted matrix data classification. Appl Intell 53, 18564–18580 (2023). https://doi.org/10.1007/s10489-022-04424-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-04424-0

Keywords

Search

Navigation