Skip to main content
SpringerLink
Log in

A two-phase projective dictionary pair learning-based classification scheme for positive and unlabeled learning

  • Theoretical Advances
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

With the recent surge of interest in machine learning, Positive and Unlabeled learning (PU learning) has also attracted much attention of scholars. A key bottleneck for addressing PU classification is the absence of training negative data, and thus many popular approaches belonging to the “two-step” strategy have been proposed. However, almost none of the existing two-step methods can thoroughly learn the feature information of samples, which makes the extracted negative samples unreliable and easily leads to undesirable results. Therefore, in this paper, we propose a two-phase projective dictionary pair learning (TPDPL) method for PU learning. The first phase of TPDPL determines reliable negatives by exploiting the reconstruction residuals and the second phase trains the DPL-based classifier with the extracted reliable negative and original positive samples to perform classification. Our experimental results demonstrate that the TPDPL approach can achieve highly competitive classification performance when compared with conventional and state-of-the-art PU learning algorithms. More importantly, due to the special dictionary pair learning framework, the computational complexity of TPDPL is extraordinarily low.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig.1

Similar content being viewed by others

Data availability

All data included in this study are available upon request by contact with the corresponding author.

References

  1. Bekker J, Davis J (2020) Learning from positive and unlabeled data: a survey. Mach Learn 109(4):719–760

    Article  MathSciNet  MATH  Google Scholar 

  2. Denis F, Gilleron R, Letouzey F (2005) Learning from positive and unlabeled examples. Theoret Comput Sci 348(1):70–83

    Article  MathSciNet  MATH  Google Scholar 

  3. Scott C, Blanchard G (2009) Novelty detection: unlabeled data definitely help. Artif Intell Stats 5:464–471

    Google Scholar 

  4. Blanchard G, Lee G, Scott C (2010) Semi-supervised novelty detection. J Mach Learn Res 11:2973–3009

    MathSciNet  MATH  Google Scholar 

  5. Pramanik S, Sagayam KM, Jena OP (2021) Machine Learning Frameworks in Cancer Detection. Int Conf Comput Sci Renew Energ (ICCSRE2021) 297:01073

    Google Scholar 

  6. Li W, Guo Q, Elkan C (2011) A positive and unlabeled learning algorithm for one-class classification of remote-sensing data. IEEE Trans Geosci Remote Sens 49(2):17–25

    Article  Google Scholar 

  7. Sansone E, De Natale FG, Zhou Z-H (2018) Efficient training for positive unlabeled learning. IEEE Trans Pattern Anal Mach Intell 41(11):2584–2598

    Article  Google Scholar 

  8. Ashok Kumar PM, Jeevan BM, Sagayam KM (2021) Enhanced facial emotion recognition by optimal descriptor selection with neural network. IETE J Res. https://doi.org/10.1090/03772063.2021.1902868

    Article  Google Scholar 

  9. Sagayam KM, Andrushia AD, Ghosh A et al (2021) Recognition of hand gesture image using deep convolutional neural network. Int J Image Graph. https://doi.org/10.1142/S0219467821400088

    Article  Google Scholar 

  10. Liu B, Dai Y, Li X, Lee WS, Yu PS (2003) Building text classifiers using positive and unlabeled examples. IEEE Int Conf Data Mining 2:179–186

    Article  Google Scholar 

  11. Shi H, Pan S, Yang J, Gong C (2018) Positive and unlabeled learning via loss decomposition and centroid estimation. Int Joint Conf Artif Intell IJCAI 18:2689–2695

    Google Scholar 

  12. Frénay B, Verleysen M (2014) Classification in the presence of label noise: a survey. IEEE Trans Neural Netw Learn Syst 25(5):845–869

    Article  MATH  Google Scholar 

  13. Natarajan N, Dhillon IS, Ravikumar PK, Tewari A (2013) Learning with noisy labels. Adv Neural Inf Process Systems (NIPS) 26:1196–1204

    MATH  Google Scholar 

  14. Tanaka D, Ikami D, Yamasaki T, and Aizawa K (2018) Joint optimization framework for learning with noisy labels. In: proceedings of the IEEE conference on computer vision and pattern recognition CVPR, pp. 5552–5560

  15. Zhang C, Ren D, Liu T, Yang J and Gong C (2019) Positive and unlabeled learning with label disambiguation. In: twenty-eighth international joint conference on artificial intelligence IJCAI-19, pp. 4250–4256

  16. Gangeh MJ., Farahat AK, Ghodsi A, and Kamel MS (2015) Supervised dictionary learning and sparse representation-a review. Comput Sci, arXiv:1502.05928

  17. Rajesh G, Raajini XM, Sagayam KM et al (2020) A statistical approach for high order epistasis interaction detection for prediction of diabetic macular Edema. Inform Med Unlocked 29:1–9. https://doi.org/10.1016/j.imu.2020.100362

    Article  Google Scholar 

  18. Harriat Christa G, JJ, A. K, and K. M. Sagayam (2021) CNN-based mask detection system using OpenCV and MobileNetV2.In: 2021 3rd international conference on signal processing and communication (ICPSC) pp. 115–119, https://doi.org/10.1109/ICSPC51351.2021.9451688

  19. Aharon M, Elad M, Bruckstein A (2006) K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans Signal Process 54(11):4311–4322

    Article  MATH  Google Scholar 

  20. Yang M, Zhang L, Feng X, Zhang D (2011) Fisher discrimination dictionary learning for sparse representation. Int Conf Comput Vis 2011:543–550

    Google Scholar 

  21. Cai S, Zuo W, Zhang L, Feng X, and Wang P (2014) Support vector guided dictionary learning. European Conf Comput Vis, pp. 624–639

  22. Gu S, Zhang L, Zuo W, Feng X (2014) Projective dictionary pair learning for pattern classification. Adv Neural Inf Process System 27:793–801

    Google Scholar 

  23. Xu Y, Zhang D, Yang J, Yang J-Y (2011) A two-phase test sample sparse representation method for use with face recognition. IEEE Trans Circuits Syst Video Technol 21(9):1255–1262

    Article  MathSciNet  Google Scholar 

  24. Xu Y, Zhu Q, Chen Y, Pan J-S (2013) An improvement to the nearest neighbor classifier and face recognition experiments. Int J Innov Comput Inf Control 9(2):543–554

    Google Scholar 

  25. Zhu X, Goldberg A (2009) Introduction to semi-supervised learning. Synth Lect Axrtif Intell Mach Learn 3(1):1–130

    MATH  Google Scholar 

  26. Gong C, Tao D, Maybank SJ, Liu W, Kang G, Yang J (2016) Multi-modal curriculum learning for semi-supervised image classification. IEEE Trans Image Process 25(7):3249–3260

    Article  MathSciNet  MATH  Google Scholar 

  27. Niu G, Plessis M, Sakai T, Ma Y, Sugiyama M (2016) Theoretical comparisons of positive-unlabeled learning against positive-negative learning. Adv Neural Inf Process Syst (NIPS) 29:1199–1207

    Google Scholar 

  28. Gong C, Liu T, Yang J, Tao D (2019) Large-margin label-calibrated support vector machines for positive and unlabeled learning. IEEE Trans Neural Netw Learn Syst 30(11):1–13

    Article  Google Scholar 

  29. Kwon Y, Kim W, Sugiyama M, Paik MC (2020) Principled analytic classifier for positive-unlabeled learning via weighted integral probability metric. Mach Learn 109(3):513–532

    Article  MathSciNet  MATH  Google Scholar 

  30. Liu B, Lee W, Yu P, Li X (2002) Partially supervised classification of text documents. Int Conf Mach Learn 2(485):387–394

    Google Scholar 

  31. Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. J Roy Stat Soc 39(1):1–38

    MathSciNet  MATH  Google Scholar 

  32. Fung GPC, Lu HJ (2006) Text classification without negative examples revisit. IEEE Trans Knowl Data Eng 18(1):6–20

    Article  Google Scholar 

  33. Wee Sun Lee and Bing Liu (2003) Learning with positive and unlabeled examples using weighted logistic regression. Int Conf Mach Learn 3:448–455

    Google Scholar 

  34. Elkan C, Noto K (2008) Learning classifiers from only positive and unlabeled data. In: proceedings of the 14th ACM SIGKDD international conference on knowledge discovery and data mining, pp. 213–220

  35. Plessis MD, Niu G, Sugiyama M (2014) Analysis of learning from positive and unlabeled data. Adv Neural Inf Process Syst 27:703–711

    Google Scholar 

  36. Plessis MD, Niu G, Sugiyama M (2015) Convex formulation for learning from positive and unlabeled data. Int Conf Mach Learn 37:1386–1394

    Google Scholar 

  37. Kiryo R, Niu G, Plessis MCD, Sugiyama M (2017) Positive-unlabeled learning with non-negative risk estimator. Adv Neural Inf Process Syst (NIPS) 30:1674–1684

    Google Scholar 

  38. Hou M, Chaib-draa B, Li C, and Zhao Q (2018) Generative adversarial positive-unlabeled learning. Int Joint Conf Artif Intell (IJCAI), pp. 2255–2261

  39. Sakai T, Plessis MCD, Niu G, Sugiyama M (2017) Semi-supervised classification based on classification from positive and unlabeled data. Int Conf Mach Learn 70:2998–3006

    Google Scholar 

  40. Gong T, Wang G, Ye J, Xu Z, Lin M (2018) Margin based PU learning. AAAI Conf Artif Intell (AAAI) 32(1):1–8

    Google Scholar 

  41. Natarajan BK (1995) Sparse approximate solutions to linear systems. SIAM J Comput 24(2):227–234

    Article  MathSciNet  MATH  Google Scholar 

  42. Huang M, Wei Y, Jiang J et al (2014) Brain extraction based on locally linear representation-based classification. Neuroimage 92(10):322–339

    Article  Google Scholar 

  43. Wright J, Yang AY, Ganesh A, Sastry SS, Ma Y (2009) Robust face recognition via sparse representation. IEEE Trans Pattern Anal Mach Intell 31(2):210–227

    Article  Google Scholar 

  44. Pati YC, Rezaiifar R, and Krishnaprasad PS (1993) Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition. In: proceedings of 27th asilomar conference on signals, systems and computers, pp. 40–44

  45. Zhang Q, Li B (2011) Discriminative K-SVD for dictionary learning in face recognition. In: IEEE conference on computer vision and pattern recognition (CVPR), pp. 2691–2698

  46. Jiang Z, Lin Z, Davis LS (2011) Learning a discriminative dictionary for sparse coding via label consistent K-SVD. In: IEEE conference on computer vision and pattern recognition (CVPR), pp. 1697–1704

  47. Goldstein TOM, Donoghue BO, Setzer S, Baraniuk R (2014) Fast alternating direction optimization methods. SIAM J Imag Sci 7(3):1588–1623

    Article  MathSciNet  MATH  Google Scholar 

  48. Vanschoren J, van Rijn JN, Bischl B, Torgo L (2014) OpenML: networked science in machine learning. ACM SIGKDD Explorations Newsl 15(2):49–60

    Article  Google Scholar 

  49. The MNIST handwritten digit database, http:// www.cad.zju.edu.cn/home/dengcai/Data/MLData.html.

  50. Ameri R, Pouyan A, Abolghasemi V (2016) Projective dictionary pair learning for eeg signal classification in brain computer interface applications. Neurocomputing 218:382–389

    Article  Google Scholar 

  51. Zhang L, Shen Y, Li HY, Lu J (2014) 3D palmprint identification using block-wise features and collaborative representation. IEEE Trans Pattern Anal Mach Intell 37(8):1730–1736

    Article  Google Scholar 

  52. Wang Y, Peng Y, He K, Liu S, Li J (2021) A two-step classification method based on collaborative representation for positive and unlabeled learning. Neural Process Lett 53(6):4239–4255

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China (No.61873155), Transfer and Promotion Plan of Scientific and Technological Achievements of Shaanxi Province (No.2019CGXNG-019), the National Natural Science Foundation of Shaanxi Province (No.2018JM6050), Innovation Chain of Key Industries of Shaanxi Province (No.2019ZDLSF07-01).

Author information

Authors and Affiliations

  1. Key Laboratory of Modern Teaching Technology, Ministry of Education, Xi’an, 710062, China

    Yijin Wang & Yali Peng

  2. School of Computer Science, Shaanxi Normal University, Xi’an, 710119, China

    Yijin Wang, Yali Peng & Shigang Liu

  3. School of Physics and Information Technology, Shaanxi Normal University, Xi’an, 710119, China

    Bao Ge

  4. School of Computer Science and Technology, Nanjing Normal University, Nanjing, 210046, China

    Jun Li

Authors
  1. Yijin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yali Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shigang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bao Ge
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jun Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yali Peng.

Ethics declarations

Conflict of interest

The authors declared that they have no conflicts of interest to this work.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Y., Peng, Y., Liu, S. et al. A two-phase projective dictionary pair learning-based classification scheme for positive and unlabeled learning. Pattern Anal Applic 26, 1253–1263 (2023). https://doi.org/10.1007/s10044-023-01151-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-023-01151-1

Keywords

Search

Navigation