Skip to main content
SpringerLink
Log in

A precise and stable machine learning algorithm: eigenvalue classification (EigenClass)

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

In this study, a precise and efficient eigenvalue-based machine learning algorithm, particularly denoted as Eigenvalue Classification (EigenClass) algorithm, has been presented to deal with classification problems. The EigenClass algorithm is constructed by exploiting an eigenvalue-based proximity evaluation. To appreciate the classification performance of EigenClass, it is compared with the well-known algorithms, such as k-nearest neighbours, fuzzy k-nearest neighbours, random forest (RF) and multi-support vector machines. Number of 20 different datasets with various attributes and classes are used for the comparison. Every algorithm is trained and tested for 30 runs through 5-fold cross-validation. The results are then compared among each other in terms of the most used measures, such as accuracy, precision, recall, micro-F-measure, and macro-F-measure. It is demonstrated that EigenClass exhibits the best classification performance for 15 datasets in terms of every metric and, in a pairwise comparison, outperforms the other algorithms for at least 16 datasets in consideration of each metric. Moreover, the algorithms are also compared through statistical analysis and computational complexity. Therefore, the achieved results show that EigenClass is a precise and stable algorithm as well as the most successful algorithm considering the overall classification performances.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Jin R, Zhang J (2007) Multi-class learning by smoothed boosting. Mach Learn 67:207–227. https://doi.org/10.1007/s10994-007-5005-y

    Article  Google Scholar 

  2. Takenouchi T, Ishii S (2018) Binary classifiers ensemble based on Bregman divergence for multi-class classification. Neurocomputing 273:424–434. https://doi.org/10.1016/j.neucom.2017.08.004

    Article  Google Scholar 

  3. Li P (2019) Research on radar signal recognition based on automatic machine learning. Neural Comput Appl. https://doi.org/10.1007/s00521-019-04494-1

    Article  Google Scholar 

  4. Takenouchi T, Ishii S (2011) Ternary Bradley-Terry model-based decoding for multi-class classification and its extensions. Mach Learn 85:249–272. https://doi.org/10.1007/s10994-011-5240-0

    Article  MathSciNet  MATH  Google Scholar 

  5. Xu H, Wang W, Qian Y (2017) Fusing complete monotonic decision trees. IEEE Trans Knowl Data Eng 29:2223–2235. https://doi.org/10.1109/TKDE.2017.2725832

    Article  Google Scholar 

  6. Liu T, Tao D (2016) Classification with noisy labels by importance reweighting. IEEE Trans Pattern Anal Mach Intell 38:447–461. https://doi.org/10.1109/TPAMI.2015.2456899

    Article  Google Scholar 

  7. Langseth H, Nielsen TD (2006) Classification using hierarchical Naïve Bayes models. Mach Learn 63:135–159. https://doi.org/10.1007/s10994-006-6136-2

    Article  MATH  Google Scholar 

  8. Nebel D, Kaden M, Villmann A, Villmann T (2017) Types of (dis-)similarities and adaptive mixtures thereof for improved classification learning. Neurocomputing 268:42–54. https://doi.org/10.1016/j.neucom.2016.12.091

    Article  Google Scholar 

  9. Quost B, Destercke S (2017) Classification by pairwise coupling of imprecise probabilities. Pattern Recognit 77:412–425. https://doi.org/10.1016/j.patcog.2017.10.019

    Article  Google Scholar 

  10. Law A, Ghosh A (2019) Multi-label classification using a cascade of stacked autoencoder and extreme learning machines. Neurocomputing 358:222–234. https://doi.org/10.1016/j.neucom.2019.05.051

    Article  Google Scholar 

  11. Samaniego L, Bárdossy A, Schulz K (2008) Supervised classification of remotely sensed imagery using a modified k-NN technique. IEEE Trans Geosci Remote Sens 46:1–26. https://doi.org/10.1109/TGRS.2008.916629

    Article  Google Scholar 

  12. Warfield S (1996) Fast k-NN classification for multichannel image data. Pattern Recognit Lett 17:713–721. https://doi.org/10.1016/0167-8655(96)00036-0

    Article  Google Scholar 

  13. Zhang JJ, Fang M, Li X (2017) Clustered intrinsic label correlations for multi-label classification. Expert Syst Appl 81:134–146. https://doi.org/10.1016/j.eswa.2017.03.054

    Article  Google Scholar 

  14. Liu Z, Cheng Y, Wang P et al (2018) A method for remaining useful life prediction of crystal oscillators using the Bayesian approach and extreme learning machine under uncertainty. Neurocomputing 305:27–38. https://doi.org/10.1016/j.neucom.2018.04.043

    Article  Google Scholar 

  15. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20:273–297. https://doi.org/10.1023/A:1022627411411

    Article  MATH  Google Scholar 

  16. Dudani SA (1976) The distance-weighted k-Nearest-neighbor rule. IEEE Trans Syst Man Cybern SMC-6:325–327. https://doi.org/10.1109/tsmc.1976.5408784

    Article  Google Scholar 

  17. Breiman L (2001) Random forests. Mach Learn 45:5–32. https://doi.org/10.1023/A:1010933404324

    Article  MATH  Google Scholar 

  18. Li S, Song S, Wan Y (2018) Laplacian twin extreme learning machine for semi-supervised classification. Neurocomputing 321:17–27. https://doi.org/10.1016/j.neucom.2018.08.028

    Article  Google Scholar 

  19. Zhang C, Liu C, Zhang X, Almpanidis G (2017) An up-to-date comparison of state-of-the-art classification algorithms. Expert Syst Appl 82:128–150. https://doi.org/10.1016/j.eswa.2017.04.003

    Article  Google Scholar 

  20. Noh Y-K, Zhang B-T, Lee DD (2018) Generative local metric learning for nearest neighbor classification. IEEE Trans Pattern Anal Mach Intell 40:106–118. https://doi.org/10.1109/TPAMI.2017.2666151

    Article  Google Scholar 

  21. Wang X, Shen S, Shi G et al (2016) Iterative non-local means filter for salt and pepper noise removal. J Vis Commun Image Represent 38:440–450. https://doi.org/10.1016/j.jvcir.2016.03.024

    Article  Google Scholar 

  22. Vladimir Naumovich V (1998) Statistical learning theory. Springer, New York

    Google Scholar 

  23. Ai Q, Wang A, Wang Y, Sun H (2019) An improved Twin-KSVC with its applications. Neural Comput Appl 31:6615–6624. https://doi.org/10.1007/s00521-018-3487-0

    Article  Google Scholar 

  24. Jolliffe IT (2002) Principal component analysis, 2nd edn. Springer, New York

    MATH  Google Scholar 

  25. Balakrishnama S, Ganapathiraju A (1998) Linear discriminant analysis—a brief tutorial. Inst Signal Inf Process 18:1–8

    Google Scholar 

  26. Keller JM, Gray MR (1985) A fuzzy K-nearest neighbor algorithm. IEEE Trans Syst Man Cybern SMC-15:580–585. https://doi.org/10.1109/tsmc.1985.6313426

    Article  Google Scholar 

  27. Shultz TR, Mareschal D, Schmidt WC (1994) Modeling cognitive development on balance scale phenomena. Mach Learn. https://doi.org/10.1023/A:1022630902151

    Article  Google Scholar 

  28. Dua D, Graff C (2019) UCI machine learning repository. School of Information and Computer Sciences University of California. http://archive.ics.uci.edu/ml. Accessed 13 Aug 2019

  29. Ustun D, Toktas A, Akdagli A (2019) Deep neural network-based soft computing the resonant frequency of E-shaped patch antennas. AEU Int J Electron Commun 102:54–61. https://doi.org/10.1016/j.aeue.2019.02.011

    Article  Google Scholar 

  30. Nguyen TT, Dang MT, Luong AV et al (2019) Multi-label classification via incremental clustering on an evolving data stream. Pattern Recognit 95:96–113. https://doi.org/10.1016/j.patcog.2019.06.001

    Article  Google Scholar 

  31. Abdar M, Zomorodi-Moghadam M, Zhou X et al (2018) A new nested ensemble technique for automated diagnosis of breast cancer. Pattern Recognit Lett. https://doi.org/10.1016/j.patrec.2018.11.004

    Article  Google Scholar 

  32. Adem K, Kiliçarslan S, Cömert O (2019) Classification and diagnosis of cervical cancer with softmax classification with stacked autoencoder. Expert Syst Appl 115:557–564. https://doi.org/10.1016/j.eswa.2018.08.050

    Article  Google Scholar 

  33. Friedman M (1940) A comparison of alternative tests of significance for the problem of m rankings. Ann Math Stat 11:86–92. https://doi.org/10.1214/aoms/1177731944

    Article  MathSciNet  MATH  Google Scholar 

  34. Nemenyi P (1963) Distribution-free multiple comparisons. Ph.D. Princeton University

  35. Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30

    MathSciNet  MATH  Google Scholar 

  36. Eigenvalues and eigenvectors. https://www.mathworks.com/help/matlab/ref/eig.html. Accessed 17 Feb 2020

  37. Bachmann P (1894) Analytische Zahlentheorie, vol 2. Teubner, Leipzig (in German)

    MATH  Google Scholar 

  38. Landau E (1909) Handbuch der Lehre von der Verteilung der Primzahlen. B. G. Teubner, Leipzig (in German)

    MATH  Google Scholar 

  39. Maillo J, Luengo J, García S et al (2017) Exact fuzzy k-nearest neighbor classification for big datasets. In: IEEE international conference on fuzzy systems (FUZZ-IEEE)

  40. Nikdel H, Forghani Y, Mohammad Hosein Moattar S (2018) Increasing the speed of fuzzy k-nearest neighbours algorithm. Expert Syst 35:e12254. https://doi.org/10.1111/exsy.12254

    Article  Google Scholar 

  41. Tsang IWH, Kwok JTY, Zurada JM (2006) Generalized core vector machines. IEEE Trans Neural Netw 17:1126–1140. https://doi.org/10.1109/TNN.2006.878123

    Article  Google Scholar 

  42. Buczak AL, Guven E (2016) A survey of data mining and machine learning methods for cyber security intrusion detection. IEEE Commun Surv Tutor 18:1153–1176. https://doi.org/10.1109/COMST.2015.2494502

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Engineering Faculty, Karamanoğlu Mehmetbey University, 70200, Karaman, Turkey

    Uğur Erkan

Authors
  1. Uğur Erkan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Uğur Erkan.

Ethics declarations

Conflict of interest

The author declares that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Erkan, U. A precise and stable machine learning algorithm: eigenvalue classification (EigenClass). Neural Comput & Applic 33, 5381–5392 (2021). https://doi.org/10.1007/s00521-020-05343-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-020-05343-2

Keywords

Search

Navigation