Skip to main content
SpringerLink
Log in

Robust classification with reject option using the self-organizing map

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Reject option is a technique used to improve classifier’s reliability in decision support systems. It consists in withholding the automatic classification of an item, if the decision is considered not sufficiently reliable. The rejected item is then handled by a different classifier or by a human expert. The vast majority of the works on this issue has been concerned with the development of reject option mechanisms to be used by supervised learning architectures (e.g., MLP, LVQ or SVM). In this paper, however, we aim at proposing alternatives to this view, which are based on the self-organizing map (SOM), originally an unsupervised learning scheme, but that has also been successfully used in the design of prototype-based classifiers. The basic hypothesis we defend is that it is possible to design SOM-based classifiers endowed with reject option mechanisms whose performances are comparable to or better than those achieved by standard supervised classifiers. For this purpose, we carried out a comprehensively evaluation of the proposed SOM-based classifiers on two synthetic and three real-world datasets. The obtained results suggest that the proposed SOM-based classifiers consistently outperform standard supervised classifiers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. There are reject option strategies that are executed during the training of the classifier. These are known as embedded reject option mechanisms. See Sect. 2 for more detail.

  2. At the limit, a classifier with a very low \({\upomega }_r\) would classify only the so-called “easy patterns”.

  3. Available for download at http://www.cis.hut.fi/somtoolbox/.

  4. Note that there are more prototypes over patterns of the class “red” in Fig. 3 (left) than in Fig. 3 (center).

  5. Values of \({\upomega }_r\) higher than \(0.5\) are equivalent to random guesses.

References

  1. Alhoniemi E, Himberg J, Vesanto J (1999) Probabilistic measures for responses of self-organizing map units. In: Proceedings of the international ICSC congress on computational intelligence methods and applications (CIMA’99). ICSC Academic Press, pp 286–290

  2. Bartlett PL, Wegkamp MH (2008) Classification with a reject option using a Hinge loss. J Mach Learn Res 9:1823–1840

    MathSciNet  MATH  Google Scholar 

  3. Bellazzi R, Abu-Hanna A (2009) Artificial intelligence in medicine AIME’07. Artif Intell Med 46(1):1–3

    Article  Google Scholar 

  4. Berglund E, Sitte J (2006) Parameterless self-organizing map algorithm. IEEE Trans Neural Netw 17(2):305–316

    Article  Google Scholar 

  5. Biehl M, Ghosh A, Hammer B (2007) Dynamics and generalization ability of LVQ algorithms. J Mach Learn Res 8:323–360

    MathSciNet  MATH  Google Scholar 

  6. Bounsiar A, Beauseroy P, Grall-Maës E (2008) General solution and learning method for binary classification with performance constraints. Pattern Recognit Lett 29(10):1455–1465

    Article  Google Scholar 

  7. Cardoso JS, Cardoso MJ (2007) Towards an intelligent medical system for the aesthetic evaluation of breast cancer conservative treatment. Artif Intell Med 40:115–126

    Article  Google Scholar 

  8. Cardoso JS, da Costa JFP (2007) Learning to classify ordinal data: the data replication method. J Mach Learn Res 8:1393–1429

    MathSciNet  MATH  Google Scholar 

  9. Carpenter GA, Grossberg S, Markuzon N, Reynolds JH, Rosen DB (1992) Fuzzy ARTMAP: a neural network architecture for incremental supervised learning of analog multidimensional maps. IEEE Trans Neural Netw 3(5):698–713

    Article  Google Scholar 

  10. Caruana R, Lawrence S, Giles CL (2000) Overfitting in neural nets: backpropagation, conjugate gradient, and early stopping. In: Proceedings of the 2000 neural information processing systems conference (NIPS’00), pp 402–408

  11. Chow C (1970) On optimum recognition error and reject tradeoff. IEEE Trans Inf Theory 16(1):41–46

    Article  MATH  Google Scholar 

  12. Cordella L, De Stefano C, Sansone C, Vento M (1995) An adaptive reject option for LVQ classifiers. In: Image analysis and processing, vol. LNCS 974/1995. Springer, pp 68–73

  13. Cordella L, De Stefano C, Tortorella F, Vento M (1995) A method for improving classification reliability of multilayer perceptrons. IEEE Trans Neural Netw 6(5):1140–1147

    Article  Google Scholar 

  14. de Bodt E, Cottrell M, Letremy P, Verleysen M (2004) On the use of self-organizing maps to accelerate vector quantization. Neurocomputing 56:187–203

    Article  Google Scholar 

  15. De Stefano C, Sansone C, Vento M (2000) To reject or not to reject: that is the question—an answer in case of neural classifiers. IEEE Trans Syst Man Cybern C Appl Rev 30(1):574–585

    Article  Google Scholar 

  16. El-Yaniv R, Wiener Y (2010) On the foundations of noise-free selective classification. J Mach Learn Res 11:1605–1641

    MathSciNet  MATH  Google Scholar 

  17. Flexer A (2001) On the use of self-organizing maps for clustering and visualization. Intell Data Anal 5(5):373–384

    MATH  Google Scholar 

  18. Fritzke B (1995) A growing neural gas network learns topologies. In: Tesauro G, Touretzky DS, Leen TK (eds) Advances in neural information processing systems vol 7. MIT Press, Cambridge, pp 625–632

  19. Fu Y, Zhu X, Li B (2013) A survey on instance selection for active learning. Knowl Inf Syst 35(2):249–283

    Article  Google Scholar 

  20. Fumera G, Pillai I, Roli F (2003) Classification with reject option in text categorisation systems. In: Proceedings of the 12th international conference on image analysis and processing (ICIAP’2003). IEEE Computer Society, pp 582–587

  21. Fumera G, Roli F (2002) Support vector machines with embedded reject option. In: Proceedings of the 1st international workshop on pattern recognition with support vector machines (SVM’2002). Springer, pp 68–82

  22. Gama J, de Carvalho AC (2012) Machine learning. In: Machine learning: concepts, methodologies, tools and applications. IGI-Global, pp 13–22. doi:10.4018/978-1-60960-818-7. http://www.igi-global.com/book/machine-learning-concepts-methodologies-tools/50312

  23. Eduardo Gasca A, Sergio Saldaña T, José S. Sánchez G, Valentín Velásquez G, Eréndira Rendón L, Itzel M. Abundez B, Rosa M. Valdovinos R, Rafael Cruz R (2011) A rejection option for the multilayer perceptron using hyperplanes. In: Proceedings of the 10th international conference on adaptive and natural computing algorithms (ICANNGA’2011), vol. LNCS 6593/2011. Springer, pp 51–60

  24. Geebelen D, Suykens J, Vandewalle J (2012) Reducing the number of support vectors of SVM classifiers using the smoothed separable case approximation. IEEE Trans Neural Netw Learn Syst 23(4):682–688

    Article  Google Scholar 

  25. Giles D (2004) Calculating a standard error for the gini coefficient: some further results. Oxford Bull Econ Stat 66(3):124–126

    Article  Google Scholar 

  26. Gini C (1921) Measurement of inequality of incomes. Econ J 31(121):124–126

    Article  Google Scholar 

  27. Goldszmidt M, Cohen I, Fox A, Zhang S (2005) Three research challenges at the intersection of machine learning, statistical induction, and systems. In: Proceedings of the 10th conference on Hot Topics in Operating Systems (HOTOS’05), vol 10, pp 1–6

  28. Guillen A, Herrera LJ, Rubio G, Pomares H, Lendasse A, Rojas I (2010) New method for instance or prototype selection using mutual information in time series prediction. Neurocomputing 73(10–12):2030–2038

    Article  Google Scholar 

  29. Han J, Gao J (2009) Research challenges for data mining in science and engineering. In: Kargupta H, Han J, Yu PS, Motwani R, Kumar V (eds) Next generation of data mining. Chapman & Hall, London, pp 1–18

    Google Scholar 

  30. Hasenjäger M, Ritter H (1998) Active learning with local models. Neural Process Lett 7(2):107–117

    Article  Google Scholar 

  31. Herbei R, Wegkamp MH (2006) Classification with reject option. Can J Stat 34(4):709–721

    Article  MathSciNet  MATH  Google Scholar 

  32. Holmström L, Hämäläinen A (1993) The self-organizing reduced kernel density estimator. In: Proceedings of the 1993 IEEE International Conference on Neural Networks (ICNN’93), pp 417–421

  33. Ishibuchi H, Nii M (2000) Neural networks for soft decision making. Fuzzy Sets Syst 34(115):121–140

    Article  Google Scholar 

  34. Kohonen T (1982) Self-organized formation of topologically correct feature maps. Biol Cybern 43(1):59–69

    Article  MathSciNet  MATH  Google Scholar 

  35. Kohonen T (1988) An introduction to neural computing. Neural Netw 1(1):3–16

    Article  Google Scholar 

  36. Kohonen T (1988) The ’neural’ phonetic typewriter. Computer 21(3):11–22

    Article  Google Scholar 

  37. Kohonen T (1990) The self-organizing map. Proc IEEE 78(9):1464–1480

    Article  Google Scholar 

  38. Kohonen T (2001) Self-organizing maps, 3rd edn. Springer, Berlin, Heidelberg, New York

    Book  MATH  Google Scholar 

  39. Kohonen T (2003) Learning vector quantization. In: Arbib MA (ed) The handbook of brain theory, neural networks, 2nd edn. MIT Press, Cambridge, pp 631–635

    Google Scholar 

  40. Lau KW, Yin H, Hubbard S (2006) Kernel self-organising maps for classification. Neurocomputing 69:2033–2040

    Article  Google Scholar 

  41. Lotte F, Mouchère H, Lécuyer A (2008) Pattern rejection strategies for the design of self-paced EEG-based brain–computer interfaces. In: Proceedings of the 19th international conference on pattern recognition (ICPR’2008), pp 1–5

  42. Malone J, McGarry K, Wermter S, Bowerman C (2005) Data mining using rule extraction from Kohonen self-organising maps. Neural Comput Appl 15:9–17

    Article  Google Scholar 

  43. Mattos CLC, Barreto GA (2013) ARTIE and MUSCLE models: building ensemble classifiers from fuzzy ART and SOM networks. Neural Comput Appl 22(1):49–61

    Article  Google Scholar 

  44. Oliveira HP, Magalhaes A, Cardoso MJ, Cardoso JS (2010) An accurate and interpretable model for BCCT.core. In: Proceedings of the 32nd annual international conference of the IEEE engineering in medicine and biology Society, pp 6158–6161

  45. Pedreira CE (2006) Learning vector quantization with training data selection. IEEE Trans Pattern Anal Mach Intell 28(1):157–162

    Article  Google Scholar 

  46. Peng H, Zhu S (2007) Handling of incomplete data sets using ICA and SOM in data mining. Neural Comput Appl 16(2):167–172

    Article  Google Scholar 

  47. Ritter H (1991) Asymptotic level density for a class of vector quantization processes. IEEE Trans Neural Netw 2(1):173–175

    Article  MathSciNet  Google Scholar 

  48. Riveiro M, Johansson F, Falkman G, Ziemke T (2008) Supporting maritime situation awareness using self organizing maps and gaussian mixture models. In: Proceedings of the 2008 conference on 10th scandinavian conference on artificial intelligence (SCAI’08). IOS Press, pp 84–91

  49. Rocha-Neto AR, Sousa R, Cardoso JS, Barreto GA (2011) Diagnostic of pathology on the vertebral column with embedded reject option. In: Proceedings of the 5th Iberian conference on pattern recognition and image analysis (IbPRIA’2011), vol. LNCS-6669, pp 588–595

  50. Santos-Pereira CM, Pires AM (2005) On optimal reject rules and ROC curves. Pattern Recogn Lett 26(7):943–952

    Article  Google Scholar 

  51. Schleif FM, Villmann T, Hammer B, Schneider P (2011) Efficient kernelized prototype based classification. Int J Neural Syst 21(6):443–57

    Article  Google Scholar 

  52. Seo S, Obermayer K (2002) Soft learning vector quantization. Neural Comput 15:1589–1604

    Article  Google Scholar 

  53. Sim SF, Sági-Kiss V (2011) Multiple self-organising maps (mSOMs) for simultaneous classification and prediction: Illustrated by spoilage in apples using volatile organic profiles. Chemometr Intell Lab Syst 109(1):57–64

    Article  Google Scholar 

  54. Sousa R, Mora B, Cardoso JS (2009) An ordinal data method for the classification with reject option. In: Proceedings of the international conference on machine learning and applications (ICMLA’09), pp 746–750

  55. Sousa R, Rocha Neto AR, Barreto GA, Cardoso JS, Coimbra MT (2014) Reject option paradigm for the reduction of support vectors. In: Proceedings of the 22th European symposium on artificial neural networks, computational intelligence and machine learning (ESANN’2014), pp 1–6

  56. Souza Júnior AH, Barreto GA, Varela AT (2011) A speech recognition system for embedded applications using the SOM and TS-SOM networks. In: Mwasiagi JI (ed) Self-organizing maps—applications and novel algorithm design. InTech Open, Rijeka, pp 97–108. doi:10.5772/14401

  57. Suutala J, Pirttikangas S, Riekki J, Röning J (2004) Reject-optional LVQ-based two-level classifier to improve reliability in footstep identification. In: Ferscha A, Mattern F (eds) Pervasive computing. Springer, Berlin, Heidelberg, pp 182–187

  58. Thomas LC, Edelman DB, Crook JN (2002) Credit scoring and its applications, 1st edn. SIAM, Philadelphia

    Book  MATH  Google Scholar 

  59. Tortorella F (2005) A ROC-based reject rule for dichotomizers. Pattern Recognit Lett 26(2):167–180

    Article  MathSciNet  Google Scholar 

  60. Turky AM, Ahmad MS (2010) The use of SOM for fingerprint classification. In: IEEE international conference on information retrieval and knowledge management (CAMP’2010), pp 287–290

  61. Umer MF, Khiyal MSH (2007) Classification of textual documents using learning vector quantization. Inf Technol J 6:154–159

    Article  Google Scholar 

  62. Utsugi A (1998) Density estimation by mixture models with smoothing priors. Neural Comput 10:2115–2135

    Article  Google Scholar 

  63. van Hulle M (2012) Self-organizing maps. In: Rozenberg G, Baeck T, Kok J (eds) Handbook of natural computing: theory, experiments, and applications. Springer, Berlin, Heidelberg, pp 585–622

    Chapter  Google Scholar 

  64. Vasconcelos GC, Fairhurst MC, Bisset DL (1993) Enhanced reliability of multilayer perceptron networks through controlled pattern rejection. Electron Lett 29(3):261–263

    Article  Google Scholar 

  65. Vasconcelos GC, Fairhurst MC, Bisset DL (1995) Investigating feedforward neural networks with respect to the rejection of spurious patterns. Pattern Recognit Lett 16(2):207–212

    Article  Google Scholar 

  66. Villmann T, Haase S (2011) Divergence-based vector quantization. Neural Comput 23(5):1343–1392

    Article  MathSciNet  MATH  Google Scholar 

  67. Yin H (2008) The self-organizing maps: background, theories, extensions and applications. In: Fulcher J, Jain LC (eds) Computational intelligence: a compendium, studies in computational intelligence, vol 115. Springer, Berlin, Heidelberg, pp 715–762

    Chapter  Google Scholar 

  68. Yin H, Allinson NM (2001) Self-organizing mixture networks for probability density estimation. IEEE Trans Neural Netw 12(2):405–411

    Article  Google Scholar 

  69. Zidelmal Z, Amirou A, Belouchrani A (2012) Heartbeat classification using support vector machines (SVMs) with an embedded reject option. Int J Pattern Recognit Artif Intell 26(1):1250,001-1–1250,001-17

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgments

This work was partially supported through Program CNPq/Universidade do Porto/590008/2009-9 and conducted when Ricardo Sousa was in internship at Universidade Federal do Ceará (UFC), Brazil. This work was also partially funded by Fundação para a Ciência e a Tecnologia (FCT)—Portugal through project PTDC/SAU-ENB/114951/2009 and by FEDER funds through the Programa Operacional Factores de Competitividade—COMPETE in the framework of the project PEst-C/SAU/LA0002/2013. The authors also thank Fundação Núcleo de Tecnologia Industrial do Ceará (NUTEC) for providing the laboratorial infrastructure for the execution of the research activities reported in this paper.

Author information

Author notes

    Authors and Affiliations

    1. Instituto de Investigação e Inovação em Saúde, Universidade do Porto, Porto, Portugal

      Ricardo Gamelas Sousa

    2. INEB – Instituto de Engenharia Biomédica, Universidade do Porto, Porto, Portugal

      Ricardo Gamelas Sousa

    3. Departamento de Telemática, Instituto Federal do Ceará (IFCE), Fortaleza, Ceará, Brasil

      Ajalmar R. Rocha Neto

    4. INESC TEC and Faculdade de Engenharia da Universidade do Porto, Porto, Portugal

      Jaime S. Cardoso

    5. Departamento de Engenharia de Teleinformática, Universidade Federal do Ceará (UFC), Fortaleza, Ceará, Brasil

      Guilherme A. Barreto

    Authors
    1. Ricardo Gamelas Sousa
      View author publications

      You can also search for this author in PubMed Google Scholar

    2. Ajalmar R. Rocha Neto
      View author publications

      You can also search for this author in PubMed Google Scholar

    3. Jaime S. Cardoso
      View author publications

      You can also search for this author in PubMed Google Scholar

    4. Guilherme A. Barreto
      View author publications

      You can also search for this author in PubMed Google Scholar

    Corresponding author

    Correspondence to Ajalmar R. Rocha Neto.

    Additional information

    Ricardo Gamelas Sousa and Ajalmar R. Rocha Neto have contributed equally to this manuscript.

    Rights and permissions

    Reprints and permissions

    About this article

    Check for updates. Verify currency and authenticity via CrossMark

    Cite this article

    Gamelas Sousa, R., Rocha Neto, A.R., Cardoso, J.S. et al. Robust classification with reject option using the self-organizing map. Neural Comput & Applic 26, 1603–1619 (2015). https://doi.org/10.1007/s00521-015-1822-2

    Download citation

    • Received:

    • Accepted:

    • Published:

    • Issue Date:

    • DOI: https://doi.org/10.1007/s00521-015-1822-2

    Keywords

    Search

    Navigation