Neural Computing and Applications

, Volume 26, Issue 7, pp 1603–1619 | Cite as

Robust classification with reject option using the self-organizing map

  • Ricardo Gamelas Sousa
  • Ajalmar R. Rocha Neto
  • Jaime S. Cardoso
  • Guilherme A. Barreto
Original Article


Reject option is a technique used to improve classifier’s reliability in decision support systems. It consists in withholding the automatic classification of an item, if the decision is considered not sufficiently reliable. The rejected item is then handled by a different classifier or by a human expert. The vast majority of the works on this issue has been concerned with the development of reject option mechanisms to be used by supervised learning architectures (e.g., MLP, LVQ or SVM). In this paper, however, we aim at proposing alternatives to this view, which are based on the self-organizing map (SOM), originally an unsupervised learning scheme, but that has also been successfully used in the design of prototype-based classifiers. The basic hypothesis we defend is that it is possible to design SOM-based classifiers endowed with reject option mechanisms whose performances are comparable to or better than those achieved by standard supervised classifiers. For this purpose, we carried out a comprehensively evaluation of the proposed SOM-based classifiers on two synthetic and three real-world datasets. The obtained results suggest that the proposed SOM-based classifiers consistently outperform standard supervised classifiers.


Self-organizing maps Reject option Robust classification Prototype-based classifiers Neuron labeling 



This work was partially supported through Program CNPq/Universidade do Porto/590008/2009-9 and conducted when Ricardo Sousa was in internship at Universidade Federal do Ceará (UFC), Brazil. This work was also partially funded by Fundação para a Ciência e a Tecnologia (FCT)—Portugal through project PTDC/SAU-ENB/114951/2009 and by FEDER funds through the Programa Operacional Factores de Competitividade—COMPETE in the framework of the project PEst-C/SAU/LA0002/2013. The authors also thank Fundação Núcleo de Tecnologia Industrial do Ceará (NUTEC) for providing the laboratorial infrastructure for the execution of the research activities reported in this paper.


  1. 1.
    Alhoniemi E, Himberg J, Vesanto J (1999) Probabilistic measures for responses of self-organizing map units. In: Proceedings of the international ICSC congress on computational intelligence methods and applications (CIMA’99). ICSC Academic Press, pp 286–290Google Scholar
  2. 2.
    Bartlett PL, Wegkamp MH (2008) Classification with a reject option using a Hinge loss. J Mach Learn Res 9:1823–1840MathSciNetzbMATHGoogle Scholar
  3. 3.
    Bellazzi R, Abu-Hanna A (2009) Artificial intelligence in medicine AIME’07. Artif Intell Med 46(1):1–3CrossRefGoogle Scholar
  4. 4.
    Berglund E, Sitte J (2006) Parameterless self-organizing map algorithm. IEEE Trans Neural Netw 17(2):305–316CrossRefGoogle Scholar
  5. 5.
    Biehl M, Ghosh A, Hammer B (2007) Dynamics and generalization ability of LVQ algorithms. J Mach Learn Res 8:323–360MathSciNetzbMATHGoogle Scholar
  6. 6.
    Bounsiar A, Beauseroy P, Grall-Maës E (2008) General solution and learning method for binary classification with performance constraints. Pattern Recognit Lett 29(10):1455–1465CrossRefGoogle Scholar
  7. 7.
    Cardoso JS, Cardoso MJ (2007) Towards an intelligent medical system for the aesthetic evaluation of breast cancer conservative treatment. Artif Intell Med 40:115–126CrossRefGoogle Scholar
  8. 8.
    Cardoso JS, da Costa JFP (2007) Learning to classify ordinal data: the data replication method. J Mach Learn Res 8:1393–1429MathSciNetzbMATHGoogle Scholar
  9. 9.
    Carpenter GA, Grossberg S, Markuzon N, Reynolds JH, Rosen DB (1992) Fuzzy ARTMAP: a neural network architecture for incremental supervised learning of analog multidimensional maps. IEEE Trans Neural Netw 3(5):698–713CrossRefGoogle Scholar
  10. 10.
    Caruana R, Lawrence S, Giles CL (2000) Overfitting in neural nets: backpropagation, conjugate gradient, and early stopping. In: Proceedings of the 2000 neural information processing systems conference (NIPS’00), pp 402–408Google Scholar
  11. 11.
    Chow C (1970) On optimum recognition error and reject tradeoff. IEEE Trans Inf Theory 16(1):41–46CrossRefzbMATHGoogle Scholar
  12. 12.
    Cordella L, De Stefano C, Sansone C, Vento M (1995) An adaptive reject option for LVQ classifiers. In: Image analysis and processing, vol. LNCS 974/1995. Springer, pp 68–73Google Scholar
  13. 13.
    Cordella L, De Stefano C, Tortorella F, Vento M (1995) A method for improving classification reliability of multilayer perceptrons. IEEE Trans Neural Netw 6(5):1140–1147CrossRefGoogle Scholar
  14. 14.
    de Bodt E, Cottrell M, Letremy P, Verleysen M (2004) On the use of self-organizing maps to accelerate vector quantization. Neurocomputing 56:187–203CrossRefGoogle Scholar
  15. 15.
    De Stefano C, Sansone C, Vento M (2000) To reject or not to reject: that is the question—an answer in case of neural classifiers. IEEE Trans Syst Man Cybern C Appl Rev 30(1):574–585CrossRefGoogle Scholar
  16. 16.
    El-Yaniv R, Wiener Y (2010) On the foundations of noise-free selective classification. J Mach Learn Res 11:1605–1641MathSciNetzbMATHGoogle Scholar
  17. 17.
    Flexer A (2001) On the use of self-organizing maps for clustering and visualization. Intell Data Anal 5(5):373–384zbMATHGoogle Scholar
  18. 18.
    Fritzke B (1995) A growing neural gas network learns topologies. In: Tesauro G, Touretzky DS, Leen TK (eds) Advances in neural information processing systems vol 7. MIT Press, Cambridge, pp 625–632Google Scholar
  19. 19.
    Fu Y, Zhu X, Li B (2013) A survey on instance selection for active learning. Knowl Inf Syst 35(2):249–283CrossRefGoogle Scholar
  20. 20.
    Fumera G, Pillai I, Roli F (2003) Classification with reject option in text categorisation systems. In: Proceedings of the 12th international conference on image analysis and processing (ICIAP’2003). IEEE Computer Society, pp 582–587Google Scholar
  21. 21.
    Fumera G, Roli F (2002) Support vector machines with embedded reject option. In: Proceedings of the 1st international workshop on pattern recognition with support vector machines (SVM’2002). Springer, pp 68–82Google Scholar
  22. 22.
    Gama J, de Carvalho AC (2012) Machine learning. In: Machine learning: concepts, methodologies, tools and applications. IGI-Global, pp 13–22. doi: 10.4018/978-1-60960-818-7.
  23. 23.
    Eduardo Gasca A, Sergio Saldaña T, José S. Sánchez G, Valentín Velásquez G, Eréndira Rendón L, Itzel M. Abundez B, Rosa M. Valdovinos R, Rafael Cruz R (2011) A rejection option for the multilayer perceptron using hyperplanes. In: Proceedings of the 10th international conference on adaptive and natural computing algorithms (ICANNGA’2011), vol. LNCS 6593/2011. Springer, pp 51–60Google Scholar
  24. 24.
    Geebelen D, Suykens J, Vandewalle J (2012) Reducing the number of support vectors of SVM classifiers using the smoothed separable case approximation. IEEE Trans Neural Netw Learn Syst 23(4):682–688CrossRefGoogle Scholar
  25. 25.
    Giles D (2004) Calculating a standard error for the gini coefficient: some further results. Oxford Bull Econ Stat 66(3):124–126CrossRefGoogle Scholar
  26. 26.
    Gini C (1921) Measurement of inequality of incomes. Econ J 31(121):124–126CrossRefGoogle Scholar
  27. 27.
    Goldszmidt M, Cohen I, Fox A, Zhang S (2005) Three research challenges at the intersection of machine learning, statistical induction, and systems. In: Proceedings of the 10th conference on Hot Topics in Operating Systems (HOTOS’05), vol 10, pp 1–6Google Scholar
  28. 28.
    Guillen A, Herrera LJ, Rubio G, Pomares H, Lendasse A, Rojas I (2010) New method for instance or prototype selection using mutual information in time series prediction. Neurocomputing 73(10–12):2030–2038CrossRefGoogle Scholar
  29. 29.
    Han J, Gao J (2009) Research challenges for data mining in science and engineering. In: Kargupta H, Han J, Yu PS, Motwani R, Kumar V (eds) Next generation of data mining. Chapman & Hall, London, pp 1–18Google Scholar
  30. 30.
    Hasenjäger M, Ritter H (1998) Active learning with local models. Neural Process Lett 7(2):107–117CrossRefGoogle Scholar
  31. 31.
    Herbei R, Wegkamp MH (2006) Classification with reject option. Can J Stat 34(4):709–721MathSciNetCrossRefzbMATHGoogle Scholar
  32. 32.
    Holmström L, Hämäläinen A (1993) The self-organizing reduced kernel density estimator. In: Proceedings of the 1993 IEEE International Conference on Neural Networks (ICNN’93), pp 417–421Google Scholar
  33. 33.
    Ishibuchi H, Nii M (2000) Neural networks for soft decision making. Fuzzy Sets Syst 34(115):121–140CrossRefGoogle Scholar
  34. 34.
    Kohonen T (1982) Self-organized formation of topologically correct feature maps. Biol Cybern 43(1):59–69MathSciNetCrossRefzbMATHGoogle Scholar
  35. 35.
    Kohonen T (1988) An introduction to neural computing. Neural Netw 1(1):3–16CrossRefGoogle Scholar
  36. 36.
    Kohonen T (1988) The ’neural’ phonetic typewriter. Computer 21(3):11–22CrossRefGoogle Scholar
  37. 37.
    Kohonen T (1990) The self-organizing map. Proc IEEE 78(9):1464–1480CrossRefGoogle Scholar
  38. 38.
    Kohonen T (2001) Self-organizing maps, 3rd edn. Springer, Berlin, Heidelberg, New YorkCrossRefzbMATHGoogle Scholar
  39. 39.
    Kohonen T (2003) Learning vector quantization. In: Arbib MA (ed) The handbook of brain theory, neural networks, 2nd edn. MIT Press, Cambridge, pp 631–635Google Scholar
  40. 40.
    Lau KW, Yin H, Hubbard S (2006) Kernel self-organising maps for classification. Neurocomputing 69:2033–2040CrossRefGoogle Scholar
  41. 41.
    Lotte F, Mouchère H, Lécuyer A (2008) Pattern rejection strategies for the design of self-paced EEG-based brain–computer interfaces. In: Proceedings of the 19th international conference on pattern recognition (ICPR’2008), pp 1–5Google Scholar
  42. 42.
    Malone J, McGarry K, Wermter S, Bowerman C (2005) Data mining using rule extraction from Kohonen self-organising maps. Neural Comput Appl 15:9–17CrossRefGoogle Scholar
  43. 43.
    Mattos CLC, Barreto GA (2013) ARTIE and MUSCLE models: building ensemble classifiers from fuzzy ART and SOM networks. Neural Comput Appl 22(1):49–61CrossRefGoogle Scholar
  44. 44.
    Oliveira HP, Magalhaes A, Cardoso MJ, Cardoso JS (2010) An accurate and interpretable model for BCCT.core. In: Proceedings of the 32nd annual international conference of the IEEE engineering in medicine and biology Society, pp 6158–6161Google Scholar
  45. 45.
    Pedreira CE (2006) Learning vector quantization with training data selection. IEEE Trans Pattern Anal Mach Intell 28(1):157–162CrossRefGoogle Scholar
  46. 46.
    Peng H, Zhu S (2007) Handling of incomplete data sets using ICA and SOM in data mining. Neural Comput Appl 16(2):167–172CrossRefGoogle Scholar
  47. 47.
    Ritter H (1991) Asymptotic level density for a class of vector quantization processes. IEEE Trans Neural Netw 2(1):173–175MathSciNetCrossRefGoogle Scholar
  48. 48.
    Riveiro M, Johansson F, Falkman G, Ziemke T (2008) Supporting maritime situation awareness using self organizing maps and gaussian mixture models. In: Proceedings of the 2008 conference on 10th scandinavian conference on artificial intelligence (SCAI’08). IOS Press, pp 84–91Google Scholar
  49. 49.
    Rocha-Neto AR, Sousa R, Cardoso JS, Barreto GA (2011) Diagnostic of pathology on the vertebral column with embedded reject option. In: Proceedings of the 5th Iberian conference on pattern recognition and image analysis (IbPRIA’2011), vol. LNCS-6669, pp 588–595Google Scholar
  50. 50.
    Santos-Pereira CM, Pires AM (2005) On optimal reject rules and ROC curves. Pattern Recogn Lett 26(7):943–952CrossRefGoogle Scholar
  51. 51.
    Schleif FM, Villmann T, Hammer B, Schneider P (2011) Efficient kernelized prototype based classification. Int J Neural Syst 21(6):443–57CrossRefGoogle Scholar
  52. 52.
    Seo S, Obermayer K (2002) Soft learning vector quantization. Neural Comput 15:1589–1604CrossRefGoogle Scholar
  53. 53.
    Sim SF, Sági-Kiss V (2011) Multiple self-organising maps (mSOMs) for simultaneous classification and prediction: Illustrated by spoilage in apples using volatile organic profiles. Chemometr Intell Lab Syst 109(1):57–64CrossRefGoogle Scholar
  54. 54.
    Sousa R, Mora B, Cardoso JS (2009) An ordinal data method for the classification with reject option. In: Proceedings of the international conference on machine learning and applications (ICMLA’09), pp 746–750Google Scholar
  55. 55.
    Sousa R, Rocha Neto AR, Barreto GA, Cardoso JS, Coimbra MT (2014) Reject option paradigm for the reduction of support vectors. In: Proceedings of the 22th European symposium on artificial neural networks, computational intelligence and machine learning (ESANN’2014), pp 1–6Google Scholar
  56. 56.
    Souza Júnior AH, Barreto GA, Varela AT (2011) A speech recognition system for embedded applications using the SOM and TS-SOM networks. In: Mwasiagi JI (ed) Self-organizing maps—applications and novel algorithm design. InTech Open, Rijeka, pp 97–108. doi: 10.5772/14401
  57. 57.
    Suutala J, Pirttikangas S, Riekki J, Röning J (2004) Reject-optional LVQ-based two-level classifier to improve reliability in footstep identification. In: Ferscha A, Mattern F (eds) Pervasive computing. Springer, Berlin, Heidelberg, pp 182–187Google Scholar
  58. 58.
    Thomas LC, Edelman DB, Crook JN (2002) Credit scoring and its applications, 1st edn. SIAM, PhiladelphiaCrossRefzbMATHGoogle Scholar
  59. 59.
    Tortorella F (2005) A ROC-based reject rule for dichotomizers. Pattern Recognit Lett 26(2):167–180MathSciNetCrossRefGoogle Scholar
  60. 60.
    Turky AM, Ahmad MS (2010) The use of SOM for fingerprint classification. In: IEEE international conference on information retrieval and knowledge management (CAMP’2010), pp 287–290Google Scholar
  61. 61.
    Umer MF, Khiyal MSH (2007) Classification of textual documents using learning vector quantization. Inf Technol J 6:154–159CrossRefGoogle Scholar
  62. 62.
    Utsugi A (1998) Density estimation by mixture models with smoothing priors. Neural Comput 10:2115–2135CrossRefGoogle Scholar
  63. 63.
    van Hulle M (2012) Self-organizing maps. In: Rozenberg G, Baeck T, Kok J (eds) Handbook of natural computing: theory, experiments, and applications. Springer, Berlin, Heidelberg, pp 585–622CrossRefGoogle Scholar
  64. 64.
    Vasconcelos GC, Fairhurst MC, Bisset DL (1993) Enhanced reliability of multilayer perceptron networks through controlled pattern rejection. Electron Lett 29(3):261–263CrossRefGoogle Scholar
  65. 65.
    Vasconcelos GC, Fairhurst MC, Bisset DL (1995) Investigating feedforward neural networks with respect to the rejection of spurious patterns. Pattern Recognit Lett 16(2):207–212CrossRefGoogle Scholar
  66. 66.
    Villmann T, Haase S (2011) Divergence-based vector quantization. Neural Comput 23(5):1343–1392MathSciNetCrossRefzbMATHGoogle Scholar
  67. 67.
    Yin H (2008) The self-organizing maps: background, theories, extensions and applications. In: Fulcher J, Jain LC (eds) Computational intelligence: a compendium, studies in computational intelligence, vol 115. Springer, Berlin, Heidelberg, pp 715–762CrossRefGoogle Scholar
  68. 68.
    Yin H, Allinson NM (2001) Self-organizing mixture networks for probability density estimation. IEEE Trans Neural Netw 12(2):405–411CrossRefGoogle Scholar
  69. 69.
    Zidelmal Z, Amirou A, Belouchrani A (2012) Heartbeat classification using support vector machines (SVMs) with an embedded reject option. Int J Pattern Recognit Artif Intell 26(1):1250,001-1–1250,001-17MathSciNetCrossRefGoogle Scholar

Copyright information

© The Natural Computing Applications Forum 2015

Authors and Affiliations

  • Ricardo Gamelas Sousa
    • 1
    • 2
  • Ajalmar R. Rocha Neto
    • 3
  • Jaime S. Cardoso
    • 4
  • Guilherme A. Barreto
    • 5
  1. 1.Instituto de Investigação e Inovação em SaúdeUniversidade do PortoPortoPortugal
  2. 2.INEB – Instituto de Engenharia BiomédicaUniversidade do PortoPortoPortugal
  3. 3.Departamento de TelemáticaInstituto Federal do Ceará (IFCE)FortalezaBrasil
  4. 4.INESC TEC and Faculdade de Engenharia da Universidade do PortoPortoPortugal
  5. 5.Departamento de Engenharia de TeleinformáticaUniversidade Federal do Ceará (UFC)FortalezaBrasil

Personalised recommendations