A Fast Computation of Inter-class Overlap Measures Using Prototype Reduction Schemes

  • Sang-Woon Kim
  • B. John Oommen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5032)


In most Pattern Recognition (PR) applications, it is advantageous if the accuracy (or error rate) of the classifier can be evaluated or bounded prior to testing it in a real-life setting. It is also well known that if the two class-conditional distributions have a large overlapping volume, the classification accuracy is poor. This is because, if we intend to use the classification accuracy as a criterion for evaluating a PR system, the points within the overlapping volume tend to have less significance in determining the prototypes. Unfortunately, the computation of the indices which quantify the overlapping volume is expensive. In this vein, we propose a strategy of using a Prototype Reduction Scheme (PRS) to approximately compute the latter. In this paper, we show that by completely discarding the points not included by the PRS, we can obtain a reduced set of sample points, using which, in turn, the measures for the overlapping volume can be computed. The value of the corresponding figures is comparable to those obtained with the original training set (i.e., the one which considers all the data points) even though the computations required to obtain the prototypes and the corresponding measures are significantly less. The proposed method has been rigorously tested on artificial and real-life data sets, and the results obtained are, in our opinion, quite impressive - sometimes faster by two orders of magnitude.


Prototype Reduction Schemes (PRS) k-Nearest Neighbor (k −NN) Classifier Data Complexity Class-Overlapping 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Jain, A.K., Duin, R.P.W., Mao, J.: Statistical pattern recognition: A review. IEEE Trans. Pattern Anal. and Machine Intell. PAMI-22(1), 4–37 (2000)CrossRefGoogle Scholar
  2. 2.
    Batista, G.E., Prati, R.C., Monard, M.C.: Balancing Strategies and Class Overlapping. In: Famili, A.F., Kok, J.N., Peña, J.M., Siebes, A., Feelders, A. (eds.) IDA 2005. LNCS, vol. 3646, pp. 24–35. Springer, Heidelberg (2005)Google Scholar
  3. 3.
    Bezdek, J.C., Kuncheva, L.I.: Nearest prototype classifier designs: An experimental study. International Journal of Intelligent Systems 16(12), 1445–1473 (2001)zbMATHCrossRefGoogle Scholar
  4. 4.
    Burges, C.J.C.: A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery 2(2), 121–167 (1998)CrossRefGoogle Scholar
  5. 5.
    Chang, C.L.: Finding prototypes for nearest neighbor classifiers. IEEE Trans. Computers C-23(11), 1179–1184 (1974)CrossRefGoogle Scholar
  6. 6.
    Dasarathy, B.V.: Nearest Neighbor (NN) Norms: NN Pattern Classification Techniques. IEEE Computer Society Press, Los Alamitos (1991)Google Scholar
  7. 7.
    Devijver, P.A., Kittler, J.: On the edited nearest neighbor rule. In: Proc. 5th Int. Conf. on Pattern Recognition, December 1980, pp. 72–80 (1980)Google Scholar
  8. 8.
    Fukunaga, K.: Introduction to Statistical Pattern Recognition, 2nd edn. Academic Press, San Diego (1990)zbMATHGoogle Scholar
  9. 9.
    Fukunaga, K., Mantock, J.M.: Nonparametric data reduction. IEEE Trans. Pattern Anal. and Machine Intell. PAMI-6(1), 115–118 (1984)CrossRefGoogle Scholar
  10. 10.
    Hart, P.E.: The condensed nearest neighbor rule. IEEE Trans. Inform. Theory IT-14, 515–516 (1968)CrossRefGoogle Scholar
  11. 11.
    Ho, T.K., Basu, M.: Complexity Measures of Supervised Classification Problems. IEEE Trans. Pattern Anal. and Machine Intell. PAMI-24(3), 289–300 (2002)Google Scholar
  12. 12.
    Hoekstra, A., Duin, R.P.W.: On the nonlinearity of pattern classifiers. In: 13th International Conference on Pattern Recognition (ICPR 1996), pp. 271–275 (1996)Google Scholar
  13. 13.
    Kim, S.-W., Oommen, B.J.: Enhancing prototype reduction schemes with LVQ3-type algorithms. Pattern Recognition 36(5), 1083–1093 (2003)zbMATHCrossRefGoogle Scholar
  14. 14.
    Kim, S.-W., Oommen, B.J.: A Brief Taxonomy and Ranking of Creative Prototype Reduction Schemes. Pattern Analysis and Applications Journal 6(3), 232–244 (2003)CrossRefMathSciNetGoogle Scholar
  15. 15.
    Kim, S.-W., Oommen, B.J.: Enhancing Prototype Reduction Schemes with Recursion: A Method Applicable for “Large” Data Sets. IEEE Trans. Systems, Man, and Cybernetics - Part B SMC-34(3), 1384–1397 (2004)CrossRefGoogle Scholar
  16. 16.
    Kim, S.-W., Oommen, B.J.: On using prototype reduction schemes to optimize kernel-based nonlinear subspace methods. Pattern Recognition 37(2), 227–239 (2004)zbMATHCrossRefGoogle Scholar
  17. 17.
    Kim, S.-W., Oommen, B.J.: On using prototype reduction schemes and classifier fusion strategies to optimize kernel-based nonlinear subspace methods. IEEE Transactions on Pattern Analysis and Machine Intelligence 27(3), 455–460 (2005)CrossRefGoogle Scholar
  18. 18.
    Mansilla, E.B., Ho, T.K.: On classifier domains of competence. In: 17th International Conference on Pattern Recognition (ICPR 2004), pp. 136–139 (2004)Google Scholar
  19. 19.
  20. 20.
    Mollineda, R.A., Sanchez, J.S., Sotoca, J.M.: Data Characterization for Effective Prototype Selection. In: Marques, J.S., Pérez de la Blanca, N., Pina, P. (eds.) IbPRIA 2005. LNCS, vol. 3523, pp. 27–34. Springer, Heidelberg (2005)Google Scholar
  21. 21.
    Ritter, G.L., Woodruff, H.B., Lowry, S.R., Isenhour, T.L.: An algorithm for a selective nearest neighbor rule. IEEE Trans. Inform. Theory IT-21, 665–669 (1975)CrossRefGoogle Scholar
  22. 22.
    Singh, S.: PRISM: A novel framework for pattern recognition. Pattern Analysis and Applications 6, 134–149 (2003)CrossRefGoogle Scholar
  23. 23.
    Sohn, S.-Y.: Meta analysis of classification algorithms for pattern recognition. IEEE Trans. Pattern Anal. and Machine Intell. PAMI-21(11), 1137–1144 (1999)CrossRefGoogle Scholar
  24. 24.
    Sotoca, J.M., Mollineda, R.A., Sanchez, J.S.: A meta-learning framework for pattern classification by means of data complexity measures. Revista Iberoamericana de Inteligencia Artificial 10(29), 31–38 (2006)Google Scholar
  25. 25.
    Gates, G.W.: The reduced nearest neighbor rule. IEEE Trans. Inform. Theory IT-18, 431–433 (1972)CrossRefGoogle Scholar
  26. 26.
    Tomek, I.: Two modifcations of CNN. IEEE Trans. Syst. Man and Cybern. SMC-6(6), 769–772 (1976)Google Scholar
  27. 27.
    Xie, Q., Laszlo, C.A., Ward, R.K.: Vector quantization techniques for nonparametric classifier design. IEEE Trans. Pattern Anal. and Machine Intell. PAMI-15(12), 1326–1330 (1993)Google Scholar
  28. 28.
    Kim, S.-W., Oommen, B.J.: On using prototype reduction schemes to enhance the computation of volume-based inter-class overlap measures (unabridged version of this paper)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Sang-Woon Kim
    • 1
  • B. John Oommen
    • 2
  1. 1.Dept. of Computer Science and EngineeringMyongji UniversityYonginSouth Korea
  2. 2.School of Computer ScienceCarleton UniversityOttawaCanada

Personalised recommendations