Pattern Recognition and Image Analysis

, Volume 20, Issue 4, pp 427–437 | Cite as

Prototype sample selection based on minimization of the complete cross validation functional

  • M. N. Ivanov
Mathematical Methods in Pattern Recognition


A method of prototype sample selection from a training set for a classifier of K nearest neighbors (KNN), based on minimization of the complete cross validation functional, is proposed. The optimization leads to reduction of the training set to the minimum sufficient number of prototypes, removal (censoring) of noise samples, and improvement of the generalization ability, simultaneously.


prototype learning k nearest neighbors generalization ability exact generalization bound complete cross validation leave-one-out similarity search 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    I. A. Borisova, V. V. Dyubanov, N. G. Zagoruiko, and O. A. Kutenko, “Similarity and Compactness,” in Proc. All-Russian Conf. Mathematical Methods for Pattern Recognition MMPR-14 (Maks Press, Moscow, 2009).Google Scholar
  2. 2.
    S. Bermejo and J. Cabestany, “Learning with Nearest Neighbour Classifiers,” Neural Process. Lett. 13, No. 2, 159–181 (2001).zbMATHCrossRefGoogle Scholar
  3. 3.
    S. Brin, “Near Neighbor Search in Large Metric Spaces,” in Proc. 21th Int. Conf. on Very Large Data Bases (Zurich, 1995), pp. 574–584.Google Scholar
  4. 4.
    C. J. C. Burges, “A Tutorial on Support Vector Machines for Pattern Recognition,” Data Min. Knowl. Discovery 2, No. 2, 121–167 (1998).CrossRefGoogle Scholar
  5. 5.
    E. Chávez, G. Navarro, R. Baeza-Yates, and J. L. Marroqün, “Searching in Metric Spaces,” ACM Comput. Surv. 33, No. 3, 273–321 (2001).CrossRefGoogle Scholar
  6. 6.
    C. Cortes and V. Vapnik, “Support-Vector Networks,” Mach. Learn. 20, No. 3, 273–297 (1995).zbMATHGoogle Scholar
  7. 7.
    F. Chang, C.-C. Lin, and C.-J. Lu, “Adaptive Phototype Learning Algorithms: Theoretical and Experimental Studies,” J. Mach. Learn. Res., No. 7, 2125–2148 (2006).Google Scholar
  8. 8.
    M. N. Ivanov and K. V. Vorontsov, “Standards Selection Based on Total Follow Control Minimization,” in Proc. All-Russian Conf. Mathematical Methods for Pattern Recognition MMPR-14 (Maks Press, Moscow, 2009).Google Scholar
  9. 9.
    P. E. Hart, “The Condensed Nearest Neighbor Rule,” IEEE Trans. Inf. Theory, No. 14, 515–516 (1968).Google Scholar
  10. 10.
    R. Kohavi, “A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection,” in Proc. 14th Int. Joint Conf. on Artificial Intelligence (Quebec, 1995), pp. 1137–1145.Google Scholar
  11. 11.
    C.-L. Liu and M. Nakagawa, “Evaluation of Phototype Learning Algorithms for Nearest-Neighbor Classifier in Application to Handwritten Character Recognition,” Pattern Recognit. 34, No. 3, 601–615 (2001).zbMATHCrossRefGoogle Scholar
  12. 12.
    M. Mullin and R. Shukhankar, “Complete Cross-Validation for Nearest Neighbor Classifiers,” in Proc. Int. Conf. of Machine Learning (Stanford Univ., 2000), pp. 639–646.Google Scholar
  13. 13.
    M. Tipping, “The Relevance Vector Machine,” in Advances in Neural Information Processing Systems (Morgan Kaufmann, San Mateo, CA, 2000).Google Scholar
  14. 14.
    K. V. Vorontsov, “Combinatorial Approach to Quality Estimation of Tutorial Algorithms,” in Mathematical Problems of Cybernetics (Fizmatlit, Moscow, 2004), Vol. 13, pp. 5–36 [in Russian].Google Scholar
  15. 15.
    K. V. Vorontsov and A. V. Koloskov, “Compactness Profiles and Reference Objects Selection in Metric Classification Algorithms,” in Artificial Intelligence (Donetsk, 2006), pp. 30–33 [in Russian].Google Scholar
  16. 16.
    D. Wilson, “Asymptotic Properties of Nearest Neighbor Rules Using Edited Data,” IEEE Trans. Syst., Man, Cybernet., No. 2, 408–421 (1972).Google Scholar
  17. 17.
    N. G. Zagoruiko, Applied Methods for Data and Knowledge Analysis (IM SO RAN, Novosibirsk, 1999) [in Russian].Google Scholar

Copyright information

© Pleiades Publishing, Ltd. 2010

Authors and Affiliations

  1. 1.Moscow Institute of Physics and TechnologyDolgoprudnyi, Moscow oblastRussia

Personalised recommendations