Support Vector Based Prototype Selection Method for Nearest Neighbor Rules

  • Yuangui Li
  • Zhonghui Hu
  • Yunze Cai
  • Weidong Zhang
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3610)

Abstract

The Support vector machines derive the class decision hyper planes from a few, selected prototypes, the support vectors (SVs) according to the principle of structure risk minimization, so they have good generalization ability. We proposed a new prototype selection method based on support vectors for nearest neighbor rules. It selects prototypes only from support vectors. During classification, for unknown example, it can be classified into the same class as the nearest neighbor in feature space among all the prototypes. Computational results show that our method can obtain higher reduction rate and accuracy than popular condensing or editing instance reduction method.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Cover, T.M., Hart, P.E.: Nearest Neighbor Pattern Classification. IEEE Transaction on Information Theory 13(1), 21–27 (1967)MATHCrossRefGoogle Scholar
  2. 2.
    Wilson, D.R., Martinez, T.R.: Reduction Techniques for Instance-based Learning Algorithm. Machine Learning 38(3), 257–286 (2000)MATHCrossRefGoogle Scholar
  3. 3.
    Toussaint, G.: Proximity Graphs for Nearest Neighbor Decision Rules: Recent Progress. In: Proceedings of INTERFACE-2002, 34th Symposium on Computing and Statistics, Ritz-Carlton Hotel, Montreal, Canada, pp. 83–106 (2002)Google Scholar
  4. 4.
    Aha, D.W., Kibler, D., Albert, M.K.: Instance-based Learning Algorithms. Machine Learning 6(19), 37–66 (1991)Google Scholar
  5. 5.
    Dasarathy, B.V.: Minimal Consistent Set (MCS) Identification for Optimal Nearest Neighbor Decision Systems Design. IEEE transaction on System, Man, and Cybernetics 24(3), 511–517 (1994)CrossRefGoogle Scholar
  6. 6.
    Kuncheva, L.I., Bezdek, J.C.: Nearest Prototype Classification: Clustering, Genetic Algorithms, or Random Search. IEEE transactions on System, Man and Cybernetics 28, 160–164 (1998)Google Scholar
  7. 7.
    Dasarathy, B.V., Sánchez, J.S., Townsend, S.: Nearest Neighbor Editing and Condensing Tools-Synergy Exploitation. Pattern Analysis & Application 3, 19–30 (2000)CrossRefGoogle Scholar
  8. 8.
    Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer, New York (1995)MATHGoogle Scholar
  9. 9.
    Vapnik, V.N.: Estimation of Dependences Based on Empirical Data. Springer, Berlin (1982)MATHGoogle Scholar
  10. 10.
    Vapnik, V.N., Chervonenkis, A.: Theory of Pattern Recognition. Nauka, Moscow (1974)Google Scholar
  11. 11.
    Kecman, V.: Learning and Soft Computing, Support Vector machines, Neural Networks and Fuzzy Logic Models. The MIT Press, Cambridge (2001)MATHGoogle Scholar
  12. 12.
    Wang, L.P. (ed.): Support Vector Machines: Theory and Application. Springer, Heidelberg (2005)Google Scholar
  13. 13.
    Burges, C.J.C.: Simplified Support Vector Decision Rules. In: The 13th international conference on Machine Learning, pp. 71–77 (1996)Google Scholar
  14. 14.
    Burges, C.J.C., Schölkopf, B.: Improving the Accuracy and Speed of Support Vector Machines. In: Mozer, M., Jordan, M., Petsche, T. (eds.) Neural Information Processing Systems, vol. 9. MIT Press, Cambridge (1997)Google Scholar
  15. 15.
    Vishwanathan, S.V.N., Murthy, M.N.: Use of Multi-category Proximal SVM for Data Set Reduction. In: Ajith, A., Köppen, M. (eds.) Hybrid Information Systems. First International Workshop on Hybrid Intelligent Systems, Adelaide, Australia, December 11-12, pp. 19–24 (2001)Google Scholar
  16. 16.
    LeCun, Y., Jackel, L., Bottou, L., Brunot, A., Cortes, C., Denker, J., Drucker, H., Guyon, I., Müller, U., Säckinger, E., Simard, P., Vapnik, V.: Comparison of Learning Algorithms for Handwritten Digit Recognition. In: Fogelman, F., Gallinari, P. (eds.) Proc. International Conference on Artificial Neural Networks, pp. 53–60 (1995)Google Scholar
  17. 17.
    Burges, C.J.C.: A Tutorial on Support Vector Machines for Pattern Recognition. Data Mining and Knowledge Discovery 2(2), 121–167 (1998)CrossRefGoogle Scholar
  18. 18.
    Keerthi, S.S., Shevade, S.K., Bhattacharyya, C., Murthy, K.R.K.: Improvements to Platt’s SMO Algorithm for SVM Classifier Design. Neural Computation 13, 37–649 (2001)CrossRefGoogle Scholar
  19. 19.
    Blake, C., Keogh, E., Merz, C.J.: UCI Repository of machine learning databases. Department of Information and Computer Science. University of California, Irvine (1998), http://www.ics.uci.edu/~mlearn/MLRepository.html Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Yuangui Li
    • 1
  • Zhonghui Hu
    • 1
  • Yunze Cai
    • 1
  • Weidong Zhang
    • 1
  1. 1.Department of AutomationShanghai Jiaotong UniversityXuhui, ShanghaiP. R. China

Personalised recommendations