In this paper, we proposed a method to speed up the test phase of SVM based on Feature Vector Selection method (FVS). In the method, the support vectors (SVs) appeared in the decision function of SVM are replaced with some feature vectors (FVs) which are selected from support vectors by FVS method. Since it is a subset of SVs set, the size of FVs set is normally smaller than that of the SVs set, therefore the decision process of SVM is speeded up. Experiments on 12 datasets of IDA show that the number of SVs can be reduced from 20% to 99% with only a slight increase on the error rate of SVM by the proposed method. The trade-off between the generalization ability of obtained SVM and the speedup ability of the proposed method can be easily controlled by one parameter.


Support Vector Machine Support Vector Decision Function Kernel Matrix Relevance Vector Machine 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Vapnik, V.N.: The Natural of Statistical Learning Theory. Springer, New York (1995)Google Scholar
  2. 2.
    Burges, C.J.C.: Simplified support vector decision rules. In: Proc. 13th International Conference on Machine Learning, San Mateo, CA, pp. 71–77 (1996)Google Scholar
  3. 3.
    Burges, C.J.C.: A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery 2, 121–167 (1998)CrossRefGoogle Scholar
  4. 4.
    Burges, C.J.C., Schoelkopf, B.: Improving the accuracy and speed of support vector learning machines. In: Mozer, M., Jordan, M., Petsche, T. (eds.) Advances in neural information processing systems, vol. 9, pp. 375–381. MIT Press, Cambridge (1997)Google Scholar
  5. 5.
    Zhu, Y.S., Zhang, Y.Y.: A new type SVM—Projected SVM. Science in China G: Physics, Mechanics & Astronomy supp. 47, 21–28 (2004)CrossRefGoogle Scholar
  6. 6.
    Osuna, E., Girosi, F.: Reducing the run-time complexity of Support Vector Machines. In: Proc.14th International Conference on Pattern Recognition, Btisbane, Austrilia (1998)Google Scholar
  7. 7.
    Tipping, M.E.: Sparse Bayesian Learning and the Relevance Vector Machine. Journal of Machine Learning Research 1, 211–244 (2001)MATHCrossRefMathSciNetGoogle Scholar
  8. 8.
    Lee, Y.-J., Mangasarian, O.L.: RSVM: reduced support vector machines. In: Proc. 1st SIAM Int. Conf. Data Mining, Chicago (2001)Google Scholar
  9. 9.
    Lin, K.-M.: A Study on Reduced Support Vector Machines. IEEE Trans. Neural Networks 14, 1449–1459 (2003)CrossRefGoogle Scholar
  10. 10.
    Schoelkopf, B., Mika, S., Burges, C.J.C., Knirsch, P., Muller, K., Ratsch, G., Smola, A.J.: Input space versus feature space in kernel-based methods. IEEE Trans. Neural Networks 10, 1000–1017 (1999)CrossRefGoogle Scholar
  11. 11.
    Nguyen, D.D., Ho, T.B.: An Efficient Method for Simplifying Support Vector Machines. In: The 22nd International Conference on Machine Learning (ICML 2005), Bonn, Germany (2005)Google Scholar
  12. 12.
    Downs, T., Gates, K.E., Masters, A.: Extract simplification of support vector solutions. Journal of Machine Learning Research 2, 293–297 (2001)CrossRefGoogle Scholar
  13. 13.
    Suykens, J.A.K., Lukas, L., Vandewalle, J.: Sparese approximation using least squares support vector machines. In: Proc. of the IEEE International Symposium on Circuits and Systems, Geneva, Switzerland, pp. II757–II760 (2000)Google Scholar
  14. 14.
    Baudat, G., Anouar, F.: Feature Vector selection and projection using kernels. Neuro-computing 55, 21–38 (2003)Google Scholar
  15. 15.
  16. 16.
    Zhu, Y.S., Li, C.H., Zhang, Y.Y.: A practical parameters selection method for SVM. In: Yin, F.-L., Wang, J., Guo, C. (eds.) ISNN 2004. LNCS, vol. 3173, pp. 518–523. Springer, Heidelberg (2004)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Yongsheng Zhu
    • 1
  • Junyan Yang
    • 1
  • Jian Ye
    • 2
  • Youyun Zhang
    • 1
  1. 1.Key Laboratory of Education Ministry for Modern Design and Rotor-Bearing SystemXi’an Jiaotong UniversityXi’anChina
  2. 2.Network & Information Technology Center of LibraryXi’an Jiaotong UniversityXi’anChina

Personalised recommendations