A Fast SVM Training Algorithm Based on a Decision Tree Data Filter

  • Jair Cervantes
  • Asdrúbal López
  • Farid García
  • Adrián Trueba
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7094)

Abstract

In this paper we present a new algorithm to speed up the training time of Support Vector Machines (SVM). SVM has some important properties like solid mathematical background and a better generalization capability than other machines like for example neural networks. On the other hand, the major drawback of SVM occurs in its training phase, which is computationally expensive and highly dependent on the size of input data set. The proposed algorithm uses a data filter to reduce the input data set to train a SVM. The data filter is based on an induction tree which effectively reduces the training data set for SVM, producing a very fast and high accuracy algorithm. According to the results, the algorithm produces results in a faster way than existing SVM implementations (SMO, LIBSVM and Simple-SVM) with similar accurateness.

Keywords

Support Vector Machine Training Time Quadratic Programming Problem Support Vector Machine Algorithm Sequential Minimal Optimization 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Vapnik, V.: Statistical Learning Theory. Wiley, New York (1998)MATHGoogle Scholar
  2. 2.
    Platt, J.C.: Fast Training of Support Vector Machines Using Sequential Minimal Optimization. In: Scholkopf, B., Burges, C.J.C., Smola, A.J. (eds.) Advances in Kernel Methods-Support Vector Learning, pp. 185–208. MIT Press (1998)Google Scholar
  3. 3.
    Cervantes, J., Li, X., Yu, W., Li, K.: Support vector machine classification for large data sets via minimum enclosing ball clustering. Neurocomputing 71(4-6), 611–619 (2008)CrossRefGoogle Scholar
  4. 4.
    Yu, H., Yang, J., Han, J.: Classifying Large Data Sets Using SVM with Hierarchical Clusters. In: Proc. of the ACM SIGKDD Intl. Conf. on Knowledge, pp. 306–315 (2003)Google Scholar
  5. 5.
    Dong, J., Krzyzak, A., Suen, C.Y.: Fast SVM Training Algorithm with Decomposition on Very Large Data Sets. IEEE Trans. Pattern Anal. Mach. Intell. 27(4), 603–618 (2005)CrossRefGoogle Scholar
  6. 6.
    López, J., Barbero, Á., Dorronsoro, J.R.: Simple Clipping Algorithms for Reduced Convex Hull SVM Training. In: Corchado, E., Abraham, A., Pedrycz, W. (eds.) HAIS 2008. LNCS (LNAI), vol. 5271, pp. 369–377. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  7. 7.
    Wu, C., Wang, X., Bai, D., Zhang, H.: Fast SVM incremental learning based on the convex hulls algorithm. In: Proc. of the Intl. Conf. on Computational Intelligence and Security, pp. 249–252. IEEE Computer Society (2008)Google Scholar
  8. 8.
    Mavroforakis, M.E., Sdralis, M., Heodoridis, S.: A geometric nearest point algorithm for the efficient solution of the SVM classification task. IEEE Trans. Neural Networks 18, 1545–1549 (2007)CrossRefGoogle Scholar
  9. 9.
    Bottou, L., Lin, C.J.: Support Vector Machine Solvers. In: Bottou, L., Chapelle, O., DeCoste, D., Weston, J. (eds.) Large Scale Kernel Machines, pp. 1–28. MIT Press (2007)Google Scholar
  10. 10.
    Chambers, R.L., Skinner, C.J (eds.): Analysis of Survey Data. Wiley (2003) ISBN 0-471-89987-9Google Scholar
  11. 11.
    Schohn, G., Cohn, D.: Less is more: Active learning with support vector machines. In: Proc. Intl. Conf. on Machine Learning, pp. 839–846 (2000)Google Scholar
  12. 12.
    Burges, C.J.: A Tutorial on Support Vector Machines for Pattern Recognition. Data Min. Knowl. Discov. 2(2), 121–167 (1998)CrossRefGoogle Scholar
  13. 13.
    Quinlan, J.R.: Improved use of continuous attributes in c4.5. J. Artificial Intell. Res. 4, 77–90 (1996)MATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Jair Cervantes
    • 1
  • Asdrúbal López
    • 2
  • Farid García
    • 3
  • Adrián Trueba
    • 1
  1. 1.UAEM-TexcocoAutonomous University of Mexico StateMéxico
  2. 2.Instituto Politécnico Nacional 2508Center of Research and Advanced Studies-IPNMéxico
  3. 3.Autonomous University of Hidalgo StateTizayuca-HidalgoMéxico

Personalised recommendations