Advertisement

Multi-threaded Support Vector Machines for Pattern Recognition

  • João Gonçalves
  • Noel Lopes
  • Bernardete Ribeiro
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7664)

Abstract

Support Vector Machines (SVM) have become indispensable tools in the area of pattern recognition. They show powerful classification and regression performance in highly non-linear problems by mapping the input vectors nonlinearly into a high-dimensional feature space through a kernel function. However, the optimization task is numerically expensive since single-threaded implementations are hardly able to cope up with the complex learning task. In this paper, we present a multi-threaded implementation of the Sequential Minimal Optimization (SMO) which reduces the numerical complexity by parallelizing the KKT conditions update, the calculation of the hyperplane offset and the classification task. Our preliminary results both in benchmark datasets and real-world problems show competitive performance to the state-of-the-art tools while the execution running times are considerably faster.

Keywords

SVM OpenMP sequential minimal optimization (SMO) 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Ando, R., Zhang, T.: A framework for learning predictive structures from multiple tasks and unlabeled data. J. of Machine Learning Research 6, 1817–1853 (2005)MathSciNetzbMATHGoogle Scholar
  2. 2.
    Asuncion, A., Newman, D.J.: UCI machine learning repository (2010), http://archive.ics.uci.edu/ml/index.html
  3. 3.
    Ayat, N., Cheriet, M., Suen, C.: KMOD - a two-parameter SVM kernel for pattern recognition. In: Proc. of ICPR, pp. 30331–30334 (2002)Google Scholar
  4. 4.
    Cao, L., Keerthi, S., Ong, C.J., Zhang, J., Periyathamby, U., Fu, X.J., Lee, H.: Parallel sequential minimal optimization for the training of support vector machines. IEEE Transactions on Neural Networks 17(4), 1039–1049 (2006)CrossRefGoogle Scholar
  5. 5.
    Catanzaro, B., Sundaram, N., Keutzer, K.: Fast support vector machine training and classification on graphics processors. In: Proc. of the 25th Int. Conf. on Machine Learning, ICML 2008, pp. 104–111. ACM, NY (2008)CrossRefGoogle Scholar
  6. 6.
    Chang, C.C., Lin, C.J.: LIBSVM: A Library for Support Vector Machines. Dep. of Computer Science National Taiwan University, Taipei, Taiwan (2011)Google Scholar
  7. 7.
    Chen, B., Zhao, S., Zhu, P., Príncipe, J.C.: Quantized kernel least mean square algorithm. IEEE Trans. Neural Netw. Learning Syst. 23(1), 22–32 (2012)CrossRefGoogle Scholar
  8. 8.
    Cortes, C., Vapnik, V.: Support-vector networks. In: Machine Learning, pp. 273–297 (1995)Google Scholar
  9. 9.
    De Brabanter, K., De Brabanter, J., Suykens, J., De Moor, B.: Optimized fixed-size kernel models for large data sets. Comput. Stat. Data Anal. 54(6), 1484–1504 (2010)CrossRefGoogle Scholar
  10. 10.
    Hoegaerts, L., Suykens, J.A.K., Vandewalle, J., De Moor, B.: Subset based least squares subspace regression in RKHS. Neurocomput. 63, 293–323 (2005)CrossRefGoogle Scholar
  11. 11.
    Keerthi, S., Shevade, S., Bhattacharyya, C., Murthy, K.: Improvements to Platt’s SMO algorithm for SVM classifier design. Neural Comput. 13(3), 637–649 (2001)zbMATHCrossRefGoogle Scholar
  12. 12.
    Lopes, N., Correia, D., Pereira, C., Ribeiro, B., Dourado, A.: An Incremental Hypersphere Learning Framework for Protein Membership Prediction. In: Corchado, E., Snášel, V., Abraham, A., Woźniak, M., Graña, M., Cho, S.-B. (eds.) HAIS 2012, Part III. LNCS, vol. 7208, pp. 429–439. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  13. 13.
    Platt, J.: Sequential minimal optimization: A fast algorithm for training support vector machines, vol. 208, pp. 1–21. MIT Press (1998)Google Scholar
  14. 14.
    Qiao, M., Sung, A.H., Liu, Q.: Feature mining and intelligent computing for MP3 steganalysis. In: Int. Joint Conf. on Bioinformatics, Systems Biology and Intelligent Computing, pp. 627–630. IEEE Computer Society (2009)Google Scholar
  15. 15.
    Vapnik, V.: The nature of statistical learning theory. Springer (1995)Google Scholar
  16. 16.
    Wu, W.W.: Beyond business failure prediction. Expert Systems with Applications 37, 2371–2376 (2010)CrossRefGoogle Scholar
  17. 17.
    Zhang, R., Wang, W.: Facilitating the applications of support vector machine by using a new kernel. Expert Systems with Applications 38, 14225–14230 (2011)Google Scholar
  18. 18.
    Zien, A., Rätsch, G., Mika, S., Schölkopf, B., Lemmen, C., Smola, A., Lengauer, T., Mueller, K.R.: Engineering support vector machine kernel that recognize translation initiation sites. Bioinformatics 16(9), 799–807 (2000)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • João Gonçalves
    • 1
  • Noel Lopes
    • 1
    • 2
  • Bernardete Ribeiro
    • 1
  1. 1.CISUC, Department of Informatics EngineeringUniversity of CoimbraPortugal
  2. 2.UDI/IPG - Research UnitPolytechnic Institute of GuardaPortugal

Personalised recommendations