Extreme Support Vector Regression

  • Wentao Zhu
  • Jun Miao
  • Laiyun Qing
Part of the Adaptation, Learning, and Optimization book series (ALO, volume 16)


Extreme Support Vector Machine (ESVM), a variant of ELM, is a nonlinear SVM algorithm based on regularized least squares optimization. In this chapter, a regression algorithm, Extreme Support Vector Regression (ESVR), is proposed based on ESVM. Experiments show that, ESVR has a better generalization ability than the traditional ELM. Furthermore, ESVM can reach comparable accuracy as SVR and LS-SVR, but has much faster learning speed.


Extreme learning machine Support vector regression  Extreme support vector machine Extreme support vector regression Regression 



The authors would like to thank Mr. Zhiguo Ma and Mr. Fuqiang Chen for their valuable comments. This research is partially sponsored by National Basic Research Program of China (No. 2009CB320900), and Natural Science Foundation of China (Nos. 61070116, 61070149, 61001108, 61175115, and 61272320), Beijing Natural Science Foundation (No. 4102013), President Fund of Graduate University of Chinese Academy of Sciences (No.Y35101CY00), and Importation and Development of High-Caliber Talents Project of Beijing Municipal Institutions.


  1. 1.
    G.-B. Huang, D.H. Wang, Y. Lan, Extreme learning machines: a survey. Int. J. Mach. Learn. Cybernet. 2(2), 107–122 (2011)CrossRefGoogle Scholar
  2. 2.
    G.-B. Huang, Q.-Y. Zhu, C.-K. Siew, Extreme learning machine: a new learning scheme of feedforward neural networks, in Proceedings. 2004 IEEE International Joint Conference on Neural Networks, 2004, vol. 2, pp. 985–990, IEEE, 2004Google Scholar
  3. 3.
    G.-B. Huang, Q.-Y. Zhu, C.-K. Siew, Extreme learning machine: theory and applications. Neurocomputing. 70(1), 489–501 (2006)Google Scholar
  4. 4.
    W. Zong, H. Zhou, G.-B. Huang, Z. Lin, Face recognition based on kernelized extreme learning machine, in Autonomous and Intelligent Systems (Springer, Berlin, 2011) pp. 263–272Google Scholar
  5. 5.
    H.-J. Rong, Y.-S. Ong, A.-H. Tan, Z. Zhu, A fast pruned-extreme learning machine for classification problem. Neurocomputing. 72(1), 359–366 (2008)Google Scholar
  6. 6.
    M. van Heeswijk, Y. Miche, T. Lindh-Knuutila, P.A. Hilbers, T. Honkela, E. Oja, A. Lendasse, Adaptive ensemble models of extreme learning machines for time series prediction, in Artificial Neural Networks—ICANN 2009 (Springer, Berlin, 2009) pp. 305–314Google Scholar
  7. 7.
    G.-B. Huang, L. Chen, Convex incremental extreme learning machine. Neurocomputing. 70(16), 3056–3062 (2007)Google Scholar
  8. 8.
    Q. He, T. Shang, F. Zhuang, Z. Shi, Parallel extreme learning machine for regression based on mapreduce. Neurocomputing. 102, 52–58 (2013)Google Scholar
  9. 9.
    Q. Liu, Q. He, Z. Shi, Extreme support vector machine classifier, in Advances in Knowledge Discovery and Data Mining (Springer, Berlin, 2008) pp. 222–233Google Scholar
  10. 10.
    Q. He, C. Du, Q. Wang, F. Zhuang, Z. Shi, A parallel incremental extreme svm classifier. Neurocomputing. 74(16), 2532–2540 (2011)Google Scholar
  11. 11.
    B. Frénay, M. Verleysen, Using SVMs with randomised feature spaces: an extreme learning approach, in Proceedings of the 18th European symposium on Artificial Neural Networks (ESANN), Bruges, Belgium, pp. 28–30 (2010)Google Scholar
  12. 12.
    A. Subasi, A decision support system for diagnosis of neuromuscular disorders using DWT and evolutionary support vector machines. Signal Image Video Process. 7, 1–10 (2013)Google Scholar
  13. 13.
    P.-F. Pai, M.-F. Hsu, An enhanced support vector machines model for classification and rule generation, in Computational Optimization, Methods and Algorithms (Springer, Berlin 2011) pp. 241–258Google Scholar
  14. 14.
    G.-B. Huang, H. Zhou, X. Ding, R. Zhang, Extreme learning machine for regression and multiclass classification. IEEE Trans. Syst. Man Cybern. Part B Cybern. 42(2), 513–529 (2012)CrossRefGoogle Scholar
  15. 15.
    C.R. Rao, S.K. Mitra, Generalized Inverse of a Matrix and Its Applications (Wiley, New York, 1971)Google Scholar
  16. 16.
    G.-B. Huang, X. Ding, H. Zhou, Optimization method based extreme learning machine for classification. Neurocomputing. 74(1), 155–163 (2010)Google Scholar
  17. 17.
    C. Cortes, V. Vapnik, Support-vector networks. Mach. Learn. 20(3), 273–297 (1995)MATHGoogle Scholar
  18. 18.
    A.J. Smola, B. Schölkopf, A tutorial on support vector regression. Stat. Comput. 14(3), 199–222 (2004)CrossRefMathSciNetGoogle Scholar
  19. 19.
    N.-Y. Liang, G.-B. Huang, P. Saratchandran, N. Sundararajan, A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans. Neural Networks. 17(6), 1411–1423 (2006)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  1. 1.Key Laboratory of Intelligent Information ProcessingInstitute of Computing Technology, Chinese Academy of SciencesBeijingChina
  2. 2.School of Computer and Control EngineeringUniversity of Chinese Academy of SciencesBeijingChina

Personalised recommendations