A Semi-supervised Online Sequential Extreme Learning Machine Method

  • Xibin Jia
  • Runyuan Wang
  • Junfa Liu
  • David M. W. Powers
Part of the Proceedings in Adaptation, Learning and Optimization book series (PALO, volume 3)

Abstract

Online sequential ELM (OS-ELM) provides a solution for streaming data application by only learning the newly arrived single or chunk of observations, and presents outstanding performance for learning problems. However, the algorithm relies on the labeled data, which usually involves high cost in labor and time. Moreover, manually labeled data suffers from inaccuracy caused by individual bias. Considering the semi-supervised ELM (SS-ELM) provides a way to fully utilize the easily acquired unlabeled data, the paper proposes a semi-supervised online sequential ELM, denoted as SOS-ELM. The proposed SOS-ELM not only has the advantage of learning in a sequential way, but also makes the most use of unlabeled data. Experiments have been done on benchmark problems of regression and classification and the results show that the proposed SOS-ELM outperforms OS-ELM in generalization performance with similar training speed and outperforms SS-ELM with much lower training time cost.

Keywords

Online Sequential ELM (OS-ELM) Semi-supervised ELM (SS-ELM) Semi-supervised online sequential ELM (SOS-ELM) 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Huang, G.-B., Zhu, Q.-Y., Siew, C.-K.: Extreme learning machine: a new learning scheme of feedforward neural networks. In: Proceedings of International Joint Conference on Neural Networks (IJCNN 2004), vol. 2, pp. 985–990 (2004)Google Scholar
  2. 2.
    Huang, G.-B., Zhu, Q.-Y., Siew, C.-K.: Extreme learning machine: theory and applications. Neurocomputing 70(1), 489–501 (2006)CrossRefGoogle Scholar
  3. 3.
    Huang, G.-B., Chen, L., Siew, C.-K.: Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans Neural Netw. 17(4), 879–892 (2006)CrossRefGoogle Scholar
  4. 4.
    Huang, G.-B., Chen, L.: Convex incremental extreme learning machine. Neurocomputing 70, 3056–3062 (2007)CrossRefGoogle Scholar
  5. 5.
    Huang, G.-B., Zhou, H., Ding, X., et al.: Extreme learning machine for regression and multiclass classification. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 42(2), 513–529 (2012)CrossRefGoogle Scholar
  6. 6.
    Huang, G.-B., Ding, X., Zhou, H.: Optimization method based extreme learning machine for classification. Neurocomputing 74(1), 155–163 (2010)CrossRefGoogle Scholar
  7. 7.
    Lan, Y., Soh, Y.C., Huang, G.-B.: Constructive hidden nodes selection of extreme learning machine for regression. Neurocomputing 73(16), 3191–3199 (2010)CrossRefGoogle Scholar
  8. 8.
    Lan, Y., Soh, Y.C., Huang, G.-B.: Two-stage extreme learning machine for regression Neurocomputing 73(16), 3028–3038 (2010)Google Scholar
  9. 9.
    Huang, G.-B.: An insight into extreme learning machines: random neurons, random features and kernels. Cognitive Computation, pp. 1–15 (2014)Google Scholar
  10. 10.
    Liang, N.-Y., Huang, G.-B., Saratchandran, P., et al.: A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Transactions on Neural Networks 17(6), 1411–1423 (2006)CrossRefGoogle Scholar
  11. 11.
    Li, K., Zhang, J., Xu, H., et al.: A Semi-supervised extreme learning machine method based on co-training. J Comput. Inf. Syst. 9(1), 207–214 (2013)MathSciNetGoogle Scholar
  12. 12.
    Huang, G., Song, S., Gupta, J.N.D., et al.: Semi-supervised and unsupervised extreme learning machines. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics (2014)Google Scholar
  13. 13.
    Tang, X., Han, M.: A semi-supervised learning method based on extreme learning machine. Journal of Dalian University of Technology 50(5), 771–776 (2010)MathSciNetGoogle Scholar
  14. 14.
    Liu, J.-F., Chen, Y.-Q., Liu, M.-J., et al.: SELM: Semi-supervised ELM with application in sparse calibrated location estimation. Neurocomputing 74(16), 2566–2572 (2011)CrossRefGoogle Scholar
  15. 15.
    Blake, C., Merz, C.: UCI repository of machine learning databases Dept. Inf. Comp. Sci. Univ.California, Irvine (1998)Google Scholar
  16. 16.
    Huang, G.-B., Zhu, Q.-Y., Siew, C.-K.: Extreme learning machine: theory and applications. Neurocomputing 70(1), 469–501 (2006)Google Scholar
  17. 17.
    Vapnik, V.: The nature of statistical learning theory. Springer (2000)Google Scholar
  18. 18.
    Chapelle, O., Sch, B., Zien, A.: Semi-Supervised Learning. MIT Press (2006), http://www.kyb.tuebingen.mpg.de/ssl-book
  19. 19.
    Belkin, M., Matveeva, I., Niyogi, P.: Regularization and semi-supervised learning on large graphs. In: Shawe-Taylor, J., Singer, Y. (eds.) COLT 2004. LNCS (LNAI), vol. 3120, pp. 624–638. Springer, Heidelberg (2004)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Xibin Jia
    • 1
  • Runyuan Wang
    • 1
  • Junfa Liu
    • 2
  • David M. W. Powers
    • 1
  1. 1.Beijing Municipal Key Laboratory of Multimedia and Intelligent Software TechnologyBeijing University of TechnologyBeijingP.R. China
  2. 2.Institute of Computing Technology, Chinese Academy of SciencesBeijingP.R. China

Personalised recommendations