Skip to main content
SpringerLink
Log in

Manifold regularized extreme learning machine

  • Extreme Learning Machine and Applications
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Extreme learning machine (ELM) works for generalized single-hidden-layer feedforward networks (SLFNs), and its essence is that the hidden layer of SLFNs need not be tuned. But ELM only utilizes labeled data to carry out the supervised learning task. In order to exploit unlabeled data in the ELM model, we first extend the manifold regularization (MR) framework and then demonstrate the relation between the extended MR framework and ELM. Finally, a manifold regularized extreme learning machine is derived from the proposed framework, which maintains the properties of ELM and can be applicable to large-scale learning problems. Experimental results show that the proposed semi-supervised extreme learning machine is the most cost-efficient method. It tends to have better scalability and achieve satisfactory generalization performance at a relatively faster learning speed than traditional semi-supervised learning algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Tang X, Han M (2009) Partial Lanczos extreme learning machine for single-output regression problems. Neurocomputing 72(13):3066–3076

    Article  Google Scholar 

  2. Liu Q, He Q, Shi ZZ (2008) Extreme support vector machine classifier. Lect Notes Comput Sci 5012:222–233

    Article  Google Scholar 

  3. Frénay B, Verleysen M (2010) Using SVMs with randomised feature spaces: an extreme learning approach. In: Proceedings of the 18th ESANN, Bruges, Belgium, pp 315–320

  4. Miche Y et al (2010) OP-ELM: optimally pruned extreme learning machine. IEEE Trans Neural Netw 21(1):158–162

    Article  Google Scholar 

  5. Deng W, Zheng Q, Chen L (2009) Regularized extreme learning machine. In: Proceedings of the IEEE symposium on CIDM, pp 389–395

  6. Huang G-B, Zhu Q-Y, Siew C-K (2004) Extreme learning machine: a new learning scheme of feedforward neural networks. In Proceedings of the IJCNN, Budapest, Hungary, vol 2, pp 985–990

  7. Huang G-B, Zhu Q-Y, Siew C-K (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1):489–501

    Article  Google Scholar 

  8. Huang G-B, Chen L, Siew C-K (2006) Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans Neural Netw 17(4):879–892

    Article  Google Scholar 

  9. Huang G-B, Chen L (2007) Convex incremental extreme learning machine. Neurocomputing 70(16):3056–3062

    Article  Google Scholar 

  10. Huang G-B, Chen L (2008) Enhanced random search based incremental extreme learning machine. Neurocomputing 71(16):3460–3468

    Article  Google Scholar 

  11. Wang G, Zhao Y, Wang D (2008) A protein secondary structure prediction frame- work based on the extreme learning machine. Neurocomputing 72(1–3):262–268

    Article  Google Scholar 

  12. Lan Y, Soh YC, Huang G-B (2008) Extreme Learning Machine based bacterial protein subcellular localization prediction. In: Proceedings of the IEEE international joint conference on neural networks, IJCNN 2008, Hong Kong, pp 1859–1863

  13. Zhang R, Huang G-B, Sundararajan N, Saratchandran P (2007) Multicategory classification using an extreme learning machine for microarray gene expression cancer diagnosis. IEEE/ACM Trans Comput Biol Bioinform 4(3):485–495

    Article  Google Scholar 

  14. Mohammed AA, Minhas R, Jonathan Wu QM, Sid-Ahmed MA (2011) Human face recognition based on multidimensional PCA and extreme learning machine. Pattern Recognit 44(10–11):2588–2597

    Article  MATH  Google Scholar 

  15. Nizar AH, Dong ZY, Wang Y (2008) Power utility nontechnical loss analysis with extreme learning machine method. IEEE Trans Power Syst 23(3):946–955

    Article  Google Scholar 

  16. Decherchi S, Gastaldo P, Dahiya RS, Valle M, Zunino R (2011) Tactile data classification of contact materials using computational intelligence. IEEE Trans Robot 27(3):635–639

    Article  Google Scholar 

  17. Decherchi S, Gastaldo P, Zunino R, Cambria E, Redi J (2013) Circular-ELM for the reduced-reference assessment of perceived image quality. Neurocomputing 102:78–89

    Article  Google Scholar 

  18. Cambria E, Hussain A (2012) Sentic album: content-, concept-, and context-based online personal photo management system. Cogn Comput 4(4):477–496

    Article  Google Scholar 

  19. Kumar MA (2010) An investigation on linear SVM and its variants for text categorization. In: Second international conference on machine learning and computing (ICMLC), pp 27–31

  20. Wang YG, Cao FL, Yuan YB (2011) A study on effectiveness of extreme learning machine. Neurocomputing 74:2483–2490

    Article  Google Scholar 

  21. Cao FL, Liu B, Park DS (2013) Image classification based on effective extreme learning machine. Neurocomputing 102:90–97

    Article  Google Scholar 

  22. Belkin M, Sindhwani V, Niyogi P (2006) Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res 7:2399–2434

    MATH  MathSciNet  Google Scholar 

  23. Xue H, Chen S, Yang Q (2009) Discriminatively regularized least-squares classification. Pattern Recognit 42(1):93–104

    Article  MATH  Google Scholar 

  24. Fan M, Gu N, Qiao H et al (2011) Sparse regularization for semi-supervised classification. Pattern Recognit 44(8):1777–1784

    Article  MATH  Google Scholar 

  25. Li Lina, Liu Dayou, Ouyang Jihong (2012) A new regularization classification method based on extreme learning machine in network data. J Inf Comput Sci 9(12):3351–3363

    Google Scholar 

  26. Liu Junfa, Chen Yiqiang, Liu Mingjie et al (2011) SELM: semi-supervised ELM with application in sparse calibrated location estimation. Neurocomputing 74(16):2566–2572

    Article  Google Scholar 

  27. Huang G-B, Zhou HM, Ding XJ (2012) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern Part B Cybern 42(2):513–529

    Article  Google Scholar 

  28. He Xiaofei, Cai Deng, Shao Yuanlong, Bao Hujun, Han Jiawei (2011) Laplacian regularized Gaussian mixture model for data clustering. IEEE Trans Knowl Data Eng 23(9):1406–1418

    Article  Google Scholar 

  29. Huang G-B et al (2011) Extreme learning machines: a survey. Int J Mach Learn Cybern 2:107–122

    Article  Google Scholar 

  30. Hull JJ (1998) A data base for hand written text recognition research. IEEE Trans Pattern Anal Mach Intell 16(5):550–554

    Article  Google Scholar 

  31. Lee KC, Ho J, Kriegman D (2005) Acquiring linear subspaces for face recognition under variable lighting. IEEE Trans Pattern Anal Mach Intell 27(5):684–698

    Article  Google Scholar 

  32. Zhao B, Wang F, Zhang C (2008) CutS3VM: a fast semi-supervised SVM algorithm. In: The 14th ACM SIGKDD international conference on knowledge discovery & data mining (KDD), Las Vegas, NV, USA, pp 830–838

  33. Sindhwani V, Keerthi SS (2006) Large scale semi-supervised linear SVMs. In: 29th Annual international ACM SIGIR. Technical report

Download references

Acknowledgments

This work is supported by the National Natural Science Foundation of China, China (No. 61403394) and the Fundamental Research Funds for the Central Universities (No. 2014QNA45).

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, 221116, Jiangsu, China

    Bing Liu, Shi-Xiong Xia, Fan-Rong Meng & Yong Zhou

Authors
  1. Bing Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shi-Xiong Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fan-Rong Meng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yong Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Shi-Xiong Xia or Fan-Rong Meng.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, B., Xia, SX., Meng, FR. et al. Manifold regularized extreme learning machine. Neural Comput & Applic 27, 255–269 (2016). https://doi.org/10.1007/s00521-014-1777-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-014-1777-8

Keywords

Search

Navigation