Skip to main content
SpringerLink
Log in

A Truly Online Learning Algorithm using Hybrid Fuzzy ARTMAP and Online Extreme Learning Machine for Pattern Classification

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

This paper presents a Hybrid Fuzzy ARTMAP (FAM) and Online Extreme learning machine (OELM), hereafter denoted as FAM-OELM, which enables online learning to start from the first trained data samples without having to set up an initialization phase which requires a chunk of data samples to be ready prior to training. The idea of developing FAM-OELM is motivated by the ELM concept proposed by Huang et al., for being an efficient learning algorithm that provides better generalization performance at a much faster learning speed. However, different from the batch learning ELM and its variant called the online sequential extreme learning machine which still requires an initial offline training phase before it can turn into online training, the proposed FAM-OELM showcases a framework that enable online learning to commence right from the first data sample. Here, classification can be conducted at any time during the training phase. Such appealing feature of the proposed algorithm has strictly fulfilled the criteria of being truly sequential, while many of the existing algorithms are not. In addition, FAM-OELM automatically grows hidden neuron such that the network can accommodate new information without over fitting and compromising on the knowledge learnt earlier. The simulation results reveal the efficacy and validity of FAM-OELM when it is applied to a real world application and various benchmark problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Duda RO, Hart PE, Stork DG (2001) Pattern classification, 2nd edn. Wiley, New York

  2. Moens M-F (2006) Information extraction: algorithms and prospects in a retrieval context, 1st edn. Springer, New York

    Google Scholar 

  3. Isaacs JC, Foo SY, Meyer-Baese A (2007) Novel kernels and kernel PCA for pattern recognition. In: Proceedings of the 2007 IEEE international symposium on computational intelligence in robotics and automation jacksonville, FL, U.S.A., 20–23 June, pp 438–443

  4. Ou G, Murphey YL (2007) Multi-class pattern classification using neural networks. Pattern Recognit 40(1):4–18

    Article  MATH  Google Scholar 

  5. Hsu C, Lin C (2002) A comparison of methods for multiclass support vector machines. IEEE Trans Neural Netw 13(2):415–425

    Article  Google Scholar 

  6. Bishop CM (1995) Neural Networks for pattern recognition. Oxford University Press, Oxford

    Google Scholar 

  7. Anand R, Mehrotra K, Mohan CK, ranka S (1995) Efficient classification for multiclass problems using modular neural networks. IEEE Trans Neural Netw 6:117–124

    Article  Google Scholar 

  8. Simpson PK (1992) Fuzzy min-max neural networks-part1: classification. IEEE Trans Neural Netw 3(5):776–786

    Article  Google Scholar 

  9. Huang G-B, Zhu Q-Y, Siew C-K (2006) Extreme learning machine: theory and applications. Neurocomputing 70:489–501

    Article  Google Scholar 

  10. Hornik K (1991) Approximation capabilities of multilayer feedforward networks. Neural Netw 4:251–257

    Article  Google Scholar 

  11. Leshno M, Lin VY, Pinkus A, Schocken S (1993) Multilayer feedforward networks with a nonpolynomial activation function can approximate any function. Neural Netw 6:861–867

    Article  Google Scholar 

  12. Huang GB, Chen Y, Babri HA (2000) Classification ability of a single hidden layer feedforward neural networks. IEEE Trans Neural Netw 11(3):799–801

    Article  Google Scholar 

  13. Huang G-B, Wang DH, Lan Y (2011) Extreme learning machines: a survey. Int J Mach Learn Cybern 2(2):107–122

    Article  Google Scholar 

  14. Liang NY, Huang G-B, Saratchandran P, Sundararajan N (2006) A fast and accurate online sequential learning algorithms for feedforward network. IEEE Trans Neural Netw 17(6):1411–1423

    Article  Google Scholar 

  15. Huang G-B, Zhou H, Ding X, Zhang R (2012) Extreme learning machine for regression and multi-class classification. IEEE Trans Syst Man Cybern Part B Cybern 42(2):513–529

    Article  Google Scholar 

  16. Li G, Liu M, Dong M (2010) A new online learning algorithm for structure-adjustable extreme learning machine. Neurocomput Comput Math Appl 60:377–389

    Article  MathSciNet  MATH  Google Scholar 

  17. Zhu Q, Qin AK, Suganthan PN, Huang GB (2005) Evolutionary extreme learning machine. Pattern Recognit 38:1759–1763

    Article  MATH  Google Scholar 

  18. Chen ZX, Zhu HY, Wang YG (2013) A modified extreme learning machine with sigmoidal activation functions. Neural Comput Appl 22:541–550

    Article  MathSciNet  Google Scholar 

  19. Nguyen D, Widrow B (1990) Improving the learning speed of 2-layer neural networks by choosing initial values of the adaptive weights. In: Proceedings of the international joint conference on neural networks IJCNN, San Diego, CA, USA

  20. Javed K, Gouriveau R, Zerhouni N (2014) SW-ELM: a summation wavelet extreme learning machine algorithm with a priori parameter initialization. Neurocomputing 123:299–307

    Article  Google Scholar 

  21. Han F, Yao H-F, Ling Q-H (2013) An improved evolutionary extreme learning machine based on particle swarm optimization. Neurocomputing 116:87–93

  22. Zong W, Huang G-B (2013) Learning to rank with extreme learning machine. Neural Process Lett 39(2):155–166

    Article  Google Scholar 

  23. Wang S-J, Chen H-L, Yan W-J, Chen Y-H, Fu X (2014) Face recognition and micro-expression recognition based on discriminant tensor subspace analysis plus extreme learning machine. Neural Process Lett 39:25–43

    Article  Google Scholar 

  24. Termenon M, Grana M, Barros-Loscertales A, Avila C (2013) Extreme learning machine for feature selection and classification of cocaine dependent patients on structural MRI data. Neural Process Lett 38:375–387

    Article  Google Scholar 

  25. Carpenter GA, Grossberg S, Markuzon N, Reynolds JH, Rosen DB (1992) Fuzzy ARTMAP: a neural network architecture for ncremental supervised learning of analog multidimensional maps. IEEE Trans Neural Netw 3(5):698–713

    Article  Google Scholar 

  26. Tan SC, Rao MVC, Lim CP (2007) A hybrid neural network classifier combining fuzzy ARTMAP and the dynamic decay adjustment algorithm. Soft Comput 12(8):765–775, Springer-Verlag

  27. Yap KS, Lim CP, Abidin IZ (2008) A hybrid ART-GRNN online learning neural network with a \(\varepsilon \)-insensitive loss function. IEEE Trans Neural Netw 19(9):1641–1646

    Article  Google Scholar 

  28. Wang Y, Cao F, Yuan Y (2011) A study on the effectiveness of extreme learning machine. Neurocomputing 74:2483–2490

    Article  Google Scholar 

  29. Blake C, Merz C (1998) UCI repository of machine learning databases, Dept. Inf. Comput. Sci., Univ. California, Irvine, CA, [Online]. Available: http://archive.ics.uci.edu/ml/. Accessed 02 April 2013

  30. Tenaga Nasional Berhad Malaysia (1999) System description and operating procedures prai power station stage 3, 14

  31. Lim CP, Harrison RF (2003) Online pattern classification with multiple neural network systems: an experimental study. IEEE Trans Syst Man Cybern Part C Appl Rev 33(2):235–247

    Google Scholar 

  32. Ankerst M, Ester M, Kriegel HP (2000) Towards an effective cooperation of the user and the computer for classification. In: Proceeding of 6th ACM SIGKDD int. conf. on knowledge discovery & data mining (KDD-2000), pp 179–189

  33. Tan SC, Lim CP (2004) Application of an adaptive neural network with symbolic rule extraction to fault detection and diagnosis an a power generation plant. IEEE Trans Energy Convers 19(2):369–377

    Article  Google Scholar 

  34. Quteishat AMA, Lim CP (2008) A modified fuzzy min-max neural network with rule extraction and its application to fault detection and classification. J Appl Soft Comput 8(2):985–995

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported by the University of Malaya Research Collaborative Grant Scheme (PRP-UM-UNITEN), under Grant Number: CG026-2013.

Author information

Authors and Affiliations

  1. College of Graduate Studies & Department of Electronics and Communication Engineering, Universiti Tenaga Nasional, Jalan IKRAM-UNITEN, 43009, Kajang, Selangor, Malaysia

    Shen Yuong Wong & Keem Siah Yap

  2. Department of Mechanical Engineering, Faculty of Engineering, University of Malaya, 50603, Kuala Lumpur, Malaysia

    Hwa Jen Yap

  3. Faculty of Information Science & Technology, Multimedia University, Jalan Ayer Keroh Lama, 75450, Bukit Beruang Melaka, Malaysia

    Shing Chiang Tan

Authors
  1. Shen Yuong Wong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Keem Siah Yap
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hwa Jen Yap
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shing Chiang Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Keem Siah Yap.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wong, S.Y., Yap, K.S., Yap, H.J. et al. A Truly Online Learning Algorithm using Hybrid Fuzzy ARTMAP and Online Extreme Learning Machine for Pattern Classification. Neural Process Lett 42, 585–602 (2015). https://doi.org/10.1007/s11063-014-9374-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-014-9374-5

Keywords

Search

Navigation