Two novel ELM-based stacking deep models focused on image recognition

  • Gang Song
  • Qun DaiEmail author
  • Xiaomeng Han
  • Lin Guo


Extreme learning machine (ELM) and its variants have been widely used in the field of object recognition and other complex classification tasks. Traditional deep learning architectures like Convolutional Neural Network (CNN) are capable of extracting high-level features, which are the key for the models to make right decisions. However, traditional deep architectures are confronted with solving a tough, non-convex optimization problem, which is a time-consuming process. In this paper, we propose two hierarchical models, i.e., Random Recursive Constrained ELM (R2CELM) and Random Recursive Local- Receptive-Fields-Based ELM (R2ELM-LRF), which are constructed by stacking with CELM or ELM-LRF, respectively. Besides, inspired by the stacking generalization philosophy, random projection and kernelization are incorporated as their constitutive elements. R2CELM and R2ELM-LRF not only fully inherit the merits of ELM, but also take advantage of the superiority of CELM and ELM-LRF in the field of image recognition, respectively. The essence of CELM is to constrain the weight vectors from the input layer to the hidden layer to be consistent with the directions from one class to another class, while ELM-LRF is adept at exploiting the local structures in images through many local receptive fields. In the empirical results, R2CELM and R2ELM-LRF demonstrate their better performance in testing accuracy on the six benchmark image recognition datasets, compared with their basic learners and other state-of-the-art algorithms. Moreover, the proposed two deep ELM models need less training time when compared with traditional Deep Neural Network (DNN) based models.


Deep learning Extreme learning machine (ELM) Stacking generalization philosophy Random recursive constrained ELM (R2CELM) Random recursive local-receptive-fields-based ELM (R2ELM-LRF) Image recognition 



This work is supported by the National Natural Science Foundation of China under the Grant no. 61473150.

Compliance with ethical standards

Conflict of interests

The authors declare that they have no conflict of interest.


  1. 1.
    Nath SS, Mishra G, Kar J, Chakraborty S, Dey N (2014) A survey of image classification methods and techniques. In: International conference on control, instrumentation, communication and computational technologies, pp 554–557Google Scholar
  2. 2.
    Li Z, Fan Y, Liu W (2015, 2015) The effect of whitening transformation on pooling operations in convolutional autoencoders. EURASIP J Adv Signal Process:37–48Google Scholar
  3. 3.
    Lowe DG, Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60:91–110CrossRefGoogle Scholar
  4. 4.
    N. Dalal and B. Triggs (2005) Histograms of oriented gradients for human detection. In: Computer vision and pattern recognition (CVPR), pp 886–893Google Scholar
  5. 5.
    Ojala T, Pietikäinen M, Mäenpää T (2001) A generalized local binary pattern operator for multiresolution gray scale and rotation invariant texture classification. Springer, Berlin HeidelbergCrossRefGoogle Scholar
  6. 6.
    Collobert R, Weston J (2008) A unified architecture for natural language processing: deep neural networks with multitask learning. In: Proceedings of the 25th International Conference on Machine Learning, Helsinki, Finland, pp 160–167Google Scholar
  7. 7.
    Goodfellow IJ, Bulatov Y, Ibarz J, Arnoud S, Shet V (2014) Multi-digit number recognition from street view imagery using deep convolutional neural networks. Computer ScienceGoogle Scholar
  8. 8.
    Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313:504–507MathSciNetCrossRefGoogle Scholar
  9. 9.
    Huval B, Coates A, Ng A (2013) Deep learning for class-generic object detection. Computer ScienceGoogle Scholar
  10. 10.
    Shin HC, Orton MR, Collins DJ, Doran SJ, Leach MO (2013) Stacked autoencoders for unsupervised feature learning and multiple organ detection in a pilot study using 4D patient data. IEEE Trans Pattern Anal Mach Intell 35:1930–1943CrossRefGoogle Scholar
  11. 11.
    Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70:489–501CrossRefGoogle Scholar
  12. 12.
    Wolpert DH (1992) Stacked generalization. Neural Netw 5:241–259CrossRefGoogle Scholar
  13. 13.
    Zhu W, Miao J, Qing L (2015) Constrained extreme learning machines: a study on classification cases. Computer ScienceGoogle Scholar
  14. 14.
    Huang GB, Bai Z, Chi MV (2015) Local receptive fields based extreme learning machine. IEEE Comput Intell Mag 10:18–29CrossRefGoogle Scholar
  15. 15.
    Hinton GE, Osindero S, Teh YW (2006) A fast learning algorithm for deep belief nets. Neural Comput 18:1527–1543MathSciNetCrossRefGoogle Scholar
  16. 16.
    Lopes N, Ribeiro B (2015) Deep belief networks (DBNs). Springer, ChamCrossRefGoogle Scholar
  17. 17.
    Salakhutdinov R, Hinton G (2009) Deep Boltzmann machines. J Mach Learn Res 5:1967–2006zbMATHGoogle Scholar
  18. 18.
    Vincent P, Larochelle H, Lajoie I, Bengio Y, Manzagol PA (2010) Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. J Mach Learn Res 11:3371–3408MathSciNetzbMATHGoogle Scholar
  19. 19.
    Dan C, Meier U, Schmidhuber J (2012) Multi-column deep neural networks for image classification. IEEE Conference on Computer Vision and Pattern Recognition:3642–3649Google Scholar
  20. 20.
    Tang J, Deng C, Huang GB (2016) Extreme learning machine for multilayer perceptron. IEEE Trans Neural Netw Learn Syst 27:809–821MathSciNetCrossRefGoogle Scholar
  21. 21.
    Haykin S, Kosko B (2009) Gradient based learning applied to document recognition. IEEE Wiley-IEEE Press:306–351Google Scholar
  22. 22.
    Krizhevsky A (2012) Learning multiple layers of features from tiny images. Master’s thesis, Department of Computer Science, University of TorontoGoogle Scholar
  23. 23.
    Lecun Y, Huang FJ, Bottou L (2004) Learning methods for generic object recognition with invariance to pose and lighting. Computer Vision and Pattern Recognition (CVPR) 2:97–104Google Scholar
  24. 24.
    Cai D, He X, Han J, Huang TS (2011) Graph regularized nonnegative matrix factorization for data representation. IEEE Trans Pattern Anal Mach Intell 33:1548–1560CrossRefGoogle Scholar
  25. 25.
    Hull JJ (1994) A database for handwritten text recognition research. IEEE Trans Pattern Anal Mach Intell 16:550–554CrossRefGoogle Scholar
  26. 26.
    Cai D, He X, Hu Y et al (2015) Learning a spatially smooth subspace for face recognition. IEEE conference on computer vision and pattern recognition:1–7Google Scholar
  27. 27.
    Cheng M (2015) The cross-field DBN for image recognition. IEEE international conference on progress in informatics and computing:83–86Google Scholar
  28. 28.
    Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. International conference on neural information processing systems:1097–1105Google Scholar
  29. 29.
    Ahmed E, Jones M, Marks TK (2015) An improved deep learning architecture for person re-identification. IEEE conference on computer vision and pattern recognition (CVPR):3908–3916Google Scholar
  30. 30.
    He K, Zhang X, Ren S, Sun J (2015, 2015) Deep residual learning for image recognition. IEEE conference on computer vision and pattern recognition (CVPR):770–778Google Scholar
  31. 31.
    Shah SAA, Bennamoun M, Boussaid F (2015) Iterative deep learning for image set based face and object recognition. Neurocomputing 174:866–874CrossRefGoogle Scholar
  32. 32.
    Donahue J, Jia Y, Vinyals O, Hoffman J, Zhang N, Tzeng E et al (2013) DeCAF: a deep convolutional activation feature for generic visual recognition. Comput Sci 50:815–830Google Scholar
  33. 33.
    Parker SP (2012) GPU implementation of a deep learning network for image recognition tasks. MS (Master of Science) thesis, University of IowaGoogle Scholar
  34. 34.
    Han X, Dai Q Batch-normalized Mlpconv-wise supervised pre-training network in network. Applied Intelligence 48(1):142–155. CrossRefGoogle Scholar
  35. 35.
    Lv Y, Duan Y, Kang W, Li Z (2015) Traffic flow prediction with big data: a deep learning approach. IEEE Trans Intell Transp Syst 16:865–873Google Scholar
  36. 36.
    Vinyals O, Jia Y, Deng L, Darrell T (2012) Learning with recursive perceptual representations. Adv Neural Inf Proces Syst:2834–2842Google Scholar
  37. 37.
    Wang S, Deng C, Lin W, Huang GB, Zhao B (2017) NMF-based image quality assessment using extreme learning machine. IEEE Trans Cybern 47(1):232–243CrossRefGoogle Scholar
  38. 38.
    Deng C, Wang S, Li Z, Huang GB, Lin W (2017) Content-insensitive blind image blurriness assessment using Weibull statistics and sparse extreme learning machine. IEEE Trans Syst Man Cybern: Systems 99:1–12Google Scholar
  39. 39.
    Decherchi S, Gastaldo P, Zunino R, Cambria E, Redi J (2013) Circular-ELM for the reduced-reference assessment of perceived image quality. Neurocomputing 102:78–89CrossRefGoogle Scholar
  40. 40.
    Liu H, Wu Y, Sun F Extreme trust region policy optimization for active object recognition. IEEE Trans Neural Netw Learn Syst 29(6):2253–2258. MathSciNetCrossRefGoogle Scholar
  41. 41.
    Liu H, Qin J, Sun F, Guo D (2017) Extreme kernel sparse learning for tactile object recognition. IEEE Trans Cybern 47(12):4509–4520CrossRefGoogle Scholar
  42. 42.
    Huang GB, Zhou H, Ding X et al (2012) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern: Systems 42:513–529CrossRefGoogle Scholar
  43. 43.
    Huang GB, Wang DH, Lan Y (2011) Extreme learning machines: a survey. Int J Mach Learn Cybern 2:107–122CrossRefGoogle Scholar
  44. 44.
    Huang GB, Chen L, Siew CK (2006) Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans Neural Netw 17:879–892CrossRefGoogle Scholar
  45. 45.
    Jin Y,Peng L,Zhang W (2018) Hybrid macro/micro level backpropagation for training deep spiking neural networks. In: 32nd conference on neural information processing systems (NeurIPS), Montréal, CanadaGoogle Scholar
  46. 46.
    Blumensath T, Davies ME (2007) On the difference between orthogonal matching pursuit and orthogonal least squares. Unpublished manuscript, available at:
  47. 47.
    Yu W, Zhuang F, He Q, Shi Z (2015) Learning deep representations via extreme learning machines. Neurocomputing 149:308–315CrossRefGoogle Scholar
  48. 48.
    Vincent P,Larochelle H,Bengio Y, Manzagol PA 2008) Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th International Conference on Machine Learning, Helsinki, Finland, pp. 1096–1103Google Scholar
  49. 49.
    Le QV, Ngiam J, Chen Z, Chia DJH, Pang WK, Ng AY (2010) Tiled convolutional neural networks. In: International conference on neural information processing systems, pp. 1279–1287,Google Scholar
  50. 50.
    Mcdonnell MD, Vladusich T (2015) Enhanced image classification with a fast-learning shallow convolutional neural network. In: International joint conference on neural networks, Killarney, IrelandGoogle Scholar
  51. 51.
    Maaløe L, Sønderby CK, Sønderby SK, Winther O (2016) Auxiliary deep generative models. arXiv:1602.05473 [stat.ML]Google Scholar
  52. 52.
    Cherla S, Tran SN, Garcez AD, Weyde T (2017) Generalising the discriminative restricted Boltzmann machines. In: Lintas A, Rovetta S, Verschure P, Villa A (eds) Artificial neural networks and machine learning – ICANN 2017, Lecture notes in computer science, vol 10614. Springer, ChamCrossRefGoogle Scholar
  53. 53.
    Mazdak F, Mahmood A, Arash A, Shahsavari M, Devienn P (2016) Towards an spiking deep belief network for face recognition application. In: 2016 6th International Conference on Computer and Knowledge Engineering (ICCKE), Mashhad, Iran
  54. 54.
    Brancati N, Gragnaniello D, Verdoliva L (2016) Scale invariant descriptor for content based image retrieval in biomedical applications. In 2016 12th international conference on signal-image technology & Internet-based systems (SITIS), Naples, Italy
  55. 55.
    Agarap AF (2019) An architecture combining convolutional neural network (CNN) and support vector machine (SVM) for image classification. arXiv:1712.03541v2 [cs.CV]Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2020

Authors and Affiliations

  1. 1.College of Computer Science and TechnologyNanjing University of Aeronautics and AstronauticsNanjingChina

Personalised recommendations