Abstract
Deep learning has become one of very important machine learning methods in image classification, but most of them require a long training time to solve a non-convex optimization problem. In comparison, the training of extreme learning machine (ELM) is very simple, fast and effective. In order to combine the advantages of both methods, many researchers have tried to introduce ELM to deep architectures (Kasun et al. in IEEE Intell Syst 28:31–34, 2013; Yu et al. in Neurocomputing 149:308–315, 2015; Tissera and McDonnell in Neurocomputing 174:42–49, 2016 and in: Proceedings of ELM-2014, vol 1, Proceedings in adaptation, learning and optimization, vol 3, 2016; Junying et al. in Neurocomputing 171:63–72, 2016; Uzair et al. in Neural Comput Appl, 2015) to solve unsupervised learning and supervised learning problems. In this paper, we propose a new deep neural network based on ELM called discriminative deep ELM (DDELM) to address the semi-supervised learning problems in image classification. The proposed deep architecture consists of several stacked unsupervised ELMs and an additional label layer on the top layer of the stacked model. Experiments on three standard image data show that DDELM outperforms both representative semi-supervised learning algorithms and existing deep architectures such as DCNN in terms of accuracy and training time.
Similar content being viewed by others
References
Hinton G, Osindero S, Teh YW (2006) A fast learning algorithm for deep belief nets. Neural Comput 18:1527–1554
Huang GB, Zhu QY, Siew C-K (2004) Extreme learning machine: a new learning scheme of feedforward neural networks. IEEE International joint conference on neural networks 2:985–990
Kasun LLC, Zhou H, Huang GB (2013) Representational learning with ELMs for big data. IEEE Intell Syst 28:31–34
Yu W, Zhuang F, He Q, Shi Z (2015) Learning deep representations via extreme learning machines. Neurocomputing 149:308–315
Tissera MD, McDonnell MD (2016) Deep extreme learning machines: supervised autoencoding architecture for classification. Neurocomputing 174:42–49
Tissera MD, McDonnell MD (2014) Deep extreme learning machines for classification. In: Proceedings of ELM-2014, vol 1, Proceedings in adaptation, learning and optimization, vol 3. pp 345–354
Junying H, Jiangshe Z, Chunxia Z et al (2016) A new deep neural network based on a stack of single-hidden-layer feedforward neural networks with randomly fixed hidden neurons. Neurocomputing 171:63–72
Uzair M, Shafait F, Ghanem B, Mian A (2015) Representation learning with deep extreme learning machines for efficient image set classification. Neural Comput Appl
Liu Y, Zhou S et al (2011) Discriminative deep belief networks for visual data classification. Pattern Recognit 44:2287–2296
Hinton GE, Salakhutdinov R (2006) Reducing the dimensionality of data with neural networks. Science 313:504–507
Bengio Y, Lamblin P, Popovici D, Larochelle H (2006) Greedy layer-wise training of deep networks. In: NIPS
Huang G, Song SJ et al (2014) Semi-supervised and unsupervised extreme learning machines. IEEE Trans Cybern Extreme Learn Mach 44:2168–2267
Chapelle O, Scholkopf B, Zien A (2006) Semi-supervised learning. MIT Press, Cambridge
Chapelle O, Scholkopf B, Zien A (2006) Semi-supervised learning. MITPress, Cambridge
Rosenberg C, Hebert M, Schneiderman H (2005) Semi-supervised self-training of object detection models. In: Seventh IEEE workshops on application of computer vision. pp 29–36
Chapelle O, Zien A (2005) Semi-supervised classification by low density separation. International workshop on artificial intelligence and statistics 1:57–64
Sindhwani V, Niyogi P, Belkin M (2005) Beyond the point cloud: from transductive to semi-supervised learning. International conference on machine learning, ACM, Bonn, Germany 22:824–831
Collobert R, Sinz F, Weston J, Bottou L (2006) Large scale transductive SVMs. J Mach Learn Res 7:1687–1712
Blum A, Lafferty J Rwebangira MR et al (2004) Semi-supervised learning using randomized mincuts. In: Proceedings of the international conference on machine learning (ICML)
Zhu X, Ghahramani Z et al (2003) Semi-supervised learning using Gaussian fields and harmonic functions. Proceddings of the international conference on machine learning (ICML) 3:912–919
Fergus R, Weiss Y, Torralba A (2009) Semi-supervised learning in gigantic image collections. In: Advances in neural information processing systems (NIPS)
Weston J, Ratle F, Collobert R (2008) Deep learning via semi-supervised embedding. International conference on machine learning. ACM, Helsinki, pp 1168–1175
Zhu X (2007) Semi-supervised learning literature survey. Technical report, University of Wisconsin Madison, Madison, 123
Salakhutdinov RR, Hinton GE (2007) Learning a nonlinear embedding by preserving class neighbourhood structure. In: Proceedings of eleventh international conference on artificial intelligence and statistics
Boser BE, Guyon IM, Vapnik VN (1992) A training algorithm for optimal margin classifiers. In: COLT
Jarrett K, Kavukcuoglu K, Ranzato M, Cun YL (2009) What is the best multi-stage architecture for object recognition. In: ICCV
Li FF, Fergus R, Pernoa P (2004) Learning generative visual models from few training examples: an incremental Bayesian approach tested on 101 object categories. In: CVPR
Oliva A, Torralba A (2001) Modeling the shape of the scene: a holistic representation of the spatial envelope. In: IJCV
Pronobis A, Caputo B, Jensfelt P, Christensen HI (2010) A realistic benchmark for visual indoor place recognition. Robot Auton Syst 58:81–96
Zhong S, Liu Y, Yang Liu (2011) Bilinear deep learning for image classification. In: ACM conference on multimedia. pp 343–352
Sim T, Baker S (2003) The CMU pose, illumination and expression database. PAMI 25:1615–1618
He X.F, Cai D, Niyogi P (2005) Tensor subspace analysis. In: NIPS
Mitchell TM (1997) Machine learning
Acknowledgements
This work is supported by the National Basic Research Program of China (973 Program, No. 2013CB329404), the National Natural Science Foundation of China (No. 61572393).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Chang, P., Zhang, J., Hu, J. et al. A Deep Neural Network Based on ELM for Semi-supervised Learning of Image Classification. Neural Process Lett 48, 375–388 (2018). https://doi.org/10.1007/s11063-017-9709-0
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11063-017-9709-0