Abstract
In this paper, we have proposed a new algorithm to train neural network, called Class-Constrained Extreme Learning Machine (C\(^{2}\)ELM), which is based on Extreme Learning Machine (ELM). In C\(^{2}\)ELM, we use class information to constrain different parts of connection weights between input layer and hidden layer using Extreme Learning Machine Auto Encoder (ELM-AE). In this way, we add class information to the connection weights and make the features in the hidden layer which are learned from input space be more discriminative than other methods based on ELM. Meanwhile, C\(^{2}\)ELM can retain the advantages of ELM. The experiments shown that C\(^{2}\)ELM is effective and efficient and can achieve a higher performance in contrast to other ELM based methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Huang, G.-B., Chen, L., Siew, C.-K.: Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans. Neural Netw. 17(4), 879–892 (2006)
Huang, G.-B., Zhu, Q.-Y., Siew, C.-K.: Extreme Learning Machine: a new learning scheme of feedforward neural networks. In: IEEE International Joint Conference on Neural Networks, vol. 2, pp. 985–990 (2004)
Zhang, R., Lan, Y., Huang, G.-B., Xu, Z.-B.: Universal approximation of Extreme Learning Machine with adaptive growth of hidden nodes. IEEE Trans. Neural Netw. Learn. Syst. 23(2), 365–371 (2012)
Huang, G.-B., Zhu, Q.-Y., Siew, C.-K.: Extreme Learning Machine: theory and applications. Neurocomputing 70, 489–501 (2006)
Huang, G.-B., Zhou, H., Ding, X., Zhang, R.: Extreme Learning Machine for regression and multiclass classification. IEEE Trans. Syst. Man Cybern. Part B 42(2), 513–529 (2012)
Huang, G., Huang, G.-B., Song, S.-J., You, K.-Y.: Trends in Extreme Learning Machine: a review. Neural Netw. 61, 32–48 (2015)
Bartlett, P.-L.: The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network. IEEE Trans. Inf. Theory 44(2), 525–536 (1998)
Huang, G., Song, S., Gupta, J.N.D., Wu, C.: Semi-supervised and unsupervised Extreme Learning Machines. IEEE Trans. Cybern. 99, 1 (2014)
Kasun, L.L.C., Liu, T.-C, Yang, Y., Lin, Z.-P., Huang, G.-B.: Extreme Learning Machine for clustering. In: Proceedings of ELM-2014, pp. 435–444 (2014)
Huang, G.-B.: An insight into Extreme Learning Machines: random neurons, random features and kernels. Cogn. Comput. 1C15 (2014)
Kasun, L.L.C., Zhou, H., Huang, G.-B., Vong, C.M.: Representational learning with Extreme Learning Machines for big data. IEEE Intell. Syst. 28(6), 31–34 (2013)
Zhu, Q.-Y., Qin, A.K., Suganthan, P.N., Huang, G.-B.: Evolutionary Extreme Learning Machine. Pattern Recogn. 38(10), 1759–1763 (2005)
Yu, D., Deng, L.: Efficient and effective algorithms for training single-hidden-layer neural networks. Pattern Recogn. 33(5), 554–558 (2012)
Zhu, W.-T., Miao, J., Qing, L.-Y.: Constrained extreme learning machine: a novel highly discriminative randlom feedforward neural network. In: IEEE International Joint Conference on Neural Networks, pp. 800–807 (2014)
McDonnell, M.-D., Tissera, M.-D., van Schaik, A., Tapson, J.: Fast, simple and accurate handwritten digit classification using extreme learning machines with shaped input-weights. arXiv:1412.8307 (2014)
Rao, C.R., Mitra, S.K.: Generalized Inverse of Matrices and its Applications. Wiley, New York (1971)
Bengio, Y., Lambin, P., Popovici, D., Larochelle, H.: Greedy layer-wise training of deep networks. In: Advances in Neural Information Processing Systems, pp. 153–160 (2007)
Hoerl, A.E., Kennard, R.W.: Ridge regression: biased estimation for nonorthogonal problems. Technometrics 12(1), 55–67 (1970)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(22), 2278–2324 (1998)
Krizhevsky, A.: Learning multiple layers of features from tiny images (2009)
Acknowledgments
This work is supported in part by the National Natural Science Foundation of China (Nos. 61272320 and 61472387) and the Beijing Natural Science Foundation (No. 4152005).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Liu, X., Miao, J., Qing, L., Cao, B. (2016). Class-Constrained Extreme Learning Machine. In: Cao, J., Mao, K., Wu, J., Lendasse, A. (eds) Proceedings of ELM-2015 Volume 1. Proceedings in Adaptation, Learning and Optimization, vol 6. Springer, Cham. https://doi.org/10.1007/978-3-319-28397-5_41
Download citation
DOI: https://doi.org/10.1007/978-3-319-28397-5_41
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-28396-8
Online ISBN: 978-3-319-28397-5
eBook Packages: EngineeringEngineering (R0)