Aiming at the small sample with high-feature dimension and few numbers will cause a serious problem if simply using the traditional Elman neural network to deal with the small sample; these problems include poor learning ability, the redundancy structure, and incomplete training; these defects will result in lower operating efficiency and poor recognition precision. In this paper, combining the theory of partial least squares (PLS) and genetic algorithm (GA), as well as the nature of Elman neural network, and an optimized Elman neural network classification algorithm based on PLS and GA (PLS-GA-Elman) is established. The new algorithm reduces the feature dimension of small samples by PLS, the relatively ideal low-dimensional data is obtained, and the purpose reduces the neural network’s inputs and simplifies its structure. Using GA to optimize the connection weights, threshold values, and the number of hidden neurons and adopting the optimized way of encoding respectively and evolving simultaneously can improve the neural network incomplete training condition, leading to fewer number of samples and improvement in training speed and generalization ability; this ensures the optimal Elman neural network algorithm. A new algorithm based on twice consecutive optimization was the basis for a precise classification model. The results of experimental analysis illustrate that operating efficiency and classification precision of the new algorithm have been improved.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Price includes VAT for USA
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
This is the net price. Taxes to be calculated in checkout.
Azali, S., & Sheikhan, M. (2016). Intelligent control of photovoltaic system using BPSO-GSA-optimized neural network and fuzzy-based PID for maximum power point tracking. Applied Intelligence, 44(1), 1–23.
Ding, S. H., & Li, S. H. (2017). Second-order sliding mode controller design subject to mismatched term. Automatica, 77, 388–392.
Ding, S. F., Jia, W. K., Su, C. Y., Zhang, L. W., & Shi, Z. Z. (2008). Neural network research progress and applications in forecast. Lecture Notes in Computer Science, 5264, 783–793.
Ding, S. F., Zhu, H., Jia, W. K., & Su, C. Y. (2012). A survey on feature extraction for pattern recognition. Artificial Intelligence Review, 37(3), 169–180.
Elman, J. L. (1990). Finding structure in time. Cognitive Science, 14(2), 179–211.
Gan, X. S., Duanmub, J. S., Wang, J. F., & Cong, W. (2013). Anomaly intrusion detection based on PLS feature extraction and core vector machine. Knowledge-Based Systems, 40, 1–6.
Garimella, S., & Hermansky, H. (2013). Factor analysis of auto-associative neural networks with application in speaker verification. IEEE Transactions on Neural Networks and Learning Systems, 24(4), 522–528.
Gopi, E. S. (2007). Digital image forgery detection using artificial neural network and independent component analysis. Applied Mathematics and Computation, 194(2), 540–543.
HOU, S. J., Chen, L., Tao, D. C., Zhou, S. B., Liu, W. J., & ZHENG, Y. J. (2017). Multi-layer multi-view topic model for classifying advertising video. Pattern Recognition, 68, 66–81.
Hu, B., Zhao, Z., & Jun, L. (2012). Multi-loop nonlinear internal model controller design under nonlinear dynamic PLS framework using ARX-neural network model. Journal of Process Control, 22(1), 207–217.
Huang, J. H., Li, X., & Wang, G. C. (2010). Maximum principles for a class of partial information risk-sensitive optimal controls. IEEE Transactions on Automatic Control, 55(6), 1438–1443.
Jane, C. J. (2017). An improved grey back propagation neural network with Levenberg-Marquardt algorithm and genetic algorithm. Journal of Grey System, 20(2), 71–77.
Janik, L. J., Forrester, S. T., & Rawson, A. (2009). The prediction of soil chemical and physical properties from mid-infrared spectroscopy and combined partial least-squares regression and neural networks (PLS-NN) analysis. Chemometrics and Intelligent Laboratory Systems, 97(2), 179–188.
Jia, W.K., Zhao, D.A. , Tang, Y.Y., Hu, C.L., and Zhao, Y.Y. (2014). An optimized classification algorithm by neural network ensemble based on PLS and OLS, Mathematical Problems in Engineering, ID: 395263.
Kaikhah, K., & Garlick, R. (2000). Variable hidden layer sizing in Elman recurrent neuro-evolution. Applied Intelligence, 12(3), 193–205.
Lee, H. K. H. (2007). Default priors for neural network classification. Journal of Classification, 24(1), 53–70.
Li, X. D., & Fu, X. L. (2012). Lag synchronization of chaotic delayed neural networks via impulsive control. IMA Journal of Mathematical Control & Information, 29(1), 133–145 (13).
Li, X. D., & Rakkiyappan, R. (2013). Stability results for Takagi–Sugeno fuzzy uncertain bam neural networks with time delays in the leakage term. Neural Computing & Applications, 22(1), 203–219.
Liu, H., Zhang, P., Hu, B., & Moore, P. (2015). A novel approach to task assignment in a cooperative multi-agent design system. Applied Intelligence, 43(1), 162–175.
Loghmanian, S. M. R., Jamaluddin, H., Ahmad, R., Yusof, R., & Khalid, M. (2012). Structure optimization of neural network for dynamic system modeling using multi-objective genetic algorithm. Neural Computing and Applications, 21(6), 1281–1295.
Lu, J. J., & Chen, H. (2006). Researching development on BP neural networks. Control Engineering of China, 13(5), 449–451 (in Chinese).
Marques, J., & Erik, D. (2011). Texture analysis by a PLS based method for combined feature extraction and selection. Lecture Notes in Computer Science, 7009, 109–116.
Mccllochw, S., & Pitts, W. (1943). A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biophysics, 10(5), 115–133.
Mcneish, D. M., & Harring, J. R. (2017). Clustered data with small sample sizes: comparing the performance of model-based and design-based approaches. Communications in Statistics-Simulation and Computation, 46(2), 855–869.
Ren, Y. W., Wang, A. P., & Wang, H. (2015). Fault diagnosis and tolerant control for discrete stochastic distribution collaborative control systems. IEEE Transactions on Systems Man & Cybernetics Systems, 45(3), 462–471.
Sharifzadeh, S., Ghodsi, A., Clemmensen, L. H., & ErsbØll, B. K. (2017). Sparse supervised principal component analysis (SSPCA) for dimension reduction and variable selection. Engineering Applications of Artificial Intelligence, 65, 168–177.
Shen, C., Song, R., LI, J., Zhang, X. M., Tang, J., Shi, Y. B., Liu, J., & Cao, H. L. (2016). Temperature drift modeling of MEMS gyroscope based on genetic-Elman neural network. Mechanical Systems and Signal Processing, 72, 897–905.
Shu, Z., Henson, R., & Willse, J. (2013). Using neural network analysis to define methods of DINA model estimation for small sample sizes. Journal of Classification, 30(2), 173–194.
Song, X. M., & Yan, X. H. (2014). Linear quadratic Gaussian control for linear time-delay systems. IET Control Theory & Applications, 8(6), 375–383.
Tsoi, A. C., & Back, A. (1997). Discrete time recurrent neural network architectures: a unifying review. Neurocomputing, 15(3), 183–223.
UCI DATA SET, (2015), http://www.ics.uci.edu/~mlearn/databases/. Accessed 20 Dec 2015
Xia, Y., & Wang, J. (2016). A bi-projection neural network for solving constrained quadratic optimization problems. IEEE transactions on neural networks and learning systems, 27(2), 214–224.
Yamamoto, M., & Hwang, H. (2017). Dimension-reduced clustering of functional data via subspace separation. Journal of Classification, 34(2), 294–326.
Zheng, X. W., Lu, D. J., Wang, X. G., & Liu, H. (2015). A cooperative coevolutionary biogeography-based optimize. Applied Intelligence, 43(1), 1–17.
The authors thank anonymous reviewers and the editor in chief Douglas Steinley Ph.D who made suggestions that helped to improve this paper.
This work is supported by Natural Science Foundation of Shandong Province in China (No.: ZR2017BC013); China Postdoctoral Science Foundation (No.: 2018M630797); Shandong Province Higher Educational Science and Technology Program (No.: J18KA308) National Nature Science Foundation of China (No.: 31571571, 61572300); Taishan Scholar Program of Shandong Province of China (No.: TSHW201502038).
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Jia, W., Zhao, D., Ding, L. et al. A Reliable Small Sample Classification Algorithm by Elman Neural Network Based on PLS and GA. J Classif 36, 306–321 (2019). https://doi.org/10.1007/s00357-018-9288-4
- Small sample
- Elman neural network
- Partial least squares
- Genetic algorithm
- PLS-GA-Elman algorithm