A Reliable Small Sample Classification Algorithm by Elman Neural Network Based on PLS and GA

Abstract

Aiming at the small sample with high-feature dimension and few numbers will cause a serious problem if simply using the traditional Elman neural network to deal with the small sample; these problems include poor learning ability, the redundancy structure, and incomplete training; these defects will result in lower operating efficiency and poor recognition precision. In this paper, combining the theory of partial least squares (PLS) and genetic algorithm (GA), as well as the nature of Elman neural network, and an optimized Elman neural network classification algorithm based on PLS and GA (PLS-GA-Elman) is established. The new algorithm reduces the feature dimension of small samples by PLS, the relatively ideal low-dimensional data is obtained, and the purpose reduces the neural network’s inputs and simplifies its structure. Using GA to optimize the connection weights, threshold values, and the number of hidden neurons and adopting the optimized way of encoding respectively and evolving simultaneously can improve the neural network incomplete training condition, leading to fewer number of samples and improvement in training speed and generalization ability; this ensures the optimal Elman neural network algorithm. A new algorithm based on twice consecutive optimization was the basis for a precise classification model. The results of experimental analysis illustrate that operating efficiency and classification precision of the new algorithm have been improved.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3

References

  1. Azali, S., & Sheikhan, M. (2016). Intelligent control of photovoltaic system using BPSO-GSA-optimized neural network and fuzzy-based PID for maximum power point tracking. Applied Intelligence, 44(1), 1–23.

    Article  Google Scholar 

  2. Ding, S. H., & Li, S. H. (2017). Second-order sliding mode controller design subject to mismatched term. Automatica, 77, 388–392.

    MathSciNet  MATH  Article  Google Scholar 

  3. Ding, S. F., Jia, W. K., Su, C. Y., Zhang, L. W., & Shi, Z. Z. (2008). Neural network research progress and applications in forecast. Lecture Notes in Computer Science, 5264, 783–793.

    Article  Google Scholar 

  4. Ding, S. F., Zhu, H., Jia, W. K., & Su, C. Y. (2012). A survey on feature extraction for pattern recognition. Artificial Intelligence Review, 37(3), 169–180.

    Article  Google Scholar 

  5. Elman, J. L. (1990). Finding structure in time. Cognitive Science, 14(2), 179–211.

    Article  Google Scholar 

  6. Gan, X. S., Duanmub, J. S., Wang, J. F., & Cong, W. (2013). Anomaly intrusion detection based on PLS feature extraction and core vector machine. Knowledge-Based Systems, 40, 1–6.

    Article  Google Scholar 

  7. Garimella, S., & Hermansky, H. (2013). Factor analysis of auto-associative neural networks with application in speaker verification. IEEE Transactions on Neural Networks and Learning Systems, 24(4), 522–528.

    Article  Google Scholar 

  8. Gopi, E. S. (2007). Digital image forgery detection using artificial neural network and independent component analysis. Applied Mathematics and Computation, 194(2), 540–543.

    MathSciNet  MATH  Article  Google Scholar 

  9. HOU, S. J., Chen, L., Tao, D. C., Zhou, S. B., Liu, W. J., & ZHENG, Y. J. (2017). Multi-layer multi-view topic model for classifying advertising video. Pattern Recognition, 68, 66–81.

    Article  Google Scholar 

  10. Hu, B., Zhao, Z., & Jun, L. (2012). Multi-loop nonlinear internal model controller design under nonlinear dynamic PLS framework using ARX-neural network model. Journal of Process Control, 22(1), 207–217.

    Article  Google Scholar 

  11. Huang, J. H., Li, X., & Wang, G. C. (2010). Maximum principles for a class of partial information risk-sensitive optimal controls. IEEE Transactions on Automatic Control, 55(6), 1438–1443.

    MathSciNet  MATH  Article  Google Scholar 

  12. Jane, C. J. (2017). An improved grey back propagation neural network with Levenberg-Marquardt algorithm and genetic algorithm. Journal of Grey System, 20(2), 71–77.

    Google Scholar 

  13. Janik, L. J., Forrester, S. T., & Rawson, A. (2009). The prediction of soil chemical and physical properties from mid-infrared spectroscopy and combined partial least-squares regression and neural networks (PLS-NN) analysis. Chemometrics and Intelligent Laboratory Systems, 97(2), 179–188.

    Article  Google Scholar 

  14. Jia, W.K., Zhao, D.A. , Tang, Y.Y., Hu, C.L., and Zhao, Y.Y. (2014). An optimized classification algorithm by neural network ensemble based on PLS and OLS, Mathematical Problems in Engineering, ID: 395263.

    Google Scholar 

  15. Kaikhah, K., & Garlick, R. (2000). Variable hidden layer sizing in Elman recurrent neuro-evolution. Applied Intelligence, 12(3), 193–205.

    Article  Google Scholar 

  16. Lee, H. K. H. (2007). Default priors for neural network classification. Journal of Classification, 24(1), 53–70.

    MathSciNet  MATH  Article  Google Scholar 

  17. Li, X. D., & Fu, X. L. (2012). Lag synchronization of chaotic delayed neural networks via impulsive control. IMA Journal of Mathematical Control & Information, 29(1), 133–145 (13).

    MathSciNet  MATH  Article  Google Scholar 

  18. Li, X. D., & Rakkiyappan, R. (2013). Stability results for Takagi–Sugeno fuzzy uncertain bam neural networks with time delays in the leakage term. Neural Computing & Applications, 22(1), 203–219.

    Article  Google Scholar 

  19. Liu, H., Zhang, P., Hu, B., & Moore, P. (2015). A novel approach to task assignment in a cooperative multi-agent design system. Applied Intelligence, 43(1), 162–175.

    Article  Google Scholar 

  20. Loghmanian, S. M. R., Jamaluddin, H., Ahmad, R., Yusof, R., & Khalid, M. (2012). Structure optimization of neural network for dynamic system modeling using multi-objective genetic algorithm. Neural Computing and Applications, 21(6), 1281–1295.

    Article  Google Scholar 

  21. Lu, J. J., & Chen, H. (2006). Researching development on BP neural networks. Control Engineering of China, 13(5), 449–451 (in Chinese).

    Google Scholar 

  22. Marques, J., & Erik, D. (2011). Texture analysis by a PLS based method for combined feature extraction and selection. Lecture Notes in Computer Science, 7009, 109–116.

    Article  Google Scholar 

  23. Mccllochw, S., & Pitts, W. (1943). A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biophysics, 10(5), 115–133.

    MathSciNet  MATH  Article  Google Scholar 

  24. Mcneish, D. M., & Harring, J. R. (2017). Clustered data with small sample sizes: comparing the performance of model-based and design-based approaches. Communications in Statistics-Simulation and Computation, 46(2), 855–869.

    MathSciNet  MATH  Article  Google Scholar 

  25. Ren, Y. W., Wang, A. P., & Wang, H. (2015). Fault diagnosis and tolerant control for discrete stochastic distribution collaborative control systems. IEEE Transactions on Systems Man & Cybernetics Systems, 45(3), 462–471.

    Article  Google Scholar 

  26. Sharifzadeh, S., Ghodsi, A., Clemmensen, L. H., & ErsbØll, B. K. (2017). Sparse supervised principal component analysis (SSPCA) for dimension reduction and variable selection. Engineering Applications of Artificial Intelligence, 65, 168–177.

    Article  Google Scholar 

  27. Shen, C., Song, R., LI, J., Zhang, X. M., Tang, J., Shi, Y. B., Liu, J., & Cao, H. L. (2016). Temperature drift modeling of MEMS gyroscope based on genetic-Elman neural network. Mechanical Systems and Signal Processing, 72, 897–905.

    Google Scholar 

  28. Shu, Z., Henson, R., & Willse, J. (2013). Using neural network analysis to define methods of DINA model estimation for small sample sizes. Journal of Classification, 30(2), 173–194.

    MathSciNet  MATH  Article  Google Scholar 

  29. Song, X. M., & Yan, X. H. (2014). Linear quadratic Gaussian control for linear time-delay systems. IET Control Theory & Applications, 8(6), 375–383.

    MathSciNet  Article  Google Scholar 

  30. Tsoi, A. C., & Back, A. (1997). Discrete time recurrent neural network architectures: a unifying review. Neurocomputing, 15(3), 183–223.

    MATH  Article  Google Scholar 

  31. UCI DATA SET, (2015), http://www.ics.uci.edu/~mlearn/databases/. Accessed 20 Dec 2015

  32. Xia, Y., & Wang, J. (2016). A bi-projection neural network for solving constrained quadratic optimization problems. IEEE transactions on neural networks and learning systems, 27(2), 214–224.

    MathSciNet  Article  Google Scholar 

  33. Yamamoto, M., & Hwang, H. (2017). Dimension-reduced clustering of functional data via subspace separation. Journal of Classification, 34(2), 294–326.

    MathSciNet  MATH  Article  Google Scholar 

  34. Zheng, X. W., Lu, D. J., Wang, X. G., & Liu, H. (2015). A cooperative coevolutionary biogeography-based optimize. Applied Intelligence, 43(1), 1–17.

    Article  Google Scholar 

Download references

Acknowledgments

The authors thank anonymous reviewers and the editor in chief Douglas Steinley Ph.D who made suggestions that helped to improve this paper.

Funding

This work is supported by Natural Science Foundation of Shandong Province in China (No.: ZR2017BC013); China Postdoctoral Science Foundation (No.: 2018M630797); Shandong Province Higher Educational Science and Technology Program (No.: J18KA308) National Nature Science Foundation of China (No.: 31571571, 61572300); Taishan Scholar Program of Shandong Province of China (No.: TSHW201502038).

Author information

Affiliations

Authors

Corresponding author

Correspondence to Weikuan Jia.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Jia, W., Zhao, D., Ding, L. et al. A Reliable Small Sample Classification Algorithm by Elman Neural Network Based on PLS and GA. J Classif 36, 306–321 (2019). https://doi.org/10.1007/s00357-018-9288-4

Download citation

Keywords

  • Small sample
  • Elman neural network
  • Partial least squares
  • Genetic algorithm
  • PLS-GA-Elman algorithm