Applied Intelligence

, Volume 43, Issue 1, pp 176–191 | Cite as

An optimized classification algorithm by BP neural network based on PLS and HCA

  • Weikuan Jia
  • Dean Zhao
  • Tian Shen
  • Shifei Ding
  • Yuyan Zhao
  • Chanli Hu


Due to some correlative or repetitive factors between features or samples with high dimension and large amount of sample data, when traditional back-propagation (BP) neural network is used to solve this classification problem, it will present a series of problems such as network structural redundancy, low learning efficiency, occupation of storage space, consumption of computing time, and so on. All of these problems will restrict the operating efficiency and classification precision of neural network. To avoid them, partial least squares (PLS) algorithm is used to reduce the feature dimension of original data into low-dimensional data as the input of BP neural network, so that it can simplify the structure and accelerate convergence, thus improving the training speed and operating efficiency. In order to improve the classification precision of BP neural network by using hierarchical cluster analysis (HCA), similar samples are put into a sub-class, and some different sub-classes can be obtained. For each sub-class, a different training session can be conducted to find a corresponding precision BP neural network model, and the simulation samples of different sub-classes can be recognized by the corresponding network model. In this paper, the theories of PLS and HCA are combined together with the property of BP neural network, and an optimized classification algorithm by BP neural network based on PLS and HCA (PLS-HCA-BP algorithm) is proposed. The new algorithm is aimed at improving the operating efficiency and classification precision so as to provide a more reliable and more convenient tool for complex pattern classification systems. Three experiments and comparisons with four other algorithms are carried out to verify the superiority of the proposed algorithm, and the results indicate a good picture of the PLS-HCA-BP algorithm, which is worthy of further promotion.


Partial least squares Hierarchical cluster analysis BP neural network PLS-HCA-BP classification algorithm 



This work is supported by the National Natural Science Foundation of China (No. 60875052, 61203014, 61379101); Priority Academic Program Development of Jiangsu Higher Education Institutions; Major Projects in the National Science & Technology Pillar Program during the Twelfth Five-year Plan Period (No. 2011BAD20B06); The Specialized Research Fund for the Doctoral Program of Higher Education of China (No. 20133227110024); Ordinary University Graduate Student Research Innovation Projects of Jiangsu Province (No. KYLX 14_1062).


  1. 1.
    Ding SF, Jia WK, Su CY, Zhang LW, Shi ZZ (2008) Neural network research progress and applications in forecast. Lect Notes Comput Sci 5264:783–793Google Scholar
  2. 2.
    Rumelhart DE, Hinton GE, Williams RJ (1986) Learning representation by Back-Propagating errors. Nature 3(6):533–536CrossRefGoogle Scholar
  3. 3.
    Khoshnevisan B, Rafiee S, Omid M (2013) Prognostication of environmental indices in potato production using artificial neural networks. J Clean Prod 52:402–409CrossRefGoogle Scholar
  4. 4.
    Xu LQ, Liu SY (2013) Study of short-term water quality prediction model based on wavelet neural network. Math Comput Model 58(3-4):801–807CrossRefGoogle Scholar
  5. 5.
    Azami H, Mosavi MR, Sanei S (2013) Classification of GPS satellites using improved back propagation training algorithms. Wirel Pers Commun 71(2):789–803CrossRefGoogle Scholar
  6. 6.
    Ma YY, Zhu LP (2013) A review on dimension reduction. Int Stat Rev 81(1):134–150CrossRefMathSciNetGoogle Scholar
  7. 7.
    Jia WK, Ding SF, Xu XZ, Su CY, Shi ZZ (2011) Factor analysis feature extraction algorithm based on shannon entropy. Pattern Recognit Artif Intell 24(3):327–331Google Scholar
  8. 8.
    Blum MGB, Nunes MA, Prangle D, Sisson SA (2013) A comparative review of dimension reduction methods in approximate bayesian computation. Stat Sci 28(2):189–208CrossRefMathSciNetGoogle Scholar
  9. 9.
    Garimella S, Hermansky H (2013) Factor analysis of auto-associative neural networks with application in speaker verification. IEEE Trans Neural Netw Learn Syst 24(4):522–528CrossRefGoogle Scholar
  10. 10.
    Pradhan B, Lee S (2010) Landslide susceptibility assessment and factor effect analysis: backpropagation artificial neural networks and their comparison with frequency ratio and bivariate logistic regression modeling. Environ Model Softw 25(6):747–759CrossRefGoogle Scholar
  11. 11.
    Chattopadhyay S, Chattopadhyay G (2008) A factor analysis and neural network-based validation of the Varotsos-Cracknell theory on the 11-year solar cycle. Int J Remote Sens 29(9):2775–2786CrossRefGoogle Scholar
  12. 12.
    Boukhatem B, Kenai S, Hamou AT et al (2012) Predicting concrete properties using neural networks (NN) with principal component analysis (PCA) technique. Comput Concr 10(6):557–573CrossRefGoogle Scholar
  13. 13.
    Xiao YQ, He YG (2011) A novel approach for analog fault diagnosis based on neural networks and improved kernel PCA. Neurocomputing 74(7):1102–1115CrossRefGoogle Scholar
  14. 14.
    Bartecki K (2012) Neural network-based PCA: an application to approximation of a distributed parameter system. Lect Notes Artif Intell 7267:3–11Google Scholar
  15. 15.
    Gopi ES (2007) Digital image forgery detection using artificial neural network and independent component analysis. Appl Math Comput 194(2):540–543CrossRefzbMATHMathSciNetGoogle Scholar
  16. 16.
    Kuo SC, Lin CJ, Liao JR (2011) 3D reconstruction and face recognition using kernel-based ICA and neural networks. Expert Syst Appl 38(5):5406–5415CrossRefGoogle Scholar
  17. 17.
    Raju KS, Kumar DN (2007) Classification of Indian meteorological stations using cluster and fuzzy cluster analysis, and Kohonen artificial neural networks. Nord Hydrol 38(3):303–314CrossRefGoogle Scholar
  18. 18.
    Garlik B, Krivan M (2013) Identification of type daily diagrams of electric consumption based on cluster analysis of multi-dimensional data by neural network. Neural Netw World 23(3):271–283CrossRefGoogle Scholar
  19. 19.
    Leiva-Valdebenito SA, Torres-Aviles FJ (2010) A review of the most common partition algorithms in cluster analysis: a comparative study. Rev Colomb Estadistica 33(2):321–339MathSciNetGoogle Scholar
  20. 20.
    Richard AJ, Dean WW (2007) Applied multivariate statistical analysis In: 6th. Prentice-Hall, Englewood CliffsGoogle Scholar
  21. 21.
    Lin SK, Zhang DY, Li WX, Zhang TC, Zhang YF (2005) Neural network forecasting model based on clustering and principle components analysis. Mini-micro Syst 26(12):2160–2163Google Scholar
  22. 22.
    Ding SF, Jia WK, Jin FX, Zhang LW, Liu LL (2011) The research of neural network algorithm based on factor analysis and cluster analysis. Neural Comput & Applic 20(2):297–302CrossRefGoogle Scholar
  23. 23.
    Wang HW (2000) Partial least squares regression method and application. National Defense Industry Press, BeijingGoogle Scholar
  24. 24.
    Ding SF, Jin FX, Shi ZZ (2005) Information feature compression based on partial least squares. J Comput Aided Des Comput Graph 17(2):368–371Google Scholar
  25. 25.
    Gan XS, Duanmub JH, Wang JF, Gong W (2013) Anomaly intrusion detection based on PLS feature extraction and core vector machine. Knowl.-Based Syst. 40:1–6CrossRefGoogle Scholar
  26. 26.
    Marques J, Erik D (2011) Texture analysis by a PLS based method for combined feature extraction and selection. Lect Notes Comput Sci 7009:109–116Google Scholar
  27. 27.
    Janik LJ, Forrester ST, Rawson A (2009) The prediction of soil chemical and physical properties from mid-infrared spectroscopy and combined partial least-squares regression and neural networks (PLS-NN) analysis. Chemometr Intell Lab Syst 97(2):179–188CrossRefGoogle Scholar
  28. 28.
    Ding SF, Jia WK, Xu XZ, Su CY (2010) Elman neural network algorithm based on PLS. Acta Electron Sin 38(2A):71–75Google Scholar
  29. 29.
    Li RH, Meng GX, Gao NK, Xie HK (2007) Combined use of partial least-squares regression and neural network for residual life estimation of large generator stator insulation. Meas Sci Technol 18(7):2074–2082CrossRefGoogle Scholar
  30. 30.
    Gao DQ (1998) On structures of supervised linear basis function feed forward three-layered neural networks. Chin J Comput 21(1):80–86Google Scholar
  31. 31.
    Zhang YM (2003) The application of artificial neural network in the forecasting of wheat midge. Northwest A&F UniversityGoogle Scholar
  32. 32.
  33. 33. Waveform Database Generator (Version 2)
  34. 34.
    Wang XY, Garibaldi JM, Bird B, George MW (2007) A novel fuzzy clustering algorithm for the analysis of axillary lymph node tissue sections. Appl Intell 27(3):237–248CrossRefGoogle Scholar
  35. 35.
    Schwenker F, Trentin E (2014) Pattern classification and clustering: a review of partially supervised learning approaches. Pattern Recogn Lett 37:4–14CrossRefGoogle Scholar
  36. 36.
    Ghaseminezhad MH, Karami A (2011) A novel self-organizing map (SOM) neural network for discrete groups of data clustering. Appl Soft Comput 11(4):3771–3778CrossRefGoogle Scholar
  37. 37.
    Avci E (2012) An expert target recognition system using a genetic wavelet neural network. Appl Intell 37(4):475–487CrossRefGoogle Scholar
  38. 38.
    Siniscalchi SM, Yu D, Deng L (2013) Exploiting deep neural networks for detection-based speech recognition. Neurocomputing 106:148–157CrossRefGoogle Scholar
  39. 39.
    Matthews SG, Gongora MA, Hopgood AA (2013) Evolutionary algorithms and fuzzy sets for discovering temporal rules. Int J Appl Math Comput Sci 23(4):855–868CrossRefzbMATHMathSciNetGoogle Scholar
  40. 40.
    Lee SH, Lim JS (2014) Minimum feature selection for epileptic seizure classification using wavelet-based feature extraction and a fuzzy neural network. Appl Math Inform Sci 8(3):1295–1300CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  • Weikuan Jia
    • 1
  • Dean Zhao
    • 1
  • Tian Shen
    • 1
  • Shifei Ding
    • 2
    • 3
  • Yuyan Zhao
    • 1
    • 4
  • Chanli Hu
    • 1
  1. 1.School of Electrical and Information EngineeringJiangsu UniversityZhenjiangChina
  2. 2.School of Computer Science and TechnologyChina University of Mining and TechnologyXuzhouChina
  3. 3.Key Laboratory of Intelligent Information ProcessingInstitute of Computing Technology, Chinese Academy of SciencesBeijingChina
  4. 4.Changzhou College of information technologyChangzhouChina

Personalised recommendations