Skip to main content
SpringerLink
Log in

Parameter determination and feature selection for back-propagation network by particle swarm optimization

  • Regular Paper
  • Published:
Knowledge and Information Systems Aims and scope Submit manuscript

Abstract

The back-propagation network (BPN) is a popular tool with applications in a variety of fields. Nevertheless, different problems may require different parameter settings for a given network architecture. A dataset may contain many features, but not all features are beneficial for classification by the BPN. Therefore, a particle-swarm-optimization-based approach, denoted as PSOBPN, is proposed to obtain the suitable parameter settings for BPN and to select the beneficial subset of features which result in a better classification accuracy rate. A set of 23 problems with a range of examples and features drawn from the UCI (University of California, Irvine) machine learning repository is adopted to test the performance of the proposed algorithm. The results are compared with several well-known published algorithms. The comparative study shows that the proposed approach improves the classification accuracy rate in most test problems. Furthermore, when the feature selection is taken into consideration, the classification accuracy rates of most datasets are increased. The proposed algorithm should thus be useful to both practitioners and researchers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Abe N, Kudo M (2006) Non-parametric classifier-independent feature selection. Pattern Recognit 39: 737–746

    Article  MATH  Google Scholar 

  2. Ahluwalia M, Bull L (1999) Coevolving functions in genetic programming: classification using k-nearest-neighbour, In: Banzhaf W, Daida J, Eiben G et al (eds) GECCO-99. In: Proceedings of the genetic and evolutionary computation conference. Morgan Kaufmann, Florida, pp 947–952

  3. Barletta M, Gisario A, Guarino S (2007) Modeling of electrostatic fluidized bed (EFB) coating process using artificial neural networks. Eng Appl Artif Intell 20: 721–733

    Article  Google Scholar 

  4. Berry MJA, Linoff G (eds) (2001) Data mining techniques: for marking, sales and customer support. Wiley, NY

    Google Scholar 

  5. Castillo PA, Carpio J, Merelo JJ et al (2000a) Evolving multilayer perceptrons. Neural Process Lett 12: 115–127

    Article  MATH  Google Scholar 

  6. Castillo PA, Merelo JJ, Prieto A et al (2000b) G-prop: global optimization of multilayer perceptrons using GAs. Neurocomputing 35: 149–163

    Article  MATH  Google Scholar 

  7. Chen W-C, Fu GL, Tai P-H et al (2009) Process parameter optimization for MIMO plastic injection molding via soft computing. Expert Syst Appl 36: 1114–1122

    Article  Google Scholar 

  8. Choi YS (2001) Discovering text databases with neural nets. Knowl Inf Syst 3: 356–373

    Article  MATH  Google Scholar 

  9. Danaher S, Datta S, Waddle I et al (2004) Erosion modeling using bayesian regulated artificial neural networks. Wear 256: 879–888

    Article  Google Scholar 

  10. Dixon PW, Corne DW, Oates MJ (2002) A preliminary investigation of modified XCS as a generic data mining tool. In: Lanai PL, Stolzmann W, Wilson S (eds) Advertences in learning classifier systems. Springer, Berlin, pp 257–269

    Google Scholar 

  11. Ebrahimzadeh A, Ranjbar A (2008) Intelligent digital signal-type identification. Eng Appl Artif Intell 21: 569–577

    Article  Google Scholar 

  12. Fung G, Mangasarian OL (2003) Finite Newton method for Lagrangian support vector machine classification. Neurocomputing 55: 39–55

    Article  Google Scholar 

  13. Ghosh A, Jain LC (2004) Evolutionary computation in data mining. Springer, Germany

    Google Scholar 

  14. Ghosh R, Verma B (2003) A hierarchical method for finding optimal architecture and weights using evolutionary least square based learning. Int J Neural Syst 13: 13–24

    Article  Google Scholar 

  15. Gupta JND, Sexton RS (1999) Comparing backpropagation with a genetic algorithm for neural network training. Int J Manag Sci 27: 679–684

    Google Scholar 

  16. Gusikhin O, Rychtyckyj N, Filev D (2007) Intelligent systems in the automotive industry: applications and trends. Knowl Inf Syst 12: 147–168

    Article  Google Scholar 

  17. Han J, Kamber M (2007) Data mining: concepts and techniques. Morgan Kaufmann, San Francisco

    Google Scholar 

  18. He Q, Wang L (2007) An effective co-evolutionary particle swarm optimization for constrained engineering design problems. Eng Appl Artif Intell 20: 89–99

    Article  Google Scholar 

  19. Heo G-S, Oh I-S (2008) Simultaneous node pruning of input and hidden layers using genetic algorithms. In: Proceedings of IEEE the seventh international conference on machine learning and cybernetics. pp 3428–3433

  20. Hettich S, Blake CL, Merz CJ (1998) UCI repository of machine learning databases, Department of Information and Computer Sciences, University of California, Irvine, CA. http://www.ics.uci.edu/~mlearn/MLRepository.html

  21. Hou T-H, Su C-H, Chang H-Z (2008) Using neural networks and immune algorithms to find the optimal parameters for an IC wire bonding process. Expert Syst Appl 34: 427–436

    Article  Google Scholar 

  22. Huang C-Y, Chen L-H, Chen Y-L et al (2009) Evaluating the process of a genetic algorithm to improve the back-propagation network: a Monte Carlo study. Expert Syst Appl 36: 1459–1465

    Article  Google Scholar 

  23. Javadi AA, Farmani R, Tan TP (2005) A hybrid intelligent genetic algorithm. Adv Eng Inform 19: 225–262

    Article  Google Scholar 

  24. Karpenko M, Sepehri N (2002) Neural network classifiers applied to condition monitoring of a pneumatic process valve actuator. Eng Appl Artif Intell 15: 273–283

    Article  Google Scholar 

  25. Kennedy J, Eberhart R (1995) Particle swarm optimization. Proc IEEE Conf Neural Netw 4: 1942–1948

    Article  Google Scholar 

  26. Khaw JFC, Lim BS, Lim LEN (1995) Optimal design of neural networks using the Taguchi method. Neurocomputing 7: 225–245

    Article  MATH  Google Scholar 

  27. Kim K-J, Han I (2000) Genetic algorithms approach to feature discretization in artificial neural networks for the prediction of stock price index. Expert Syst Appl 19: 125–132

    Article  Google Scholar 

  28. Ko Y-D, Moon P, Kim CE et al (2009) Modeling and optimization of the growth rate for ZnO thin films using neural networks and genetic algorithms. Expert Syst Appl 36: 4061–4066

    Article  Google Scholar 

  29. Kohavi R, John GH (1997) Wrappers for feature subset selection. Artif Intell 97: 273–324

    Article  MATH  Google Scholar 

  30. Kwok NM, Liu DK, Dissanayake G (2006) Evolutionary computing based mobile robot localization. Eng Appl Artif Intell 19: 857–868

    Article  Google Scholar 

  31. Lee J, Kang S (2007) GA based meta-modeling of BPN architecture for constrained approximate optimization. Int J Solids Struct 44: 5980–5993

    Article  MATH  Google Scholar 

  32. Lezoray O, Cardot H (2001) A neural network architecture for data classification. Int J Neural Syst 11: 33–42

    Google Scholar 

  33. Li S, Li Y, Liu Y et al (2007) A GA-based NN approach for makespan estimation. Appl Math Comput 185: 1003–1014

    Article  MATH  Google Scholar 

  34. Li W-S, Clifton C, Liu S-Y (2000) Database integration using neural networks: implementation and experiences. Knowl Inf Syst 2: 73–96

    Article  MATH  Google Scholar 

  35. Lin SC, Ting CJ (1996) Drill wear monitoring using neural networks. Int J Mach Tools Manuf 36: 465–475

    Article  Google Scholar 

  36. Lin S-W, Tseng T-Y, Chou S-Y et al (2008) A simulated annealing based approach for simultaneous parameter optimization and feature selection of back-propagation networks. Expert Syst Appl 34: 1491–1499

    Article  Google Scholar 

  37. Lin S-W, Lee Z-J, Chen S-C et al (2008) Parameter determination of support vector machines and feature selection using simulated annealing approach. Appl Soft Comput 8: 1505–1512

    Article  Google Scholar 

  38. Lin S-W, Ying K-C, Chen S-C et al (2008) Particle swarm optimization for parameter determination and feature selection of support vector machines. Expert Syst Appl 35: 1817–1824

    Article  Google Scholar 

  39. Lin S-W, Chou S-Y, Chen S-C (2007) Irregular shapes classification by back-propagation neural network. Int J Adv Manuf Technol 34: 1164–1172

    Article  Google Scholar 

  40. Liu H, Motoda H (1998) Feature selection for knowledge discovery and data mining. Kluwer Academic, Boston

    MATH  Google Scholar 

  41. Liu L, Liu W, Cartes DA (2008) Particle swarm optimization-based parameter identification applied to permanent magnet synchronous motors. Eng Appl Artif Intell 21: 1092–1100

    Article  Google Scholar 

  42. Mahesh Parappagoudar B, Pratihar DK, Datta GL (2008) Forward and reverse mappings in green sand mould system using neural networks. Appl Soft Comput 8: 239–260

    Article  Google Scholar 

  43. Malhotra R, Malhotra DK (2003) Evaluating consumer loans using neural networks. Int J Manag Sci 31: 83–96

    Google Scholar 

  44. Mangasarian OL, Musicant DR (2001) Lagrangian support vector machine. J Mach Res 1: 161–177

    Article  MATH  MathSciNet  Google Scholar 

  45. Pudil P, Novovičová J, Kittler J (1994) Floating search methods in feature selection. Pattern Recognit Lett 15: 1119–1125

    Article  Google Scholar 

  46. Peng T, Zuo W, He F (2008) SVM based adaptive learning method for text classification from positive and unlabeled documents. Knowl Inf Syst 16: 281–301

    Article  Google Scholar 

  47. Qiao W, Gao Z, Harley RG et al (2008) Robust neuro-identification of nonlinear plants in electric power systems with missing sensor measurements. Eng Appl Artif Intell 21: 604–618

    Article  Google Scholar 

  48. Sasaki T, Tokoro M (2000) Comparison between Lamarckian and Darwinian evolution on a model using neural networks and genetic algorithms. Knowl Inf Syst 2: 201–222

    Article  MATH  Google Scholar 

  49. Salzberg SL (1997) On comparing classifiers: pitfalls to avoid and a recommended approach. Data Min Knowl Discov 1: 317–327

    Article  Google Scholar 

  50. Schittenkopf C, Deco G, Brauer W (1997) Two strategies to avoid overfitting in feedforward networks. Neural Netw 10: 505–516

    Article  Google Scholar 

  51. Senthil Arumugam M, Rao MVC, Chandramohan A (2008) A new and improved version of particle swarm optimization algorithm with global-local best parameters. Knowl Inf Syst 16: 331–357

    Article  Google Scholar 

  52. Sedki A, Ouazar D, El Mazoudi E (2009) Evolving neural network using real coded genetic algorithm for daily rainfall-runoff forecasting. Expert Syst Appl 36: 4523–4527

    Article  Google Scholar 

  53. Sexton RS, Alidaee B, Dorsey RE et al (1998) Global optimization for artificial neural networks: a tabu search application. Eur J Oper Res 106: 570–584

    Article  MATH  Google Scholar 

  54. Sexton RS, Dorsey RE, Johnson JD (1999) Optimization of neural networks: a comparative analysis of the genetic algorithm and simulated annealing. Eur J Oper Res 114: 589–601

    Article  MATH  Google Scholar 

  55. Sexton RS, McMurtrey S, Cleavenger DJ (2006) Knowledge discovery using a neural network simultaneous optimization algorithm on a real world classification problem. Eur J Oper Res 168: 1009–1018

    Article  MATH  Google Scholar 

  56. Shi Y, Eberhart R (1998) A modified particle swarm optimizer. In: Proceeding of the IEEE congress on Evolutionary Computation. pp 69–73

  57. Shi Y, Eberhart R (1998) Parameter selection in particle swarm optimization. Lecture notes in computer science, vol 1447. pp 591–600

  58. Sivagaminathan RK, Ramakrishnan S (2007) A hybrid approach for feature subset selection using neural networks and ant colony optimization. Expert Syst Appl 33: 49–60

    Article  Google Scholar 

  59. Smith M, Bull L (2005) GAP: constructing and selection features with evolutionary computing. In: Ghosh A, Jain LC (eds) Evolutionary computation in data mining. Springer, Heidelberg

    Google Scholar 

  60. Srinivas V, Ramanjaneyulu K (2007) An integrated approach for optimum design of bridge decks using genetic algorithms and artificial neural networks. Adv Eng Softw 38: 475–487

    Article  Google Scholar 

  61. Tsai C-Y, Chou S-Y, Lin S-W et al (2009) Location determination of mobile devices for an indoor WLAN application using a neural network. Knowl Inf Syst 20: 81–93

    Article  Google Scholar 

  62. Verikas A, Bacauskiene M (2002) Feature selection with neural networks. Pattern Recognit Lett 23: 1323–1335

    Article  MATH  Google Scholar 

  63. Wang G, Zhang C, Huang L (2008) A study of classification algorithm for data mining based on hybrid intelligent systems. In: Proceeding of IEEE Ninth ACIS international conference on software engineering, artificial intelligence, networking, and parallel/distributed computing. pp 371–375

  64. Wang T-Y, Huang C-Y (2007) Applying optimized BPN to a chaotic time series problem. Expert Syst Appl 32: 193–200

    Article  Google Scholar 

  65. Wang X, Wang W, Huang Y et al (2008) Design of neural network-based estimator for tool wear modeling in hard turning. J Intell Manuf 19: 383–396

    Article  Google Scholar 

  66. Yamazaki A, de Souto MCP, Ludermir TB (2002) Optimization of neural network weights and architectures for odor recognition using simulated annealing. Proc IEEE Neural Netw 1: 547–552

    Google Scholar 

  67. Yan H, Jiang Y, Zheng J et al (2006) A multilayer perceptorn-based medical decision support system for heart disease diagnosis. Expert Syst Appl 30: 272–281

    Article  Google Scholar 

  68. Yang J, Honavar V (1998) Feature subset selection using a genetic algorithm. IEEE Intell Syst 34: 44–49

    Article  Google Scholar 

  69. Yeung DS, Zeng X-Q (2002) Hidden neuron pruning for multilayer perceptrons using a sensitivity measure. Proc IEEE Mach Learn Cybern 4: 1751–1757

    Article  Google Scholar 

  70. Yu S, Zhu K, Diao F (2008) A dynamic all parameters adaptive BP neural networks model and its application on oil reservoir prediction. Appl Math Comput 195: 66–75

    Article  MATH  MathSciNet  Google Scholar 

  71. Yuen CWM, Wong WK, Qian SQ et al (2009) A hybrid model using genetic algorithm and neural network for classifying garment defects. Expert Syst Appl 36: 2037–2047

    Article  Google Scholar 

  72. Zhang L, Jack LB, Nandi AK (2005) Fault detection using genetic programming. Mech Syst Signal Process 19: 271–289

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Information Management, Chang Gung University, 259 Wen-Hwa 1st Road, Kwei-Shan, Tao-Yuan, 333, Taiwan, ROC

    Shih-Wei Lin & Wen-Jie Wu

  2. Department of Industrial Engineering, National Taiwan University of Science and Technology, No. 43, Sec. 4, Keelung Rd., Taipei, 106, Taiwan, ROC

    Shih-Chieh Chen

  3. Department of Management, Fo-Guang University, No.160, Linwei Rd., Jiaosi Shiang, Yilan, 26247, Taiwan, ROC

    Chih-Hsien Chen

Authors
  1. Shih-Wei Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shih-Chieh Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wen-Jie Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chih-Hsien Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shih-Wei Lin.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lin, SW., Chen, SC., Wu, WJ. et al. Parameter determination and feature selection for back-propagation network by particle swarm optimization. Knowl Inf Syst 21, 249–266 (2009). https://doi.org/10.1007/s10115-009-0242-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-009-0242-y

Keywords

Search

Navigation