Skip to main content
SpringerLink
Log in

Hybrid PSO (SGPSO) with the Incorporation of Discretization Operator for Training RBF Neural Network and Optimal Feature Selection

  • Research Article-Computer Engineering and Computer Science
  • Published:
Arabian Journal for Science and Engineering Aims and scope Submit manuscript

Abstract

Particle swarm optimization (PSO) is a computational method that emerged recently based on swarm intelligence techniques for resolving optimization complications. The popularity and wide acceptance of PSO among the research community are because of its simplicity in implementation and fewer parameters to fine-tune. The velocity and position of particles are adjusted in the typical PSO algorithm based on their personal best and global best positions. The concept of updating the particle’s velocity concerning personal and global optimal positions is simple and appealing. However, this learning technique might result in the “oscillation” and “two-steps-forward, one-step-back” phenomena. To avoid such imperfections associated with the standard PSO, we have suggested a hybrid PSO algorithm and have named it single-guided-PSO (SGPSO) that incorporates the discretization operator with standard PSO to effectively stabilize the oscillation effect and achieve a smoother balance between exploration and exploitation capability with an enhanced convergence rate towards global optima. Following IEEE-CEC-2014, a set of 14 basic functions and 30 advanced standard functions were chosen to validate the proficiency of the proposed SGPSO technique. The results of the suggested strategy have also been compared with some state-of-the-art existing meta-heuristic methodologies in literature. Two nonparametric tests have also been performed to substantiate statistical significance. In addition, the suggested approach SGPSO has been used to train radial basis function neural network by selecting datasets from the UCI repository. Furthermore, the same SGPSO is utilized to select optimum features from benchmark datasets, simultaneously maintaining the accuracy to minimize the complexity of neural networks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Abbreviations

HCEF:

High-conditioned elliptic function

BCF:

Bent-cigar function

DF:

Discuss function

RBF:

Rosenbrock’s function

AF:

Ackley’s function

WF:

Weierstrass function

GF:

Griewank’s function

RF:

Rastrigin’s function

SF:

Schwefel’s function

KF:

Katsuura function

HCF:

HappyCat function

HGBF:

HGBat function

EGRF:

Expanded Griewank’s plus Rosenbrock’s function

ESF6:

Expanded Scaffer’s F6 function

Iono:

Ionosphere

WBC:

Wisconsin breast cancer

Pima:

Pima Indian diabetes

Heart:

Heart

E. coli:

E. coli

liver:

Liver

Hepa:

Hepatitis

PDC:

Parkinson’s disease classification

QAR:

QSAR androgen receptor

MED:

Malware executable detection

References

  1. Hashim, F.A.; Hussain, K.; Houssein, E.H.; Mabrouk, M.S.; Al-Atabany, W.: Archimedes optimization algorithm: a new metaheuristic algorithm for solving optimization problems. Appl. Intell. 51(3), 1531–1551 (2021)

    MATH  Google Scholar 

  2. Yazdani, D.; Cheng, R.; Yazdani, D.; Branke, J.; Jin, Y.; Yao, X.: A survey of evolutionary continuous dynamic optimization over two decades—part B. IEEE Trans. Evol. Comput. 25(4), 630–650 (2021)

    Google Scholar 

  3. Hare, W.; Jarry-Bolduc, G.: A deterministic algorithm to compute the cosine measure of a finite positive spanning set. Optim. Lett. 14(6), 1305–1316 (2020)

    MathSciNet  MATH  Google Scholar 

  4. Krishna, M.M.; Panda, N.; Majhi, S.K.: Solving traveling salesman problem using hybridization of rider optimization and spotted hyena optimization algorithm. Expert Syst. Appl. 183, 115353 (2021)

    Google Scholar 

  5. Panda, N.; Majhi, S.K.; Pradhan, R.: A hybrid approach of spotted Hyena optimization integrated with quadratic approximation for training wavelet neural network. Arab. J. Sci. Eng. 47, 1–17 (2022)

    Google Scholar 

  6. Deep, K.; Bansal, J.C.: Mean particle swarm optimisation for function optimisation. Int. J. Comput. Intell. Stud. 1(1), 72–92 (2009)

    Google Scholar 

  7. Wolpert, D.H.; Macready, W.G.: No free lunch theorems for optimization. IEEE Trans. Evol. Comput. 1(1), 67–82 (1997)

    Google Scholar 

  8. Panda, N.; Majhi, S.K.: Oppositional salp swarm algorithm with mutation operator for global optimization and application in training higher order neural networks. Multimed. Tools Appl. 80(28), 35415–35439 (2021)

    Google Scholar 

  9. Ab Aziz, M.F.; Mostafa, S.A.; Foozy, C.F.M.; Mohammed, M.A.; Elhoseny, M.; Abualkishik, A.Z.: Integrating Elman recurrent neural network with particle swarm optimization algorithms for an improved hybrid training of multidisciplinary datasets. Expert Syst. Appl. 183, 115441 (2021)

    Google Scholar 

  10. Panda, N.; Majhi, S.K.; Singh, S.; Khanna, A.: Oppositional spotted hyena optimizer with mutation operator for global optimization and application in training wavelet neural network. J. Intell. Fuzzy Syst. 38(5), 6677–6690 (2020)

    Google Scholar 

  11. Zhan, Z.H.; Zhang, J.; Li, Y.; Shi, Y.H.: Orthogonal learning particle swarm optimization. IEEE Trans. Evol. Comput. 15(6), 832–847 (2010)

    Google Scholar 

  12. Liu, Z.; Deshpande, S.; Hui, Q.: Quantized particle swarm optimization: an improved algorithm based on group effect and its convergence analysis. In: Dynamic Systems and Control Conference, vol. 54754, pp. 321–327 (2011)

  13. Kalita, D.J.; Singh, S.: SVM hyper-parameters optimization using quantized multi-PSO in dynamic environment. Soft. Comput. 24(2), 1225–1241 (2020)

    Google Scholar 

  14. Li, Q.; Li, S.: Optimization of artificial CNN based on swarm intelligence algorithm. J. Intell. Fuzzy Syst. 40, 1–11 (2021)

    Google Scholar 

  15. Singh, P.; Chaudhury, S.; Panigrahi, B.K.: Hybrid MPSO-CNN: multi-level particle swarm optimized hyper parameters of convolutional neural network. Swarm Evol. Comput. 63, 100863 (2021)

    Google Scholar 

  16. Chen, D.: Evaluation model of physical education effect: on the application of radial basis function-particle swarm optimization neural network (RBFNN-PSO). Comput. Intell. Neurosci. (2021). https://doi.org/10.1155/2021/6819493

    Article  Google Scholar 

  17. Jabbar, S.; A.: Hybrid PSO-RBFNN and proposed algorithms of DDDWT for the heart disease classification. Eng. Technol. J. 39(4), 520–527 (2021)

    Google Scholar 

  18. Srinivasulu, M.: Multi-lead ECG signal analysis using RBFNN-MSO algorithm. Int. J. Speech Technol. 24(2), 341–350 (2021)

    Google Scholar 

  19. Luo, J.; Zhou, D.; Jiang, L.; Ma, H.: A particle swarm optimization based multiobjective memetic algorithm for high-dimensional feature selection. Memetic Comput. 14, 1–17 (2022)

    Google Scholar 

  20. Zhang, L.; Zhao, Z.; Zhang, D.; Luo, C.; Li, C.: Particle swarm optimization pattern recognition neural network for transmission lines faults classification. Intell. Data Anal. 26(1), 189–203 (2022)

    Google Scholar 

  21. Lawrence, T.; Zhang, L.; Lim, C.P.; Phillips, E.J.: Particle swarm optimization for automatically evolving convolutional neural networks for image classification. IEEE Access 9, 14369–14386 (2021)

    Google Scholar 

  22. Kumari, K.; Singh, J.P.; Dwivedi, Y.K.; Rana, N.P.: Multi-modal aggression identification using convolutional neural network and binary particle swarm optimization. Futur. Gener. Comput. Syst. 118, 187–197 (2021)

    Google Scholar 

  23. Ding, L.; Zhang, X.Y.; Wu, D.Y.; Liu, M.L.: Application of an extreme learning machine network with particle swarm optimization in syndrome classification of primary liver cancer. J. Integr. Med. 19(5), 395–407 (2021)

    Google Scholar 

  24. Surantha, N.; Lesmana, T.F.; Isa, S.M.: Sleep stage classification using extreme learning machine and particle swarm optimization for healthcare big data. J. Big Data 8(1), 1–17 (2021)

    Google Scholar 

  25. Vijayashree, J.; Sultana, H.P.: A machine learning framework for feature selection in heart disease classification using improved particle swarm optimization with support vector machine classifier. Program. Comput. Softw. 44(6), 388–397 (2018)

    Google Scholar 

  26. Narayanan, S.J.; Soundrapandiyan, R.; Perumal, B.; Baby, C.J.: Emphysema medical image classification using fuzzy decision tree with fuzzy particle swarm optimization clustering, p. 305–313. In Smart Intelligent Computing and Applications Springer, Singapore (2019)

    Google Scholar 

  27. Talita, A.S.; Nataza, O.S.; Rustam, Z.: Nave bayes classifier and particle swarm optimization feature selection method for classifying intrusion detection system dataset. J. Phys. Conf. Ser. 1752, 012021 (2021)

    Google Scholar 

  28. Yang, L.; Chen, H.: Fault diagnosis of gearbox based on RBF-PF and particle swarm optimization wavelet neural network. Neural Comput. Appl. 31(9), 4463–4478 (2019)

    Google Scholar 

  29. Zheng, J.; Li, Z.L.; Dou, B.; Lu, C.: Multi-objective cellular particle swarm optimization and RBF for drilling parameters optimization. Math. Biosci. Eng. MBE 16(3), 1258–1279 (2019)

    MathSciNet  MATH  Google Scholar 

  30. Wang, H.; Liu, K.; Wu, Y.; Wang, S.; Zhang, Z.; Li, F.; Yao, J.: Image reconstruction for electrical impedance tomography using radial basis function neural network based on hybrid particle swarm optimization algorithm. IEEE Sens. J. 21(2), 1926–1934 (2020)

    Google Scholar 

  31. Koupaei, J.A.; Firouznia, M.; Hosseini, S.M.M.: Finding a good shape parameter of RBF to solve PDEs based on the particle swarm optimization algorithm. Alex. Eng. J. 57(4), 3641–3652 (2018)

    Google Scholar 

  32. Song, X.F.; Zhang, Y.; Gong, D.W.; Gao, X.Z.: A fast hybrid feature selection based on correlation-guided clustering and particle swarm optimization for high-dimensional data. IEEE Trans. Cybern. (2021). https://doi.org/10.1109/TCYB.2021.3061152

    Article  Google Scholar 

  33. Boussaïd, I.; Lepagnot, J.; Siarry, P.: A survey on optimization metaheuristics. Inf. Sci. 237, 82–117 (2013)

    MathSciNet  MATH  Google Scholar 

  34. Dokeroglu, T.; Sevinc, E.; Kucukyilmaz, T.; Cosar, A.: A survey on new generation metaheuristic algorithms. Comput. Ind. Eng. 137, 106040 (2019)

    Google Scholar 

  35. Hussain, K.; Mohd Salleh, M.N.; Cheng, S.; Shi, Y.: Metaheuristic research: a comprehensive survey. Artif. Intell. Rev. 52(4), 2191–2233 (2019)

    Google Scholar 

  36. Li, J.; Cheng, K.; Wang, S.; Morstatter, F.; Trevino, R.P.; Tang, J.; Liu, H.: Feature selection: a data perspective. ACM Comput. Surv. (CSUR) 50(6), 1–45 (2017)

    Google Scholar 

  37. Xue, B.; Zhang, M.; Browne, W.N.; Yao, X.: A survey on evolutionary computation approaches to feature selection. IEEE Trans. Evol. Comput. 20(4), 606–626 (2015)

    Google Scholar 

  38. Dokeroglu, T.; Deniz, A.; Kiziloz, H.E.: A Comprehensive Survey on Recent Metaheuristics for Feature Selection. Neurocomputing 494, 269–296 (2022)

    Google Scholar 

  39. Kennedy, J.; Eberhart, R.: Particle swarm optimization. In: Proceedings of ICNN'95-International Conference on Neural Networks, vol. 4, pp. 1942–1948. IEEE (1995)

  40. Parsopoulos, K.E.; Vrahatis, M.N.: On the computation of all global minimizers through particle swarm optimization. IEEE Trans. Evol. Comput. 8(3), 211–224 (2004)

    Google Scholar 

  41. Van den Bergh, F.; Engelbrecht, A.P.: A cooperative approach to particle swarm optimization. IEEE Trans. Evol. Comput. 8(3), 225–239 (2004)

    Google Scholar 

  42. Liang, J.J.; Qu, B.Y.; Suganthan, P.N.: Problem definitions and evaluation criteria for the CEC 2014 special session and competition on single objective real-parameter numerical optimization. Comput. Intell. Lab. Zhengzhou Univ. Zhengzhou China Tech. Rep. Nanyang Technol. Univ. Singapore 635, 490 (2013)

    Google Scholar 

  43. Yang, X.S.; Deb, S.: Engineering optimisation by cuckoo search. Int. J. Math. Model. Numer. Optim. 1(4), 330–343 (2010)

    MATH  Google Scholar 

  44. Yang, X.S.: Firefly algorithm, stochastic test functions and design optimisation. Int. J. Bio-Inspir. Com. 2(2), 78–84 (2010)

    Google Scholar 

  45. Mirjalili, S.: Moth-flame optimization algorithm: a novel nature-inspired heuristic paradigm. Knowl. Based Syst. 89, 228–249 (2015)

    Google Scholar 

  46. Mirjalili, S.: SCA: a sine cosine algorithm for solving optimization problems. Knowl. Based Syst. 96, 120–133 (2016)

    Google Scholar 

  47. Mirjalili, S.; Lewis, A.: The whale optimization algorithm. Adv. Eng. Softw. 95, 51–67 (2016)

    Google Scholar 

  48. Friedman, M.: The use of ranks to avoid the assumption of normality implicit in the analysis of variance. J. Am. Stat. Assoc. 32(200), 675–701 (1937)

    MATH  Google Scholar 

  49. Friedman, M.: A comparison of alternative tests of significance for the problem of m rankings. Ann. Math. Stat. 11(1), 86–92 (1940)

    MathSciNet  MATH  Google Scholar 

  50. F Distribution Table, 2018. Retrieved from http://www.socr.ucla.edu/applets.dir/f_table.html

  51. Normal Distribution Table. Retrievedfrom http://math.arizona.edu/~rsims/ma464/standardnormaltable.pdf

  52. Broomhead, D.S.; Lowe, D.: Radial basis functions, multi-variable functional interpolation and adaptive networks. Royal Signals and Radar Establishment Malvern, United Kingdom (1988)

    MATH  Google Scholar 

  53. Cover, T.M.: Geometrical and statistical properties of systems of linear inequalities with applications in pattern recognition. IEEE Trans. Electron. Comput. 3, 326–334 (1965)

    MATH  Google Scholar 

  54. Panda, N.; Majhi, S.K.: How effective is spotted Hyena optimizer for training multilayer perceptrons. Int. J. Recent Technol. Eng. 8, 4915–4927 (2019)

    Google Scholar 

  55. Mohanty, A.K.; Panda, M.; Mishra, D.: Foreign exchange price prediction using artificial neural network optimized by salp swarm algorithm. In: Advances in Distributed Computing and Machine Learning. Springer, Singapore (2022)

  56. Panda, N.; Majhi, S.K.: Effectiveness of swarm-based metaheuristic algorithm in data classification using pi-sigma higher order neural network. In: Progress in Advanced Computing and Intelligent Engineering (pp. 77-88). Springer, Singapore (2021)

  57. Powell, M.J.: Radial basis functions for multivariable interpolation: a review. Algorithms for approximation (1987)

  58. Shanmukhi, K.; Vundavilli, P.R.; Surekha, B.: Modeling of ECDM micro-drilling process using GA-and PSO-trained radial basis function neural network. Soft. Comput. 19(8), 2193–2202 (2015)

    Google Scholar 

  59. Dyn, N.; Micchelli, C.A.: Interpolation by sums of radial functions. Numer. Math. 58(1), 1–9 (1990)

    MathSciNet  MATH  Google Scholar 

  60. Jiang, Q.; Zhu, L.; Shu, C.; Sekar, V.: Multilayer perceptron neural network activated by adaptive Gaussian radial basis function and its application to predict lid-driven cavity flow. Acta Mech. Sinica 37, 1–16 (2022)

    MathSciNet  Google Scholar 

  61. Van der Merwe, D.W.; Engelbrecht, A.P.: Data clustering using particle swarm optimization. In: The 2003 Congress on Evolutionary Computation, 2003. CEC'03, vol. 1, pp. 215–220. IEEE (2003)

  62. Bache, K.; Lichman, M.: UCI Machine Learning Repository [https://archive.ics.uci.edu/ml]. Irvine, CA: University of California. School of information and computer science, vol. 28 (2013)

  63. Yu, J.; Nguyen, T.T.T.; Lai, Q.A.; Ngo, T.G.; Dao, T.K.: An optimizing parameters and feature selection in SVM based on improved cockroach swarm optimization. In: Advances in Intelligent Information Hiding and Multimedia Signal Processing, pp. 349–357. Springer, Singapore (2021)

  64. Bansal, S.R.; Wadhawan, S.; Goel, R.: mRMR-PSO: a hybrid feature selection technique with a multiobjective approach for sign language recognition. Arab. J. Sci. Eng. 47, 1–16 (2022)

    Google Scholar 

  65. Rostami, O.; Kaveh, M.: Optimal feature selection for SAR image classification using biogeography-based optimization (BBO), artificial bee colony (ABC) and support vector machine (SVM): a combined approach of optimization and machine learning. Comput. Geosci. 25(3), 911–930 (2021)

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of CSIT, Institute of Technical Education and Research, Siksha ‘O’ Anusandhan (Deemed to Be) University, Bhubaneswar, India

    Ajit Kumar Mahapatra

  2. School of Computer Engineering, KIIT Deemed to Be University, Bhubaneswar, Odisha, 751024, India

    Nibedan Panda

  3. Department of CSE, Institute of Technical Education and Research, Siksha ‘O’ Anusandhan (Deemed to Be) University, Bhubaneswar, India

    Binod Kumar Pattanayak

Authors
  1. Ajit Kumar Mahapatra
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nibedan Panda
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Binod Kumar Pattanayak
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nibedan Panda.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Human or Animal Rights

Furthermore, the research study did not involve human participants and/or animals.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mahapatra, A.K., Panda, N. & Pattanayak, B.K. Hybrid PSO (SGPSO) with the Incorporation of Discretization Operator for Training RBF Neural Network and Optimal Feature Selection. Arab J Sci Eng 48, 9991–10019 (2023). https://doi.org/10.1007/s13369-022-07408-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13369-022-07408-x

Keywords

Search

Navigation