Skip to main content
Log in

A novel hybrid sine cosine algorithm for global optimization and its application to train multilayer perceptrons

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The Sine Cosine Algorithm (SCA) is a recently developed efficient metaheuristic algorithm to find the solution of global optimization problems. However, in some circumstances, this algorithm suffers the problem of low exploitation, skipping of true solutions and insufficient balance between exploration and exploitation. Therefore, the present paper aims to alleviate these issues from SCA by proposing an improved variant of SCA called HSCA. The HSCA modifies the search mechanism of classical SCA by including the leading guidance and hybridizing with simulated quenching algorithm. The proposed HSCA is tested on classical benchmark set, standard and complex benchmarks sets IEEE CEC 2014 and CEC 2017 and four engineering optimization problems. In addition to these problems, the HSCA is also used to train multilayer perceptrons as a real-life application. The experimental results and analysis on benchmark problems and real-life application problems demonstrate the superiority of the HSCA as compared to other comparative optimization algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Rao SS (1983) Optimization theory and applications. John Wiley & Sons, Inc., New York, p 550

    Google Scholar 

  2. Zhou F, Yang S, Fujita H, Chen D, Wen C (2019) Deep learning fault diagnosis method based on global optimization GAN for unbalanced data. Knowl-Based Syst. https://doi.org/10.1016/j.knosys.2019.07.008

    Google Scholar 

  3. Lai T, Chen R, Yang C, Li Q, Fujita H, Sadri A, Wang H (2019) Efficient Robust Model Fitting for Multistructure Data Using Global Greedy Search. IEEE transactions on cybernetics. https://doi.org/10.1109/TCYB.2019.2900096

  4. Holland JH (1992) Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence. MIT Press, Cambridge

    Google Scholar 

  5. Storn R, Price K (1997) Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces. J Glob Optim 11(4):341–359

    MathSciNet  MATH  Google Scholar 

  6. Eberhart R, Kennedy J (1995) A new optimizer using particle swarm theory. In Micro Machine and Human Science, 1995. MHS'95., Proceedings of the Sixth International Symposium on (pp. 39-43). IEEE

  7. Karaboga D, Basturk B (2007) A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm. J Glob Optim 39(3):459–471

    MathSciNet  MATH  Google Scholar 

  8. Dorigo, M., & Birattari, M. (2011). Ant colony optimization. In Encyclopedia of machine learning (pp. 36-39). Springer, Boston

  9. Simon D (2008) Biogeography-based optimization. IEEE Trans Evol Comput 12(6):702–713

    Google Scholar 

  10. Geem ZW, Kim JH, Loganathan GV (2001) A new heuristic optimization algorithm: harmony search. Simulation 76(2):60–68

    Google Scholar 

  11. Mirjalili S, Mirjalili SM, Lewis A (2014) Grey wolf optimizer. Adv Eng Softw 69:46–61

    Google Scholar 

  12. Mirjalili S (2015) The ant lion optimizer. Adv Eng Softw 83:80–98

    Google Scholar 

  13. Mirjalili S, Lewis A (2016) The whale optimization algorithm. Adv Eng Softw 95:51–67

    Google Scholar 

  14. Rao RV, Savsani VJ, Vakharia DP (2011) Teaching–learning-based optimization: a novel method for constrained mechanical design optimization problems. Comput Aided Des 43(3):303–315

    Google Scholar 

  15. Askarzadeh A (2016) A novel metaheuristic method for solving constrained engineering optimization problems: crow search algorithm. Comput Struct 169:1–12

    Google Scholar 

  16. Mirjalili S (2015) Moth-flame optimization algorithm: A novel nature-inspired heuristic paradigm. Knowl-Based Syst 89:228–249

    Google Scholar 

  17. Wolpert DH, Macready WG (1997) No free lunch theorems for optimization. IEEE Trans Evol Comput 1(1):67–82

    Google Scholar 

  18. Mirjalili S (2016) SCA: a sine cosine algorithm for solving optimization problems. Knowl-Based Syst 96:120–133

    Google Scholar 

  19. Nenavath, H., Jatoth, R. K., & Das, S. (2018). A synergy of the sine-cosine algorithm and particle swarm optimizer for improved global optimization and object tracking. Swarm and Evolutionary Computation

  20. Yao X, Liu Y, Lin G (1999) Evolutionary programming made faster. IEEE Trans Evol Comput 3(2):82–102

    Google Scholar 

  21. Liang JJ, Qu BY, Suganthan PN (2013) Problem definitions and evaluation criteria for the CEC 2014 special session and competition on single objective real-parameter numerical optimization. Computational Intelligence Laboratory, Zhengzhou University, Zhengzhou China and Technical Report, Nanyang Technological University, Singapore

  22. Awad NH, Ali MZ, Liang JJ, Qu BY, Suganthan PN (2016) Problem definitions and evaluation criteria for the CEC 2017 special session and competition on single objective bound constrained real-parameter numerical optimization. In Technical Report. Nanyang Technological University Singapore

  23. Hafez AI, Zawbaa HM, Emary E, Hassanien AE (2016) Sine cosine optimization algorithm for feature selection. In Innovations in Intelligent Systems and Applications (INISTA), 2016 International Symposium on (pp. 1-5). IEEE

  24. Elfattah MA, Abuelenin S, Hassanien AE, Pan JS (2016) Handwritten arabic manuscript image binarization using sine cosine optimization algorithm. In International Conference on Genetic and Evolutionary Computing (pp. 273-280). Springer, Cham

  25. Yıldız BS, Yıldız AR (2018) Comparison of grey wolf, whale, water cycle, ant lion and sine-cosine algorithms for the optimization of a vehicle engine connecting rod. Materials Testing 60(3):311–315

    Google Scholar 

  26. Sahlol, A. T., Ewees, A. A., Hemdan, A. M., & Hassanien, A. E. (2016, December). Training feedforward neural networks using Sine-Cosine algorithm to improve the prediction of liver enzymes on fish farmed on nano-selenite. In Computer Engineering Conference (ICENCO), 2016 12th International (pp. 35-40). IEEE

  27. Li S, Fang H, Liu X (2018) Parameter optimization of support vector regression based on sine cosine algorithm. Expert Syst Appl 91:63–77

    Google Scholar 

  28. Kumar, V., & Kumar, D. (2017). Data clustering using sine cosine algorithm: Data clustering using SCA. In Handbook of Research on Machine Learning Innovations and Trends (pp. 715-726). IGI Global

  29. Das S, Bhattacharya A, Chakraborty AK (2017) Solution of short-term hydrothermal scheduling using sine cosine algorithm. Soft Comput:1–19

  30. Meshkat, M., & Parhizgar, M. (2017, March). A novel weighted update position mechanism to improve the performance of sine cosine algorithm. In Fuzzy and Intelligent Systems (CFIS), 2017 5th Iranian Joint Congress on (pp. 166-171). IEEE

  31. Elaziz MA, Oliva D, Xiong S (2017) An improved opposition-based sine cosine algorithm for global optimization. Expert Syst Appl 90:484–500

    Google Scholar 

  32. Sindhu R, Ngadiran R, Yacob YM, Zahri NAH, Hariharan M (2017) Sine–cosine algorithm for feature selection with elitism strategy and new updating mechanism. Neural Comput & Applic 28(10):2947–2958

    Google Scholar 

  33. Elaziz MEA, Ewees AA, Oliva D, Duan P, Xiong S (2017) A hybrid method of sine cosine algorithm and differential evolution for feature selection. In International Conference on Neural Information Processing (pp. 145-155). Springer, Cham

  34. Rizk-Allah RM (2018) Hybridizing sine cosine algorithm with multi-orthogonal search strategy for engineering design problems. Journal of Computational Design and Engineering 5(2):249–273

    MathSciNet  Google Scholar 

  35. Turgut OE (2017) Thermal and Economical Optimization of a Shell and Tube Evaporator Using Hybrid Backtracking Search—Sine–Cosine Algorithm. Arab J Sci Eng 42(5):2105–2123

    Google Scholar 

  36. Li N, Li G, Deng Z (2017) An improved sine cosine algorithm based on levy flight. In Ninth International Conference on Digital Image Processing (ICDIP 2017) (Vol. 10420, p. 104204R). International Society for Optics and Photonics

  37. Bairathi D, Gopalani D (2017) Opposition-Based Sine Cosine Algorithm (OSCA) for Training Feed-Forward Neural Networks. In: Signal-Image Technology & Internet-Based Systems (SITIS), 2017 13th International Conference on (pp. 438-444). IEEE

  38. Nayak DR, Dash R, Majhi B, Wang S (2018) Combining extreme learning machine with modified sine cosine algorithm for detection of pathological brain. Comput Electr Eng 68:366–380

    Google Scholar 

  39. Attia AF, El Sehiemy RA, Hasanien HM (2018) Optimal power flow solution in power systems using a novel Sine-Cosine algorithm. Int J Electr Power Energy Syst 99:331–343

    Google Scholar 

  40. Gupta S, Deep K (2018) Improved sine cosine algorithm with crossover scheme for global optimization. Knowl-Based Syst

  41. Gupta S, Deep K (2019) A hybrid self-adaptive sine cosine algorithm with opposition based learning. Expert Syst Appl 119:210–230

    Google Scholar 

  42. Chen H, Jiao S, Heidari AA, Wang M, Chen X, Zhao X (2019) An opposition-based sine cosine approach with local search for parameter estimation of photovoltaic models. Energy Convers Manag 195:927–942

    Google Scholar 

  43. Lin A, Wu Q, Heidari AA, Xu Y, Chen H, Geng W, Li C (2019) Predicting Intentions of Students for Master Programs using a Chaos-induced Sine Cosine-based Fuzzy K-Nearest Neighbor Classifier. IEEE Access

  44. Belazzoug M, Touahria M, Nouioua F, Brahimi M (2019) An improved sine cosine algorithm to select features for text categorization. Journal of King Saud University-Computer and Information Sciences. https://doi.org/10.1016/j.jksuci.2019.07.003

  45. Gupta S, Deep K (2019) Hybrid sine cosine artificial bee colony algorithm for global optimization and image segmentation. Neural Comput & Applic. https://doi.org/10.1007/s00521-019-04465-6

  46. Issa M, Hassanien AE, Oliva D, Helmi A, Ziedan I, Alzohairy A (2018) ASCA-PSO: Adaptive sine cosine optimization algorithm integrated with particle swarm for pairwise local sequence alignment. Expert Syst Appl 99:56–70

    Google Scholar 

  47. Tawhid MA, Savsani V (2019) Multi-objective sine-cosine algorithm (MO-SCA) for multi-objective engineering design problems. Neural Comput & Applic 31(2):915–929

    Google Scholar 

  48. Gupta S, Deep K (2019) Enhanced leadership-inspired grey wolf optimizer for global optimization problems. Eng Comput:1–24. https://doi.org/10.1007/s00366-019-00795-0

  49. Shi Y, Eberhart R (1998) A modified particle swarm optimizer. In Evolutionary Computation Proceedings, 1998. IEEE World Congress on Computational Intelligence, The 1998 IEEE International Conference on (pp. 69-73). IEEE

  50. Derrac J, García S, Molina D, Herrera F (2011) A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm and Evolutionary Computation 1(1):3–18

    Google Scholar 

  51. Deb K (2000) An efficient constraint handling method for genetic algorithms. Comput Methods Appl Mech Eng 186(2-4):311–338

    MATH  Google Scholar 

  52. Das S, Suganthan PN (2010) Problem definitions and evaluation criteria for CEC 2011 competition on testing evolutionary algorithms on real world optimization problems. Jadavpur University, Nanyang Technological University, Kolkata

    Google Scholar 

  53. Mirjalili S, Gandomi AH, Mirjalili SZ, Saremi S, Faris H, Mirjalili SM (2017) Salp Swarm Algorithm: A bio-inspired optimizer for engineering design problems. Adv Eng Softw 114:163–191

    Google Scholar 

  54. Liang JJ, Qin AK, Suganthan PN, Baskar S (2006) Comprehensive learning particle swarm optimizer for global optimization of multimodal functions. IEEE Trans Evol Comput 10(3):281–295

    Google Scholar 

  55. Van Laarhoven PJ, Aarts EH (1987) Simulated annealing. In Simulated Annealing: Theory and applications (pp. 7-15). Springer, Dordrecht

  56. Auger A, Hansen N (2005) A restart CMA evolution strategy with increasing population size. In Evolutionary Computation, 2005. The 2005 IEEE Congress on (Vol. 2, pp. 1769-1776). IEEE

  57. Sandgren E (1990) Nonlinear integer and discrete programming in mechanical design optimization. J Mech Des 112(2):223–229

    Google Scholar 

  58. Wu SJ, Chow PT (1995) Genetic algorithms for nonlinear mixed discrete-integer optimization problems via meta-genetic parameter optimization. Engineering Optimization+ A35 24(2):137–159

    Google Scholar 

  59. Gandomi AH, Yang XS, Alavi AH (2013) Cuckoo search algorithm: a metaheuristic approach to solve structural optimization problems. Eng Comput 29(1):17–35

    Google Scholar 

  60. Deb K, Goyal M (1996) A combined genetic adaptive search (GeneAS) for engineering design. Computer Science and Informatics 26:30–45

    Google Scholar 

  61. Kannan BK, Kramer SN (1994) An augmented Lagrange multiplier based method for mixed integer discrete continuous optimization and its applications to mechanical design. J Mech Des 116(2):405–411

    Google Scholar 

  62. Sadollah A, Bahreininejad A, Eskandar H, Hamdi M (2013) Mine blast algorithm: A new population based algorithm for solving constrained engineering optimization problems. Appl Soft Comput 13(5):2592–2612

    Google Scholar 

  63. Fleury C, Braibant V (1986) Structural optimization: a new dual method using mixed variables. Int J Numer Methods Eng 23(3):409–428

    MathSciNet  MATH  Google Scholar 

  64. Chickermane H, Gea HC (1996) Structural optimization using a new local approximation method. Int J Numer Methods Eng 39(5):829–846

    MathSciNet  MATH  Google Scholar 

  65. Arora JS (2004) Introduction to optimum design. Elsevier

  66. Belegundu AD, Arora JS (1985) A study of mathematical programming methods for structural optimization. Part I: Theory. Int J Numer Methods Eng 21(9):1583–1599

    MATH  Google Scholar 

  67. Coello CAC, Montes EM (2002) Constraint-handling in genetic algorithms through the use of dominance-based tournament selection. Adv Eng Inform 16(3):193–203

    Google Scholar 

  68. He Q, Wang L (2007) An effective co-evolutionary particle swarm optimization for constrained engineering design problems. Eng Appl Artif Intell 20(1):89–99

    Google Scholar 

  69. Coello CAC (2000) Use of a self-adaptive penalty approach for engineering optimization problems. Comput Ind 41(2):113–127

    Google Scholar 

  70. Rashedi E, Nezamabadi-Pour H, Saryazdi S (2009) GSA: a gravitational search algorithm. Inf Sci 179(13):2232–2248

    MATH  Google Scholar 

  71. Huang FZ, Wang L, He Q (2007) An effective co-evolutionary differential evolution for constrained optimization. Appl Math Comput 186(1):340–356

    MathSciNet  MATH  Google Scholar 

  72. Mezura-Montes E, Coello CAC (2008) An empirical study about the usefulness of evolution strategies to solve constrained optimization problems. Int J Gen Syst 37(4):443–473

    MathSciNet  MATH  Google Scholar 

  73. Mahdavi M, Fesanghary M, Damangir E (2007) An improved harmony search algorithm for solving optimization problems. Appl Math Comput 188(2):1567–1579

    MathSciNet  MATH  Google Scholar 

  74. Belegundu AD (1983) A study of mathematical programming methods for structural optimization

  75. Suratgar AA, Tavakoli MB, Hoseinabadi A (2005) Modified Levenberg-Marquardt method for neural networks training. World Acad Sci Eng Technol 6(1):46–48

    Google Scholar 

  76. Moré JJ (1978) The Levenberg-Marquardt algorithm: implementation and theory. In Numerical analysis (pp. 105-116). Springer, Berlin, Heidelberg

    Google Scholar 

  77. Hagan MT, Menhaj MB (1994) Training feedforward networks with the Marquardt algorithm. IEEE Trans Neural Netw 5(6):989–993

    Google Scholar 

  78. Leonard J, Kramer MA (1990) Improvement of the backpropagation algorithm for training neural networks. Comput Chem Eng 14(3):337–341

    Google Scholar 

  79. Wienholt W (1993) Minimizing the system error in feedforward neural networks with evolution strategy. In ICANN’93 (pp. 490-493). Springer, London

    Google Scholar 

  80. Parmee IC (2012) Evolutionary and adaptive computing in engineering design. Springer Science & Business Media, Berlin

    Google Scholar 

  81. Zhang JR, Zhang J, Lok TM, Lyu MR (2007) A hybrid particle swarm optimization–back-propagation algorithm for feedforward neural network training. Appl Math Comput 185(2):1026–1037

    MATH  Google Scholar 

  82. Blake C, Merz CJ (1998) UCI Repository of Machine Learning Databases. http://www.ics.uci.edu/~mlearn/MLRepository.html (accessed)

  83. Mirjalili S (2015) How effective is the Grey Wolf optimizer in training multi-layer perceptrons. Appl Intell 43(1):150–161

    Google Scholar 

  84. UCI Machine Learning Repository. http://archive.ics.uci.edu/ml/datasets.html (accessed 2012)

  85. Mangasarian OL, Wolberg WH (1990) Cancer Diagnosis via Linear Programming. University of Wisconsin-Madison, Computer Sciences Department, Madison

    MATH  Google Scholar 

Download references

Acknowledgements

The first author gratefully acknowledges to the Ministry of Human Resource and Development (MHRD), Government of India for their financial support. Grant No. MHR-02-41-113-429.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Shubham Gupta.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

Table 15 Comparison of different statistics obtained from the proposed HSCA and classical version of SCA on benchmark set I
Table 16 Comparison of different statistics obtained from the proposed HSCA and classical version of SCA on benchmark set II with dimension 10
Table 17 Comparison of different statistics obtained from the proposed HSCA and classical version of SCA on benchmark set II with dimension 30
Table 18 Comparison of different statistics obtained from the proposed HSCA and classical version of SCA on benchmark set III with dimension 10
Table 19 Comparison of different statistics obtained from the proposed HSCA and classical version of SCA on benchmark set III with dimension 30

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gupta, S., Deep, K. A novel hybrid sine cosine algorithm for global optimization and its application to train multilayer perceptrons. Appl Intell 50, 993–1026 (2020). https://doi.org/10.1007/s10489-019-01570-w

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-019-01570-w

Keywords

Navigation