Skip to main content
SpringerLink
Log in

Hybrid Nelder–Mead Algorithm and Dragonfly Algorithm for Function Optimization and the Training of a Multilayer Perceptron

  • Research Article - Computer Engineering and Computer Science
  • Published:
Arabian Journal for Science and Engineering Aims and scope Submit manuscript

Abstract

Dragonfly algorithm (DA) is a new optimization technique based on swarm intelligence. DA simulates the static and dynamic swarming behaviors of dragonflies in nature. The search pattern of DA consists of two essential phases: exploration and exploitation that are inspired by the survival rule of dragonflies in navigating, searching for food and fleeing enemies when dynamically or statistically swarming. This method is straightforward to implement and is efficient in solving real-world problems. However, an excessive number of social interactions in DA may result in low solution accuracy, easy stagnation at local optima and an imbalance between exploration and exploitation. To overcome these deficiencies, an improved Nelder–Mead algorithm is added to the conventional DA (INMDA) to strengthen its local explorative capability and avoid the possibility of falling into local optima. Simulation experiments were conducted on several well-known benchmark functions with different dimensions. In addition, the three classic classification problems are utilized to benchmark the performance of the proposed algorithm in training a multilayer perceptron. The experimental results and statistical significance show that the performance of the proposed INMDA is superior to that of the other algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Eberhart, R.; Kennedy, J.: Particle swarm optimization. In: Proceeding IEEE International Conference on Neural Networks, vol. 4, pp. 1942-1948. Perth, Australia, Piscat-away (1995)

  2. Yelghi, A.; Köse, C.: A modified firefly algorithm for global minimum optimization. Appl. Soft Comput. 62, 29–44 (2018)

    Article  Google Scholar 

  3. Mirjalili, S.; Lewis, A.: The whale optimization algorithm. Adv. Eng. Softw. 95, 51–67 (2016)

    Article  Google Scholar 

  4. Mirjalili, S.; Mirjalili, S.M.; Lewis, A.: Grey wolf optimizer. Adv. Eng. Softw. 69, 46–61 (2014)

    Article  Google Scholar 

  5. Fausto, F.; Cuevas, E.; Valdivia, A.; González, A.: A global optimization algorithm inspired in the behavior of selfish herds. Biosystems 160, 39–55 (2017)

    Article  Google Scholar 

  6. Mirjalili, S.: Dragonfly algorithm: a new meta-heuristic optimization technique for solving single-objective, discrete, and multi-objective problems. Neural Comput. Appl. 27, 1053–1073 (2016)

    Article  Google Scholar 

  7. Blum, C.; Li, X.: Swarm intelligence in optimization. In: Blum, C., Merkle, D. (eds.) Swarm Intelligence: Introduction and Applications, pp. 43–85. Springer, Berlin (2008)

    Chapter  Google Scholar 

  8. Holland, J.H.: Genetic algorithms. Sci. Am. 267, 66–72 (1992)

    Article  Google Scholar 

  9. Bou-Daher, E.; Hoorfar, A.: Electromagnetic optimization using mixed-parameter and multiobjective covariance matrix adaptation evolution strategy. IEEE Trans. Antennas Propag. 63, 1712–1724 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  10. Duan, Q.: A global optimization strategy for efficient and effective calibration of hydrologic models. Ph.D. dissertation, University of Arizona, Tucson, AZ, USA (1991)

  11. Storn, R.; Price, K.: Differential evolution—a simple and efficient heuristic for global optimization over continuous spaces. J. Global Optim. 11, 341–59 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  12. Rashedi, E.; Nezamabadi-Pour, H.; Saryazdi, S.: GSA: a gravitational search algorithm. Inf. Sci. 179, 2232–48 (2009)

    Article  MATH  Google Scholar 

  13. Kirkpatrick Jr., S.; Gelatt, C.D.; Vecchi, M.P.: Optimization by simulated annealing. Science 220, 671–80 (1983)

    Article  MathSciNet  MATH  Google Scholar 

  14. Lee, J.; Kwak, J.; Lee, H.-W.; Shroff, N.B.: Finding minimum node separators: a Markov chain Monte Carlo method. Reliab. Eng. Syst. Saf. 178, 225–235 (2018)

    Article  Google Scholar 

  15. Tolson, B.A.; Shoemaker, C.A.: Dynamically dimensioned search algorithm for computationally efficient watershed model calibration. Water Resour. Res. 413, W01413(1–6) (2007)

    Google Scholar 

  16. Kaveh, A.; Khayatazad, M.: A new meta-heuristic method: ray optimization. Comput. Struct. 112, 283–94 (2012)

    Article  Google Scholar 

  17. Gholizadeh, S.; Poorhoseini, H.: Optimum design of steel frame structures by a modified dolphin echolocation algorithm. Struct. Eng. Mech. 55, 535–554 (2015)

    Article  Google Scholar 

  18. Gholizadeh, S.; Shahrezaei, A.M.: Optimal placement of steel plate shear walls for steel frames by bat algorithm. Struct. Des. Tall Spec. 24, 1–18 (2015)

    Article  Google Scholar 

  19. Gholizadeh, S.; Seyedpoor, S.M.: Shape optimization of arch dams by metaheuristics and neural networks for frequency constraints. Sci. Iran. 18, 1020–1027 (2011)

    Article  Google Scholar 

  20. Gholizadeh, S.; Barzegar, A.: Shape optimization of structures for frequency constraints by sequential harmony search algorithm. Eng. Optim. 45, 627–646 (2013)

    Article  MathSciNet  Google Scholar 

  21. Behrangi, A.; Vrugt, J.; Khakbaz, B.; Sorooshian, S.: Comment on ‘dynamically dimensioned search algorithm for computationally efficient watershed model calibration’ by Bryan A. Tolson and Christine A. Shoemaker. Water Resour. Res. 44, W12603 (2008)

    Article  Google Scholar 

  22. Özekici, S.; Soyer, R.: Network reliability assessment in a random environment. Naval Res. Logist. 50, 574–91 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  23. Green, P.; Worden, K.: Bayesian and Markov chain Monte Carlo methods for identifying nonlinear systems in the presence of uncertainty. Philos. Trans. R. Soc. A 373, 20140405 (2015)

    Article  Google Scholar 

  24. Nunoo, C.; Mrawira, D.: Shuffled complex evolution algorithms in infrastructure works programming. J. Comput. Civ. Eng. 18, 257–266 (2004)

    Article  Google Scholar 

  25. Gabriel, P.; Marcus, R.; Leonardo, B.; Luciana, B.: Simulated annealing for the machine reassignment problem. Ann. Oper. Res. 242, 93–114 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  26. Yang, H.; Kang, L.S.; Chen, Y.P.: A gene-based genetic algorithm for TSP. Chin. J. Comput. 26, 1753–1758 (2003)

    MathSciNet  Google Scholar 

  27. Gomes, H.M.: Truss optimization with dynamic constraints using a particle swarm algorithm. Expert Syst. Appl. 38, 957–968 (2011)

    Article  Google Scholar 

  28. Tirumala, S.S.: A quantum-inspired evolutionary algorithm using gaussian distribution-based quantization. Arab. J. Sci. Eng. 43, 471–482 (2018)

    Article  MATH  Google Scholar 

  29. Suresh, V.; Sreejith, S.: Generation dispatch of combined solar thermal systems using dragonfly algorithm. Computing 99, 59–80 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  30. Mohammad, J.; Mohammad, H.B.C.: Using dragonfly algorithm for optimization of orthotropic infinite plates with a quasi-triangular cut-out. Eur. J. Mech. A Solid 66, 1–14 (2017)

    MathSciNet  MATH  Google Scholar 

  31. Hema, C.; Sankar, S.; Sandhya.: Energy efficient cluster based protocol to extend the rfid network lifetime using dragonfly algorithm. In: 2016 International Conference on Communication and Signal Processing (ICCSP), pp. 0530–0534 (2016)

  32. Daely, P.T.; Shin, S.Y.: Range based wireless node localization using dragonfly algorithm. In: 2016 Eighth International Conference on Ubiquitous and Future Networks (ICUFN), pp. 1012–1015 (2016)

  33. Sambandam, R.K.; Jayaraman, S.: Self-adaptive dragonfly based optimal thresholding for multilevel segmentation of digital images. J. King Saud Univ. Comput. Inf. Sci. (2016) https://doi.org/10.1016/j.jksuci.2016.11.002

  34. Raman, G.; Raman, G.; Manickam, C.; Ganesan, S.L.: Dragonfly algorithm based global maximum power point tracker for photovoltaic systems. Adv. Swarm Intell. 9712, 211–219 (2016)

    Google Scholar 

  35. Sree Ranjini, K.S.; Murugan, S.: Memory based Hybrid Dragonfly Algorithm for numerical optimization problems. Expert Syst. Appl. 83, 63–78 (2017)

    Article  Google Scholar 

  36. Majdi, M.; Mirjalili, M.S.: Hybrid Whale Optimization Algorithm with simulated annealing for feature selection. Neurocomputing 260, 302–312 (2017)

    Article  Google Scholar 

  37. Shilaja, C.; Ravi, K.: Optimal power flow using hybrid DA-APSO algorithm in renewable energy resources. Energy Procedia 117, 1085–1092 (2017)

    Article  Google Scholar 

  38. Nelder, J.A.; Mead, R.: A simplex method for function minimization. Comput. J. 7, 308–313 (1965)

    Article  MathSciNet  MATH  Google Scholar 

  39. Reynolds, C.W.: Flocks, herds and schools: a distributed behavioral model. In: Proceedings of the 14th Annual Conference on Computer Graphics and Interactive Techniques. In SIGGRAPH ’87, pp. 25–34. ACM, New York, NY, USA (1987)

  40. Abhishek, R.; Malakar, T.: Optimal reactive power dispatch using hybrid Nelder–Mead simplex based firefly algorithm. Int. J. Electr. Power 66, 9–24 (2015)

    Article  Google Scholar 

  41. Blondin, M.J.; Sanchis, J.; Sicard, P.; Herrero, J.M.: New optimal controller tuning method for an AVR system using a simplified Ant Colony Optimization with a new constrained Nelder–Mead algorithm. Appl. Soft Comput. 62, 216–229 (2018)

    Article  Google Scholar 

  42. Yang, X.-S.: Chapter 2: analysis of algorithms. In: Yang, X.-S. (ed.) Nature-in-Spired Optimization Algorithms, pp. 23–44. Elsevier, Oxford (2014b)

    Chapter  Google Scholar 

  43. Mirjalili, S.: The ant lion optimizer. Adv. Eng. Softw. 83, 80–98 (2015)

    Article  Google Scholar 

  44. Demsar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)

    MathSciNet  MATH  Google Scholar 

  45. Li, Z.; Wang, W.; Yan, Y.; Li, Z.: PS–ABC: a hybrid algorithm based on particle swarm and artificial bee colony for high-dimensional optimization problems. Expert Syst. Appl. 42, 8881–8895 (2015)

    Article  Google Scholar 

  46. Engelbrecht, A.P.: Fundamentals of Computational Swarm Intelligence. Wiley, New York (2004)

    Google Scholar 

  47. Mirjalili, S.; Mirjalili, S.M.; Lewis, A.: Let a biogeography-based on optimizer train your multi-layer perceptron. Inf. Sci. 269, 188–209 (2014)

    Article  MathSciNet  Google Scholar 

  48. Mirjalili, S.: How effective is the Grey Wolf optimizer in training multi-layer perceptrons. Appl. Intell. 43, 150–161 (2015)

    Article  Google Scholar 

Download references

Acknowledgements

Project supported by the National Social Science Foundation of China (Grant No. 16BJY078), Soft Science Foundation of Heilongjiang Province (Grant No.GC16D102),Key Program of Economic and Social of Heilongjiang Province (Grant No. KY10900170004), Philosophy and Social Science Research Planning Program of Heilongjiang Province (Grant No. 17JYH49). In addition, we are grateful to the anonymous reviewers for their valuable comments that helped us improve this paper.

Author information

Authors and Affiliations

  1. School of Economics and Management, Harbin Engineering University, Harbin, 150001, China

    Jianzhong Xu & Fu Yan

Authors
  1. Jianzhong Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fu Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fu Yan.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, J., Yan, F. Hybrid Nelder–Mead Algorithm and Dragonfly Algorithm for Function Optimization and the Training of a Multilayer Perceptron. Arab J Sci Eng 44, 3473–3487 (2019). https://doi.org/10.1007/s13369-018-3536-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13369-018-3536-0

Keywords

Search

Navigation