Neural Computing and Applications

, Volume 26, Issue 3, pp 625–640 | Cite as

How important is a transfer function in discrete heuristic algorithms

  • Shahrzad Saremi
  • Seyedali Mirjalili
  • Andrew Lewis
Original Article

Abstract

Transfer functions are considered the simplest and cheapest operators in designing discrete heuristic algorithms. The main advantage of such operators is the maintenance of the structure and other continuous operators of a continuous algorithm. However, a transfer function may show different behaviour in various heuristic algorithms. This paper investigates the behaviour and importance of transfer functions in improving performance of heuristic algorithms. As case studies, two algorithms with different mechanisms of optimisation were chosen: Gravitational Search Algorithm and Particle Swarm Optimisation. Eight transfer functions were integrated in these two algorithms and compared on a set of test functions. The results show that transfer functions may show diverse behaviours and have different impacts on the performance of algorithms, which should be considered when designing a discrete algorithm. The results also demonstrate the significant role of the transfer function in terms of improved exploration and exploitation of a heuristic algorithm.

Keywords

Binary optimisation Discrete optimisation Transfer function Evolutionary algorithm Heuristic algorithm Binary algorithm Discrete algorithm 

References

  1. 1.
    Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Proceedings of the IEEE international conference on neural networks, Perth, pp 1942–1948Google Scholar
  2. 2.
    Robinson J, Rahmat-Samii Y (2004) Particle swarm optimization in electromagnetics. IEEE Trans Antennas Propagon 52:397–407CrossRefMathSciNetGoogle Scholar
  3. 3.
    Nagesh Kumar D, Janga Reddy M (2007) Multipurpose reservoir operation using particle swarm optimization. J Water Resour Plan Manag 133:192–201CrossRefGoogle Scholar
  4. 4.
    He S, Prempain E, Wu Q (2004) An improved particle swarm optimizer for mechanical design optimization problems. Eng Optim 36:585–605CrossRefMathSciNetGoogle Scholar
  5. 5.
    Karaboga D, Basturk B (2007) A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm. J Glob Optim 39:459–471CrossRefMATHMathSciNetGoogle Scholar
  6. 6.
    Dorigo M, Di Caro G (1999) Ant colony optimization: a new meta-heuristic. In: Proceedings of the 1999 congress on evolutionary computation, CEC 99Google Scholar
  7. 7.
    Kirkpatrick S, Gellatt CD Jr, Vecchi MP (1983) Optimization by simulated annealing. Science 220:671–680CrossRefMATHMathSciNetGoogle Scholar
  8. 8.
    Rashedi E, Nezamabadi-Pour H, Saryazdi S (2009) GSA: a gravitational search algorithm. Inf Sci 179:2232–2248CrossRefMATHGoogle Scholar
  9. 9.
    Holland JH (1992) Genetic algorithms. Sci Am 267:66–72CrossRefGoogle Scholar
  10. 10.
    Simon D (2008) Biogeography-based optimization. IEEE Trans Evol Comput 12:702–713CrossRefGoogle Scholar
  11. 11.
    Rechenberg I (1994) Evolution strategy. Comput Intellect Imitating Life 1Google Scholar
  12. 12.
    Storn R, Price K (1997) Differential evolution—a simple and efficient heuristic for global optimization over continuous spaces. J Glob Optim 11:341–359CrossRefMATHMathSciNetGoogle Scholar
  13. 13.
    Wang L, Pan Q-K, Suganthan P, Wang W-H, Wang Y-M (2010) A novel hybrid discrete differential evolution algorithm for blocking flow shop scheduling problems. Comput Oper Res 37:509–520CrossRefMATHMathSciNetGoogle Scholar
  14. 14.
    Geem ZW, Kim JH, Loganathan G (2001) A new heuristic optimization algorithm: harmony search. Simulation 76:60–68CrossRefGoogle Scholar
  15. 15.
    Wang L, Xu Y, Mao Y, Fei M (2010) A discrete harmony search algorithm. In: Li K, Li X, Ma S, Irwin GW (eds) Life system modeling and intelligent computing. Communications in computer and information science, vol 98. Springer, Heidelberg, pp 37–43Google Scholar
  16. 16.
    Tayarani NM, Akbarzadeh TM (2008) Magnetic Optimization Algorithms a new synthesis. In: IEEE Congress on evolutionary computation, 2008. CEC (IEEE world congress on computational intelligence). 2008, pp 2659–2664Google Scholar
  17. 17.
    Mirjalili S, Sadiq AS (2011) Magnetic optimization algorithm for training multi layer perceptron. In: 2011 IEEE 3rd international conference on communication software and networks (ICCSN), pp 42–46Google Scholar
  18. 18.
    Yang X-S (2010) A new metaheuristic bat-inspired algorithm. In: González JR, Pelta DA, Cruz C, Terrazas G, Krasnogor N (eds) Nature inspired cooperative strategies for optimization (NICSO 2010). Studies in computational intelligence, vol 284. Springer, Heidelberg, pp 65–74Google Scholar
  19. 19.
    Mirjalili S, Hashim SZM (2012) BMOA: binary magnetic optimization algorithm. Int J Mach Learn Comput 2:204–208CrossRefGoogle Scholar
  20. 20.
    Kennedy J, Eberhart RC (1997) A discrete binary version of the particle swarm algorithm. In: IEEE international conference on systems, man, and cybernetics, 1997. Computational cybernetics and Simulation, pp 4104–4108Google Scholar
  21. 21.
    Rashedi E, Nezamabadi-Pour H, Saryazdi S (2010) BGSA: binary gravitational search algorithm. Nat Comput 9:727–745CrossRefMATHMathSciNetGoogle Scholar
  22. 22.
    Mirjalili S, Mirjalili SM, Yang X-S (2014) Binary bat algorithm. Neural Comput Appl 25:663–681CrossRefGoogle Scholar
  23. 23.
    Mirjalili S, Lewis A (2013) S-shaped versus V-shaped transfer functions for binary Particle Swarm Optimization. Swarm Evol Comput 9:1–14CrossRefGoogle Scholar
  24. 24.
    Sinaie S (2010) Solving shortest path problem using gravitational search algorithm and neural networks. Master, Faculty of Computer Science and Information Systems, Universiti Teknologi Malaysia (UTM), Johor Bahru, MalaysiaGoogle Scholar
  25. 25.
    Shaw B, Mukherjee V, Ghoshal SP (2012) A novel opposition-based gravitational search algorithm for combined economic and emission dispatch problems of power systems. Int J Electr Power Energy Syst 35:21–33CrossRefGoogle Scholar
  26. 26.
    Mirjalili S, Hashim SZM (2010) A new hybrid PSOGSA algorithm for function optimization. In: 2010 international conference on computer and information application (ICCIA), pp 374–377Google Scholar
  27. 27.
    Mirjalili S, Mohd Hashim SZ, Moradian Sardroudi H (2012) Training feedforward neural networks using hybrid particle swarm optimization and gravitational search algorithm. Appl Math Comput 218:11125–11137CrossRefMATHMathSciNetGoogle Scholar
  28. 28.
    Chen H, Li S, Tang Z (2011) Hybrid gravitational search algorithm with random-key encoding scheme combined with simulated annealing. IJCSNS 11:208MATHGoogle Scholar
  29. 29.
    Zhang Y, Wu L, Zhang Y, Wang J (2012) Immune gravitation inspired optimization algorithm. In: Huang D-S, Gan Y, Bevilacqua V, Figueroa JC (eds) Advanced intelligent computing. Lecture notes in computer science, vol 6838. Springer, Heidelberg, pp 178–185 Google Scholar
  30. 30.
    Hatamlou A, Abdullah S, Othman Z (2011) Gravitational search algorithm with heuristic search for clustering problems. In: 2011 3rd conference on data mining and optimization (DMO), pp 190–193Google Scholar
  31. 31.
    Mirjalili S, Lewis A (2014) Adaptive gbest-guided gravitational search algorithm. Neural Comput Appl 1–16. doi:10.1007/s00521-014-1640-y
  32. 32.
    Eberhart RC, Kennedy J (1995) A new optimizer using particle swarm theory. In: Proceedings of the sixth international symposium on micro machine and human science, pp 39–43Google Scholar
  33. 33.
    AlRashidi MR, El-Hawary ME (2009) A survey of particle swarm optimization applications in electric power systems. IEEE Trans Evol Comput 13:913–918CrossRefGoogle Scholar
  34. 34.
    Mirjalili S, Lewis A, Sadiq AS (2014) Autonomous particles groups for particle swarm optimization. Arabian J Sci Eng 39(6):4683–4697. doi:10.1007/s13369-014-1156-x
  35. 35.
    Song M-P, Gu G-C (2004) Research on particle swarm optimization: a review. In: Proceedings of 2004 international conference on machine learning and cybernetics, pp 2236–2241Google Scholar
  36. 36.
    Wei Y, Qiqiang L (2004) Survey on Particle Swarm Optimization Algorithm. Eng Sci 5:87–94Google Scholar
  37. 37.
    Khanesar MA, Teshnehlab M, Shoorehdeli MA (2007) A novel binary particle swarm optimization. In: Mediterranean conference on control and automation, 2007. MED’07, pp 1–6Google Scholar
  38. 38.
    Chuang L-Y, Chang H-W, Tu C-J, Yang C-H (2008) Improved binary PSO for feature selection using gene expression data. Comput Biol Chem 32:29–38CrossRefMATHGoogle Scholar
  39. 39.
    Sudholt D, Witt C (2008) Runtime analysis of binary PSO. In: Proceedings of the 10th annual conference on genetic and evolutionary computation, pp 135–142Google Scholar
  40. 40.
    Mirjalili S, Wang G-G, Coelho LdS (2014) Binary optimization using hybrid particle swarm optimization and gravitational search algorithm. Neural Comput Appl 1–13. doi:10.1007/s00521-014-1629-6
  41. 41.
    Yao X, Liu Y, Lin G (1999) Evolutionary programming made faster. IEEE Trans Evol Comput 3:82–102CrossRefGoogle Scholar
  42. 42.
    Digalakis J, Margaritis K (2001) On benchmarking functions for genetic algorithms. Int J Comput Math 77:481–506CrossRefMATHMathSciNetGoogle Scholar
  43. 43.
    Molga M, Smutnicki C (2005) Test functions for optimization needs. Test Functions for Optimization NeedsGoogle Scholar
  44. 44.
    Yang X-S (2010) Appendix A: test problems in optimization. In: Engineering optimization. Wiley, pp 261–266. doi:10.1002/9780470640425.app1
  45. 45.
    Saremi S, Mirjalili S, Lewis A (2014) Biogeography-based optimisation with chaos. Neural Comput Appl 25(5):1077–1097. doi:10.1007/s00521-014-1597-x

Copyright information

© The Natural Computing Applications Forum 2014

Authors and Affiliations

  • Shahrzad Saremi
    • 1
  • Seyedali Mirjalili
    • 1
  • Andrew Lewis
    • 1
  1. 1.School of Information and Communication TechnologyGriffith UniversityBrisbaneAustralia

Personalised recommendations