SQG-Differential Evolution for Difficult Optimization Problems under a Tight Function Evaluation Budget
In the context of industrial engineering, it is important to integrate efficient computational optimization methods in the product development process. Some of the most challenging simulation-based engineering design optimization problems are characterized by: a large number of design variables, the absence of analytical gradients, highly non-linear objectives and a limited function evaluation budget. Although a huge variety of different optimization algorithms is available, the development and selection of efficient algorithms for problems with these industrial relevant characteristics, remains a challenge. In this communication, a hybrid variant of Differential Evolution (DE) is introduced which combines aspects of Stochastic Quasi-Gradient (SQG) methods within the framework of DE, in order to improve optimization efficiency on problems with the previously mentioned characteristics. The performance of the resulting derivative-free algorithm is compared with other state-of-the-art DE variants on 25 commonly used benchmark functions, under tight function evaluation budget constraints of 1000 evaluations. The experimental results indicate that the new algorithm performs excellent on the “difficult” (high dimensional, multi-modal, inseparable) test functions. The operations used in the proposed mutation scheme, are computationally inexpensive, and can be easily implemented in existing differential evolution variants or other population-based optimization algorithms by a few lines of program code as an non-invasive optional setting. Besides the applicability of the presented algorithm by itself, the described concepts can serve as a useful and interesting addition to the algorithmic operators in the frameworks of heuristics and evolutionary optimization and computing.
KeywordsMeta-heuristics Derivative-free optimization Evolutionary computing Differential evolution Black box optimization Stochastic Quasi-Gradient Descend SQG-DE
This work was partially funded by the GRESIMO project grant agreement no. 290050 by the European community 7th Framework program. We would like to thank the anonymous reviewers for their remarks to improve the manuscript. Furthermore, we like to express our gratitude to Qingfu Zhang and all other cited authors who made code of their algorithms and test benches publicly available on their websites.
- 5.Schaffer, C.: A conservation law for generalization performance. In: Proceedings of the 11th International Conference on Machine Learning, pp. 259–265 (1994)Google Scholar
- 7.Aissa, M.H., Verstraete, T., Vuik, C.: Aerodynamic optimization of supersonic compressor cascade using differential evolution on GPU. In: Simos, T., Tsitouras, C. (eds.) AIP Conference Proceedings, p. 480077 (2016)Google Scholar
- 11.Sala, R., Pierini, M., Baldanzini, N.: Optimization efficiency in multidisciplinary vehicle design including NVH criteria. In: Proceedings of the 26th International Conference on Noise and Vibration Engineering, ISMA, pp. 1571–1585 (2014)Google Scholar
- 14.Jansen, T., Zarges, C.: Fixed budget computations: a different perspective on run time analysis. In: Proceedings of the 14th Annual Conference on Genetic and Evolutionary Computation, pp. 1325–1332. ACM (2012)Google Scholar
- 15.Storn, R., Price, K.: Differential evolution - a simple and efficient adaptive scheme for global optimization over continuous spaces, Technical report TR-95-012, ICSI (1995)Google Scholar
- 24.Knowles, J., Corne, D., Reynolds, A.: Noisy multiobjective optimization on a budget of 250 evaluations. In: Ehrgott, M., Fonseca, C.M., Gandibleux, X., Hao, J.-K., Sevaux, M. (eds.) EMO 2009. LNCS, vol. 5467, pp. 36–50. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-01020-0_8 CrossRefGoogle Scholar
- 29.Kennedy, J., Eberhart, R.: Particle swarm optimization. In: Proceedings of the IEEE International Conference on Neural Networks, vol. 4, pp. 1942–1948. IEEE (1995)Google Scholar
- 30.Suganthan, P.N., Hansen, N., Liang, J.J., Deb, K., Chen, Y.P., Auger, A., Tiwari, S.: Problem definitions and evaluation criteria for the CEC 2005 special session on real-parameter optimization. KanGAL report, 2005005 (2005)Google Scholar
- 33.Auger, A., Hansen, N.: Performance evaluation of an advanced local search evolutionary algorithm. In: Proceedings of the IEEE Congress on Evolutionary Computation, vol. 2, pp. 1777–1784. IEEE (2005)Google Scholar
- 34.Qingfu Zhang’s Homepage. http://dces.essex.ac.uk/staff/qzhang/code/codealgorithm/. Accessed 5 Apr 2017