SQG-Differential Evolution for Difficult Optimization Problems under a Tight Function Evaluation Budget

  • Ramses SalaEmail author
  • Niccolò Baldanzini
  • Marco Pierini
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10710)


In the context of industrial engineering, it is important to integrate efficient computational optimization methods in the product development process. Some of the most challenging simulation-based engineering design optimization problems are characterized by: a large number of design variables, the absence of analytical gradients, highly non-linear objectives and a limited function evaluation budget. Although a huge variety of different optimization algorithms is available, the development and selection of efficient algorithms for problems with these industrial relevant characteristics, remains a challenge. In this communication, a hybrid variant of Differential Evolution (DE) is introduced which combines aspects of Stochastic Quasi-Gradient (SQG) methods within the framework of DE, in order to improve optimization efficiency on problems with the previously mentioned characteristics. The performance of the resulting derivative-free algorithm is compared with other state-of-the-art DE variants on 25 commonly used benchmark functions, under tight function evaluation budget constraints of 1000 evaluations. The experimental results indicate that the new algorithm performs excellent on the “difficult” (high dimensional, multi-modal, inseparable) test functions. The operations used in the proposed mutation scheme, are computationally inexpensive, and can be easily implemented in existing differential evolution variants or other population-based optimization algorithms by a few lines of program code as an non-invasive optional setting. Besides the applicability of the presented algorithm by itself, the described concepts can serve as a useful and interesting addition to the algorithmic operators in the frameworks of heuristics and evolutionary optimization and computing.


Meta-heuristics Derivative-free optimization Evolutionary computing Differential evolution Black box optimization Stochastic Quasi-Gradient Descend SQG-DE 



This work was partially funded by the GRESIMO project grant agreement no. 290050 by the European community 7th Framework program. We would like to thank the anonymous reviewers for their remarks to improve the manuscript. Furthermore, we like to express our gratitude to Qingfu Zhang and all other cited authors who made code of their algorithms and test benches publicly available on their websites.


  1. 1.
    Floudas, C.A., Gounaris, C.E.: A review of recent advances in global optimization. J. Global Optim. 45(1), 3–38 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  2. 2.
    Rios, L.M., Sahinidis, N.V.: Derivative-free optimization: a review of algorithms and comparison of software implementations. J. Global Optim. 56(3), 1247–1293 (2013)MathSciNetzbMATHCrossRefGoogle Scholar
  3. 3.
    Sobieszczanski-Sobieski, J., Haftka, R.T.: Multidisciplinary aerospace design optimization: survey of recent developments. Struct. Optim. 14(1), 1–23 (1997)CrossRefGoogle Scholar
  4. 4.
    Venkataraman, S., Haftka, R.T.: Structural optimization complexity: what has Moore’s law done for us? Struct. Multidisc. Optim. 28(6), 375–387 (2004)CrossRefGoogle Scholar
  5. 5.
    Schaffer, C.: A conservation law for generalization performance. In: Proceedings of the 11th International Conference on Machine Learning, pp. 259–265 (1994)CrossRefGoogle Scholar
  6. 6.
    Wolpert, D.H., Macready, W.G.: No free lunch theorems for optimization. IEEE Trans. Evol. Comput. 1(1), 67–82 (1997)CrossRefGoogle Scholar
  7. 7.
    Aissa, M.H., Verstraete, T., Vuik, C.: Aerodynamic optimization of supersonic compressor cascade using differential evolution on GPU. In: Simos, T., Tsitouras, C. (eds.) AIP Conference Proceedings, p. 480077 (2016)Google Scholar
  8. 8.
    Carrigan, T.J., Dennis, B.H., Han, Z.X., Wang, B.P.: Aerodynamic shape optimization of a vertical-axis wind turbine using differential evolution. ISRN Renew. Energy 2012, 1–16 (2012)CrossRefGoogle Scholar
  9. 9.
    Kiani, M., Yildiz, A.R.: A comparative study of non-traditional methods for vehicle crashworthiness and NVH optimization. Arch. Comput. Meth. Eng. 23(4), 723–734 (2016)MathSciNetzbMATHCrossRefGoogle Scholar
  10. 10.
    Duddeck, F.: Multidisciplinary optimization of car bodies. Struct. Multidisc. Optim. 35(4), 375–389 (2008)CrossRefGoogle Scholar
  11. 11.
    Sala, R., Pierini, M., Baldanzini, N.: Optimization efficiency in multidisciplinary vehicle design including NVH criteria. In: Proceedings of the 26th International Conference on Noise and Vibration Engineering, ISMA, pp. 1571–1585 (2014)Google Scholar
  12. 12.
    Sala, R., Baldanzini, N., Pierini, M.: Representative surrogate problems as test functions for expensive simulators in multidisciplinary design optimization of vehicle structures. Struct. Multidisc. Optim. 54(3), 449–468 (2016)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Haftka, R.T., Watson, L.T.: Multidisciplinary design optimization with quasiseparable subsystems. Optim. Eng. 6(1), 9–20 (2005)MathSciNetzbMATHCrossRefGoogle Scholar
  14. 14.
    Jansen, T., Zarges, C.: Fixed budget computations: a different perspective on run time analysis. In: Proceedings of the 14th Annual Conference on Genetic and Evolutionary Computation, pp. 1325–1332. ACM (2012)Google Scholar
  15. 15.
    Storn, R., Price, K.: Differential evolution - a simple and efficient adaptive scheme for global optimization over continuous spaces, Technical report TR-95-012, ICSI (1995)Google Scholar
  16. 16.
    Storn, R., Price, K.: Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces. J. Global Optim. 11(4), 341–359 (1997)MathSciNetzbMATHCrossRefGoogle Scholar
  17. 17.
    Brest, J., Greiner, S., Boskovic, B., Mernik, M., Zumer, V.: Self-adapting control parameters in differential evolution: a comparative study on numerical benchmark problems. IEEE Trans. Evol. Comput. 10(6), 646–657 (2006)CrossRefGoogle Scholar
  18. 18.
    Qin, A.K., Huang, V.L., Suganthan, P.N.: Differential evolution algorithm with strategy adaptation for global numerical optimization. IEEE Trans. Evol. Comput. 13(2), 398–417 (2009)CrossRefGoogle Scholar
  19. 19.
    Mallipeddi, R., Suganthan, P.N., Pan, Q.K., Tasgetiren, M.F.: Differential evolution algorithm with ensemble of parameters and mutation strategies. Appl. Soft Comput. 11(2), 1679–1696 (2011)CrossRefGoogle Scholar
  20. 20.
    Zhang, J., Sanderson, A.C.: JADE: adaptive differential evolution with optional external archive. IEEE Trans. Evol. Comput. 13(5), 945–958 (2009)CrossRefGoogle Scholar
  21. 21.
    Wang, Y., Cai, Z., Zhang, Q.: Differential evolution with composite trial vector generation strategies and control parameters. IEEE Trans. Evol. Comput. 15(1), 55–66 (2011)CrossRefGoogle Scholar
  22. 22.
    Das, S., Suganthan, P.N.: Differential evolution: a survey of the state-of-the-art. IEEE Trans. Evol. Comput. 15(1), 4–31 (2011)CrossRefGoogle Scholar
  23. 23.
    Das, S., Mullick, S.S., Suganthan, P.N.: Recent advances in differential evolution–an updated survey. Swarm Evol. Comput. 27, 1–30 (2016)CrossRefGoogle Scholar
  24. 24.
    Knowles, J., Corne, D., Reynolds, A.: Noisy multiobjective optimization on a budget of 250 evaluations. In: Ehrgott, M., Fonseca, C.M., Gandibleux, X., Hao, J.-K., Sevaux, M. (eds.) EMO 2009. LNCS, vol. 5467, pp. 36–50. Springer, Heidelberg (2009). Scholar
  25. 25.
    Ermoliev, Y.M.: Methods of solution of nonlinear extremal problems. Cybernetics 2(4), 1–14 (1966)MathSciNetCrossRefGoogle Scholar
  26. 26.
    Ermoliev, Y.M.: Stochastic quasigradient methods and their application to system optimization. Stochast. Int. J. Probab. Stochast. Process. 9(1–2), 1–36 (1983)MathSciNetzbMATHGoogle Scholar
  27. 27.
    Wang, G.G., Shan, S.: Review of metamodeling techniques in support of engineering design optimization. J. Mech. Des. 129(4), 370–380 (2007)CrossRefGoogle Scholar
  28. 28.
    Krityakierne, T., Ginsbourger, D.: Global optimization with sparse and local Gaussian process models. In: Pardalos, P., Pavone, M., Farinella, G.M., Cutello, V. (eds.) MOD 2015. LNCS, vol. 9432, pp. 185–196. Springer, Cham (2015). Scholar
  29. 29.
    Kennedy, J., Eberhart, R.: Particle swarm optimization. In: Proceedings of the IEEE International Conference on Neural Networks, vol. 4, pp. 1942–1948. IEEE (1995)Google Scholar
  30. 30.
    Suganthan, P.N., Hansen, N., Liang, J.J., Deb, K., Chen, Y.P., Auger, A., Tiwari, S.: Problem definitions and evaluation criteria for the CEC 2005 special session on real-parameter optimization. KanGAL report, 2005005 (2005)Google Scholar
  31. 31.
    Yao, X., Liu, Y., Lin, G.: Evolutionary programming made faster. IEEE Trans. Evol. Comput. 3(2), 82–102 (1999)CrossRefGoogle Scholar
  32. 32.
    Brest, J., Zamuda, A., Bošković, B., Greiner, S., Žumer, V.: An analysis of the control parameters’ adaptation in DE. In: Chakraborty, U.K. (ed.) Advances in Differential Evolution, vol. 143, pp. 89–110. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  33. 33.
    Auger, A., Hansen, N.: Performance evaluation of an advanced local search evolutionary algorithm. In: Proceedings of the IEEE Congress on Evolutionary Computation, vol. 2, pp. 1777–1784. IEEE (2005)Google Scholar
  34. 34.
    Qingfu Zhang’s Homepage. Accessed 5 Apr 2017
  35. 35.
    García, S., Molina, D., Lozano, M., Herrera, F.: A study on the use of non-parametric tests for analyzing the evolutionary algorithms’ behaviour: a case study on the CEC’2005 special session on real parameter optimization. J. Heuristics 15(6), 617–644 (2009)zbMATHCrossRefGoogle Scholar
  36. 36.
    Wilcoxon, F.: Individual comparisons by ranking methods. Biometrics Bull. 1(6), 80–83 (1945)CrossRefGoogle Scholar
  37. 37.
    Sala, R., Baldanzini, N., Pierini, M.: Global optimization test problems based on random field composition. Optim. Lett. 11(4), 699–713 (2017)MathSciNetzbMATHCrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  • Ramses Sala
    • 1
    Email author
  • Niccolò Baldanzini
    • 1
  • Marco Pierini
    • 1
  1. 1.Department of Industrial EngineeringUniversity of FlorenceFlorenceItaly

Personalised recommendations