Skip to main content
Book cover

International Workshop on Machine Learning, Optimization, and Big Data

MOD 2017: Machine Learning, Optimization, and Big Data pp 322–336Cite as

SQG-Differential Evolution for Difficult Optimization Problems under a Tight Function Evaluation Budget

Part of the Lecture Notes in Computer Science book series (LNISA,volume 10710)

Abstract

In the context of industrial engineering, it is important to integrate efficient computational optimization methods in the product development process. Some of the most challenging simulation-based engineering design optimization problems are characterized by: a large number of design variables, the absence of analytical gradients, highly non-linear objectives and a limited function evaluation budget. Although a huge variety of different optimization algorithms is available, the development and selection of efficient algorithms for problems with these industrial relevant characteristics, remains a challenge. In this communication, a hybrid variant of Differential Evolution (DE) is introduced which combines aspects of Stochastic Quasi-Gradient (SQG) methods within the framework of DE, in order to improve optimization efficiency on problems with the previously mentioned characteristics. The performance of the resulting derivative-free algorithm is compared with other state-of-the-art DE variants on 25 commonly used benchmark functions, under tight function evaluation budget constraints of 1000 evaluations. The experimental results indicate that the new algorithm performs excellent on the “difficult” (high dimensional, multi-modal, inseparable) test functions. The operations used in the proposed mutation scheme, are computationally inexpensive, and can be easily implemented in existing differential evolution variants or other population-based optimization algorithms by a few lines of program code as an non-invasive optional setting. Besides the applicability of the presented algorithm by itself, the described concepts can serve as a useful and interesting addition to the algorithmic operators in the frameworks of heuristics and evolutionary optimization and computing.

Keywords

  • Meta-heuristics
  • Derivative-free optimization
  • Evolutionary computing
  • Differential evolution
  • Black box optimization
  • Stochastic Quasi-Gradient Descend
  • SQG-DE

This is a preview of subscription content, access via your institution.

Chapter
EUR   29.95
Price includes VAT (Finland)
  • DOI: 10.1007/978-3-319-72926-8_27
  • Chapter length: 15 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
EUR   69.54
Price includes VAT (Finland)
  • ISBN: 978-3-319-72926-8
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
EUR   90.20
Price includes VAT (Finland)
Learn about institutional subscriptions
Fig. 1.

References

  1. Floudas, C.A., Gounaris, C.E.: A review of recent advances in global optimization. J. Global Optim. 45(1), 3–38 (2009)

    MathSciNet  MATH  CrossRef  Google Scholar 

  2. Rios, L.M., Sahinidis, N.V.: Derivative-free optimization: a review of algorithms and comparison of software implementations. J. Global Optim. 56(3), 1247–1293 (2013)

    MathSciNet  MATH  CrossRef  Google Scholar 

  3. Sobieszczanski-Sobieski, J., Haftka, R.T.: Multidisciplinary aerospace design optimization: survey of recent developments. Struct. Optim. 14(1), 1–23 (1997)

    CrossRef  Google Scholar 

  4. Venkataraman, S., Haftka, R.T.: Structural optimization complexity: what has Moore’s law done for us? Struct. Multidisc. Optim. 28(6), 375–387 (2004)

    CrossRef  Google Scholar 

  5. Schaffer, C.: A conservation law for generalization performance. In: Proceedings of the 11th International Conference on Machine Learning, pp. 259–265 (1994)

    CrossRef  Google Scholar 

  6. Wolpert, D.H., Macready, W.G.: No free lunch theorems for optimization. IEEE Trans. Evol. Comput. 1(1), 67–82 (1997)

    CrossRef  Google Scholar 

  7. Aissa, M.H., Verstraete, T., Vuik, C.: Aerodynamic optimization of supersonic compressor cascade using differential evolution on GPU. In: Simos, T., Tsitouras, C. (eds.) AIP Conference Proceedings, p. 480077 (2016)

    Google Scholar 

  8. Carrigan, T.J., Dennis, B.H., Han, Z.X., Wang, B.P.: Aerodynamic shape optimization of a vertical-axis wind turbine using differential evolution. ISRN Renew. Energy 2012, 1–16 (2012)

    CrossRef  Google Scholar 

  9. Kiani, M., Yildiz, A.R.: A comparative study of non-traditional methods for vehicle crashworthiness and NVH optimization. Arch. Comput. Meth. Eng. 23(4), 723–734 (2016)

    MathSciNet  MATH  CrossRef  Google Scholar 

  10. Duddeck, F.: Multidisciplinary optimization of car bodies. Struct. Multidisc. Optim. 35(4), 375–389 (2008)

    CrossRef  Google Scholar 

  11. Sala, R., Pierini, M., Baldanzini, N.: Optimization efficiency in multidisciplinary vehicle design including NVH criteria. In: Proceedings of the 26th International Conference on Noise and Vibration Engineering, ISMA, pp. 1571–1585 (2014)

    Google Scholar 

  12. Sala, R., Baldanzini, N., Pierini, M.: Representative surrogate problems as test functions for expensive simulators in multidisciplinary design optimization of vehicle structures. Struct. Multidisc. Optim. 54(3), 449–468 (2016)

    MathSciNet  CrossRef  Google Scholar 

  13. Haftka, R.T., Watson, L.T.: Multidisciplinary design optimization with quasiseparable subsystems. Optim. Eng. 6(1), 9–20 (2005)

    MathSciNet  MATH  CrossRef  Google Scholar 

  14. Jansen, T., Zarges, C.: Fixed budget computations: a different perspective on run time analysis. In: Proceedings of the 14th Annual Conference on Genetic and Evolutionary Computation, pp. 1325–1332. ACM (2012)

    Google Scholar 

  15. Storn, R., Price, K.: Differential evolution - a simple and efficient adaptive scheme for global optimization over continuous spaces, Technical report TR-95-012, ICSI (1995)

    Google Scholar 

  16. Storn, R., Price, K.: Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces. J. Global Optim. 11(4), 341–359 (1997)

    MathSciNet  MATH  CrossRef  Google Scholar 

  17. Brest, J., Greiner, S., Boskovic, B., Mernik, M., Zumer, V.: Self-adapting control parameters in differential evolution: a comparative study on numerical benchmark problems. IEEE Trans. Evol. Comput. 10(6), 646–657 (2006)

    CrossRef  Google Scholar 

  18. Qin, A.K., Huang, V.L., Suganthan, P.N.: Differential evolution algorithm with strategy adaptation for global numerical optimization. IEEE Trans. Evol. Comput. 13(2), 398–417 (2009)

    CrossRef  Google Scholar 

  19. Mallipeddi, R., Suganthan, P.N., Pan, Q.K., Tasgetiren, M.F.: Differential evolution algorithm with ensemble of parameters and mutation strategies. Appl. Soft Comput. 11(2), 1679–1696 (2011)

    CrossRef  Google Scholar 

  20. Zhang, J., Sanderson, A.C.: JADE: adaptive differential evolution with optional external archive. IEEE Trans. Evol. Comput. 13(5), 945–958 (2009)

    CrossRef  Google Scholar 

  21. Wang, Y., Cai, Z., Zhang, Q.: Differential evolution with composite trial vector generation strategies and control parameters. IEEE Trans. Evol. Comput. 15(1), 55–66 (2011)

    CrossRef  Google Scholar 

  22. Das, S., Suganthan, P.N.: Differential evolution: a survey of the state-of-the-art. IEEE Trans. Evol. Comput. 15(1), 4–31 (2011)

    CrossRef  Google Scholar 

  23. Das, S., Mullick, S.S., Suganthan, P.N.: Recent advances in differential evolution–an updated survey. Swarm Evol. Comput. 27, 1–30 (2016)

    CrossRef  Google Scholar 

  24. Knowles, J., Corne, D., Reynolds, A.: Noisy multiobjective optimization on a budget of 250 evaluations. In: Ehrgott, M., Fonseca, C.M., Gandibleux, X., Hao, J.-K., Sevaux, M. (eds.) EMO 2009. LNCS, vol. 5467, pp. 36–50. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-01020-0_8

    CrossRef  Google Scholar 

  25. Ermoliev, Y.M.: Methods of solution of nonlinear extremal problems. Cybernetics 2(4), 1–14 (1966)

    MathSciNet  CrossRef  Google Scholar 

  26. Ermoliev, Y.M.: Stochastic quasigradient methods and their application to system optimization. Stochast. Int. J. Probab. Stochast. Process. 9(1–2), 1–36 (1983)

    MathSciNet  MATH  Google Scholar 

  27. Wang, G.G., Shan, S.: Review of metamodeling techniques in support of engineering design optimization. J. Mech. Des. 129(4), 370–380 (2007)

    CrossRef  Google Scholar 

  28. Krityakierne, T., Ginsbourger, D.: Global optimization with sparse and local Gaussian process models. In: Pardalos, P., Pavone, M., Farinella, G.M., Cutello, V. (eds.) MOD 2015. LNCS, vol. 9432, pp. 185–196. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-27926-8_16

    CrossRef  Google Scholar 

  29. Kennedy, J., Eberhart, R.: Particle swarm optimization. In: Proceedings of the IEEE International Conference on Neural Networks, vol. 4, pp. 1942–1948. IEEE (1995)

    Google Scholar 

  30. Suganthan, P.N., Hansen, N., Liang, J.J., Deb, K., Chen, Y.P., Auger, A., Tiwari, S.: Problem definitions and evaluation criteria for the CEC 2005 special session on real-parameter optimization. KanGAL report, 2005005 (2005)

    Google Scholar 

  31. Yao, X., Liu, Y., Lin, G.: Evolutionary programming made faster. IEEE Trans. Evol. Comput. 3(2), 82–102 (1999)

    CrossRef  Google Scholar 

  32. Brest, J., Zamuda, A., Bošković, B., Greiner, S., Žumer, V.: An analysis of the control parameters’ adaptation in DE. In: Chakraborty, U.K. (ed.) Advances in Differential Evolution, vol. 143, pp. 89–110. Springer, Heidelberg (2008)

    CrossRef  Google Scholar 

  33. Auger, A., Hansen, N.: Performance evaluation of an advanced local search evolutionary algorithm. In: Proceedings of the IEEE Congress on Evolutionary Computation, vol. 2, pp. 1777–1784. IEEE (2005)

    Google Scholar 

  34. Qingfu Zhang’s Homepage. http://dces.essex.ac.uk/staff/qzhang/code/codealgorithm/. Accessed 5 Apr 2017

  35. García, S., Molina, D., Lozano, M., Herrera, F.: A study on the use of non-parametric tests for analyzing the evolutionary algorithms’ behaviour: a case study on the CEC’2005 special session on real parameter optimization. J. Heuristics 15(6), 617–644 (2009)

    MATH  CrossRef  Google Scholar 

  36. Wilcoxon, F.: Individual comparisons by ranking methods. Biometrics Bull. 1(6), 80–83 (1945)

    CrossRef  Google Scholar 

  37. Sala, R., Baldanzini, N., Pierini, M.: Global optimization test problems based on random field composition. Optim. Lett. 11(4), 699–713 (2017)

    MathSciNet  MATH  CrossRef  Google Scholar 

Download references

Acknowledgments

This work was partially funded by the GRESIMO project grant agreement no. 290050 by the European community 7th Framework program. We would like to thank the anonymous reviewers for their remarks to improve the manuscript. Furthermore, we like to express our gratitude to Qingfu Zhang and all other cited authors who made code of their algorithms and test benches publicly available on their websites.

Author information

Authors and Affiliations

  1. Department of Industrial Engineering, University of Florence, Florence, Italy

    Ramses Sala, Niccolò Baldanzini & Marco Pierini

Authors
  1. Ramses Sala
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Niccolò Baldanzini
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marco Pierini
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ramses Sala .

Editor information

Editors and Affiliations

  1. University of Catania, Catania, Italy

    Giuseppe Nicosia

  2. University of Florida, Gainesville, FL, USA

    Dr. Panos Pardalos

  3. University of Catania, Catania, Italy

    Prof. Giovanni Giuffrida

  4. Harvard University, Cambridge, MA, USA

    Renato Umeton

Rights and permissions

Reprints and Permissions

Copyright information

© 2018 Springer International Publishing AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Sala, R., Baldanzini, N., Pierini, M. (2018). SQG-Differential Evolution for Difficult Optimization Problems under a Tight Function Evaluation Budget. In: Nicosia, G., Pardalos, P., Giuffrida, G., Umeton, R. (eds) Machine Learning, Optimization, and Big Data. MOD 2017. Lecture Notes in Computer Science(), vol 10710. Springer, Cham. https://doi.org/10.1007/978-3-319-72926-8_27

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-72926-8_27

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-72925-1

  • Online ISBN: 978-3-319-72926-8

  • eBook Packages: Computer ScienceComputer Science (R0)