Skip to main content
Log in

Integrating \(\varepsilon \)-dominance and RBF surrogate optimization for solving computationally expensive many-objective optimization problems

  • Published:
Journal of Global Optimization Aims and scope Submit manuscript

Abstract

Multi-objective optimization of computationally expensive, multimodal problems is very challenging, and is even more difficult for problems with many objectives (more than three). Optimization methods that incorporate surrogates within iterative frameworks, can be effective for solving such problems by reducing the number of expensive objective function evaluations that need to be done to find a good solution. However, only a few surrogate algorithms have been developed that are suitable for solving expensive many-objective problems. We propose a novel and effective optimization algorithm, \(\varepsilon \)-MaSO, that integrates \(\varepsilon \)-dominance with iterative Radial Basis Function surrogate-assisted framework to solve problems with many expensive objectives. \(\varepsilon \)-MaSO also incorporates a new strategy for selecting points for expensive evaluations, that is specially designed for many-objective problems. Moreover, a bi-level restart mechanism is introduced to prevent the algorithm from remaining in a local optimum and hence, increase the probability of finding the global optimum. Effectiveness of \(\varepsilon \)-MaSO is illustrated via application to DTLZ test suite with 2 to 8 objectives and to a simulation model of an environmental application. Results on both test problems and the environmental application indicate that \(\varepsilon \)-MaSO outperforms the other two surrogate-assisted many-objective methods, CSEA and K-RVEA, and an evolutionary many-objective method Borg within limited budget.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Ahmadi, M., Arabi, M., Ascough, J.C., Fontane, D.G., Engel, B.A.: Toward improved calibration of watershed models: Multisite multiobjective measures of information. Environ. Modell. Softw. 59, 135–145 (2014). https://doi.org/10.1016/j.envsoft.2014.05.012

    Article  Google Scholar 

  2. Ahmed, M.O., Vaswani, S., Schmidt, M.: Combining Bayesian optimization and Lipschitz optimization. Mach. Learn. 109, 79–102 (2020). https://doi.org/10.1007/s10994-019-05833-y

    Article  MathSciNet  MATH  Google Scholar 

  3. Akhtar, T., Shoemaker, C.A.: Multi objective optimization of computationally expensive multi-modal functions with RBF surrogates and multi-rule selection. J. Glob. Optim. 64(1), 17–32 (2016). https://doi.org/10.1007/s10898-015-0270-y

    Article  MathSciNet  MATH  Google Scholar 

  4. Akhtar, T., Shoemaker, C.A., Wang, W.: Surrogate many objective optimization: Combining evolutionary search, \(\epsilon \)-dominance and connected restarts. In: Le Thi, H., Le, H., Pham Dinh, T. (eds.) Optimization of Complex Systems: Theory, Models, Algorithms and Applications. WCGO 2019. Advances in Intelligent Systems and Computing, vol. 991. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-21803-4_68

  5. Allmendinger, R., Emmerich, M.T., Hakanen, J., Jin, Y., Rigoni, E.: Surrogate-assisted multicriteria optimization: Complexities, prospective solutions, and business case. J. Multi-Criteria Decis. Anal. 24(1–2), 5–24 (2017). https://doi.org/10.1002/mcda.1605

    Article  Google Scholar 

  6. Arnold, J.G., Srinivasan, R., Muttiah, R.S., Williams, J.R.: Large area hydrologic modeling and assessment part I: model development. J. Am. Water Resour. Assoc. 34(1), 73–89 (1998). https://doi.org/10.1111/j.1752-1688.1998.tb05961.x

    Article  Google Scholar 

  7. Asadzadeh, M., Tolson, B.A., Burn, D.H.: A new selection metric for multiobjective hydrologic model calibration. Water Resour. Res. 50(9), 7082–7099 (2014). https://doi.org/10.1002/2013WR014970

    Article  Google Scholar 

  8. Audet, C., Savard, G., Zghal, W.: A mesh adaptive direct search algorithm for multiobjective optimization. Eur. J. Oper. Res. 204(3), 545–556 (2010). https://doi.org/10.1016/j.ejor.2009.11.010

    Article  MathSciNet  MATH  Google Scholar 

  9. Beume, N., Rudolph, G.: Faster S-metric calculation by considering dominated hypervolume as Klee’s measure problem. In: Proceedings of the 2nd IASTED International Conference on Computational Intelligence, pp. 233–238 (2006)

  10. Bhattacharjee, K.S., Singh, H.K., Ray, T.: Multiple surrogate-Assisted many-objective optimization for computationally expensive engineering design. J. Mech. Des. Trans. ASME (2018). https://doi.org/10.1115/1.4039450

    Article  Google Scholar 

  11. Buhmann, M.: Radial Basis Functions: Theory and Implementations. Cambridge University Press, NY (2003)

    Book  Google Scholar 

  12. Chugh, T., Jin, Y., Miettinen, K., Hakanen, J., Sindhya, K.: A surrogate-assisted reference vector guided evolutionary algorithm for computationally expensive many-objective optimization. IEEE Trans. Evolution. Comput. 22(1), 129–142 (2018). https://doi.org/10.1109/TEVC.2016.2622301

    Article  Google Scholar 

  13. Cressie, N.: Statistics for Spatial Data. Wiley, NY (1993)

    Book  Google Scholar 

  14. Custódio, A.L., Madeira, J.F.: MultiGLODS: global and local multiobjective optimization using direct search. J. Glob. Optim. 72, 323–345 (2018). https://doi.org/10.1007/s10898-018-0618-1

    Article  MathSciNet  MATH  Google Scholar 

  15. Deb, K.: Multi-Objective Optimization Using Evolutionary Algorithms. John Wiley & Sons Inc, USA (2001)

    MATH  Google Scholar 

  16. Deb, K., Jain, H.: An Evolutionary Many-objective Optimization Algorithm Using Reference-point-based Nondominated Sorting Approach, Part I: Solving Problems with Box Constraints. IEEE Trans. Evolution. Comput. 18(4), 577–601 (2014). https://doi.org/10.1109/TEVC.2013.2281535

    Article  Google Scholar 

  17. Deb, K., Mohan, M., Mishra, S.: Evaluating the \(\epsilon \)-domination based multi-objective evolutionary algorithm for a quick computation of Pareto-optimal solutions. Evolution. Comput. 13(4), 501–525 (2005). https://doi.org/10.1162/106365605774666895

    Article  Google Scholar 

  18. Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evolution. Comput. 6(2), 182–197 (2002). https://doi.org/10.1109/4235.996017

    Article  Google Scholar 

  19. Deb, K., Thiele, L., Laumanns, M., Zitzler, E.: Scalable multi-objective optimization test problems. In: Proceedings of the 2002 Congress on Evolutionary Computation, pp. 825–830 (2002). https://doi.org/10.1109/CEC.2002.1007032

  20. Díaz-Manriquez, A., Toscano-Pulido, G., Gómez-Flores, W.: On the selection of surrogate models in evolutionary optimization algorithms. In: 2011 IEEE Congress of Evolutionary Computation, CEC 2011, pp. 2155–2162 (2011). https://doi.org/10.1109/CEC.2011.5949881

  21. Emmerich, M.T., Giannakoglou, K.C., Naujoks, B.: Single- and multiobjective evolutionary optimization assisted by Gaussian random field metamodels. IEEE Trans. Evolution. Comput. 10(4), 421–439 (2006). https://doi.org/10.1109/TEVC.2005.859463

    Article  Google Scholar 

  22. Eriksson, D., Bindel, D., Shoemaker, C.A.: pySOT and POAP: An event-driven asynchronous framework for surrogate optimization (2019). arXiv:1908.00420

  23. Evtushenko, Y.G., Posypkin, M.A.: A deterministic algorithm for global multi-objective optimization. Optim. Methods Softw. 29(5), 1005–1019 (2014). https://doi.org/10.1080/10556788.2013.854357

    Article  MathSciNet  MATH  Google Scholar 

  24. Farina, M., Amato, P.: A fuzzy definition of “optimality” for many-criteria optimization problems. IEEE Trans.Syst. Man Cybernet. Part A: Syst. Human. 34(3), 315–326 (2004). https://doi.org/10.1109/TSMCA.2004.824873

    Article  Google Scholar 

  25. Floudas, C., Pardalos, P.: Encyclopedia of Optimization. Springer, US (2009)

    Book  Google Scholar 

  26. Georgopoulou, C.A., Giannakoglou, K.C.: A multi-objective metamodel-assisted memetic algorithm with strength-based local refinement. Eng. Optim. 41(10), 909–923 (2009). https://doi.org/10.1080/03052150902866577

    Article  Google Scholar 

  27. Gergel, V., Kozinov, E.: Efficient multicriterial optimization based on intensive reuse of search information. J. Glob. Optim. 71, 73–90 (2018). https://doi.org/10.1007/s10898-018-0624-3

    Article  MathSciNet  MATH  Google Scholar 

  28. Goldberg, D.: Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley Longman Publishing Co., Inc, USA (1989)

    MATH  Google Scholar 

  29. Gong, W., Duan, Q., Li, J., Wang, C., Di, Z., Ye, A., Miao, C., Dai, Y.: Multiobjective adaptive surrogate modeling-based optimization for parameter estimation of large, complex geophysical models. Water Resour. Res. 52(3), 1984–2008 (2016). https://doi.org/10.1002/2015WR018230

    Article  Google Scholar 

  30. Grishagin, V.A.: Operating characteristics of some global search algorithms. Probl Stoch Search 7, 198–206 (1978) (In Russian)

  31. Hadka, D.: Platypus—Multiobjective Optimization in Python (2018). https://github.com/Project-Platypus/Platypus

  32. Hadka, D., Reed, P.: Borg: An auto-adaptive many-objective evolutionary computing framework. Evolution. Comput. 21(2), 231–259 (2013). https://doi.org/10.1162/EVCO_a_00075

    Article  Google Scholar 

  33. Horn, D., Wagner, T., Biermann, D., Weihs, C., Bischl, B.: Model-based multi-objective optimization: Taxonomy, multi-point proposal, toolbox and benchmark. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), pp. 64–78 (2015). https://doi.org/10.1007/978-3-319-15934-8_5

  34. Ilievski, I., Akhtar, T., Feng, J., Shoemaker, C.A.: Efficient hyperparameter optimization of deep learning algorithms using deterministic RBF surrogates. In: 31st AAAI Conference on Artificial Intelligence, pp. 822–829 (2017)

  35. Jones, D.R., Perttunen, C.D., Stuckman, B.E.: Lipschitzian optimization without the Lipschitz constant. J. Optim. Theory Appl. 79, 157–181 (1993). https://doi.org/10.1007/BF00941892

    Article  MathSciNet  MATH  Google Scholar 

  36. Jones, D.R., Schonlau, M., Welch, W.J.: Efficient global optimization of expensive black-box functions. J. Glob. Optim. 13(4), 455–492 (1998). https://doi.org/10.1023/A:1008306431147

    Article  MathSciNet  MATH  Google Scholar 

  37. Knowles, J.: ParEGO: a hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems. IEEE Trans. Evolution. Comput. 8(5), 1341–1366 (2006). https://doi.org/10.1109/TEVC.2005.851274

    Article  Google Scholar 

  38. Kollat, J.B., Reed, P.M.: Comparing state-of-the-art evolutionary multi-objective algorithms for long-term groundwater monitoring design. Adv. Water Resour. 29(6), 792–807 (2006). https://doi.org/10.1016/j.advwatres.2005.07.010

    Article  Google Scholar 

  39. Koziel, S., Bekasiewicz, A., Couckuyt, I., Dhaene, T.: Efficient multi-objective simulation-driven antenna design using co-kriging. IEEE Trans. Antennas Propagat. 62(11), 5900–5905 (2014). https://doi.org/10.1109/TAP.2014.2354673

    Article  MathSciNet  MATH  Google Scholar 

  40. Lai, L., Fiaschi, L., Cococcioni, M.: Solving mixed Pareto-Lexicographic multi-objective optimization problems: The case of priority chains. Swarm Evolution. Comput. (2020). https://doi.org/10.1016/j.swevo.2020.100687

    Article  Google Scholar 

  41. Laumanns, M., Thiele, L., Deb, K., Zitzler, E.: Combining convergence and diversity in evolutionary multiobjective optimization. Evolution. Comput. 10(3), 263–282 (2002). https://doi.org/10.1162/106365602760234108

    Article  Google Scholar 

  42. von Lücken, C., Brizuela, C., Barán, B.: An overview on evolutionary algorithms for many-objective optimization problems. Wiley Interdisciplin. Rev. Data Min. Knowl. Dis. 9(1), e1267 (2019). https://doi.org/10.1002/widm.1267

    Article  Google Scholar 

  43. Martínez, S.Z., Coello Coello, C.A.: A memetic algorithm with non gradient-based local search assisted by a meta-model. In: Proceedings of the 11th International Conference on Parallel Problem Solving from Nature: Part I, pp. 576–585 (2010). https://doi.org/10.1007/978-3-642-15844-5_58

  44. Matrosov, E.S., Huskova, I., Kasprzyk, J.R., Harou, J.J., Lambert, C., Reed, P.M.: Many-objective optimization and visual analytics reveal key trade-offs for London’s water supply. J. Hydrol. 531, 1040–1053 (2015). https://doi.org/10.1016/j.jhydrol.2015.11.003

    Article  Google Scholar 

  45. McKay, M.D., Beckman, R.J., Conover, W.J.: A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21(2), 239–245 (1979). https://doi.org/10.2307/1268522

    Article  MathSciNet  MATH  Google Scholar 

  46. Müller, J.: SOCEMO: Surrogate optimization of computationally expensive multiobjective problems. INFORMS J. Comput. 29(4), 581–596 (2017). https://doi.org/10.1287/ijoc.2017.0749

    Article  MathSciNet  MATH  Google Scholar 

  47. Nash, J.E., Sutcliffe, J.V.: River flow forecasting through conceptual models part I—A discussion of principles. J. Hydrol. 10(3), 282–290 (1970). https://doi.org/10.1016/0022-1694(70)90255-6

    Article  Google Scholar 

  48. Pan, L., He, C., Tian, Y., Wang, H., Zhang, X., Jin, Y.: A classification-based surrogate-assisted evolutionary algorithm for expensive many-objective optimization. IEEE Trans. Evolution. Comput. 23(1), 74–88 (2019). https://doi.org/10.1109/TEVC.2018.2802784

    Article  Google Scholar 

  49. Pardalos, P., Žilinskas, A., Žilinskas, J.: Non-convex multi-objective optimization. Springer, Cham (2017)

    Book  Google Scholar 

  50. Powell, M.: The Theory of Radial Basis Function Approximation in 1990. Oxford University Press, NY (1992)

    MATH  Google Scholar 

  51. Razavi, S., Tolson, B.A., Burn, D.H.: Review of surrogate modeling in water resources. Water Resour. Res. (2012). https://doi.org/10.1029/2011WR011527

    Article  Google Scholar 

  52. Regis, R.G., Shoemaker, C.A.: Local function approximation in evolutionary algorithms for the optimization of costly functions. IEEE Trans. Evolution. Comput. 8(5), 490–505 (2004). https://doi.org/10.1109/TEVC.2004.835247

    Article  Google Scholar 

  53. Regis, R.G., Shoemaker, C.A.: A stochastic radial basis function method for the global optimization of expensive functions. INFORMS J. Comput. 19(4), 497–509 (2007)

    Article  MathSciNet  Google Scholar 

  54. Regis, R.G., Shoemaker, C.A.: Combining radial basis function surrogates and dynamic coordinate search in high-dimensional expensive black-box optimization. Eng. Optim. 45(5), 529–555 (2013). https://doi.org/10.1080/0305215X.2012.687731

    Article  MathSciNet  Google Scholar 

  55. Rosales-Pérez, A., García, S., Gonzalez, J.A., Coello Coello, C.A., Herrera, F.: An evolutionary multiobjective model and instance selection for support vector machines with pareto-based ensembles. IEEE Trans. Evolution. Comput. 21(6), 863–877 (2017). https://doi.org/10.1109/TEVC.2017.2688863

    Article  Google Scholar 

  56. Sergeyev, Y.D., Kvasov, D.E., Mukhametzhanov, M.S.: Operational zones for comparing metaheuristic and deterministic one-dimensional global optimization algorithms. Math. Comput. Simul. 141, 96–109 (2017). https://doi.org/10.1016/j.matcom.2016.05.006

    Article  MathSciNet  MATH  Google Scholar 

  57. Shafii, M., Tolson, B.A.: Optimizing hydrological consistency by incorporating hydrological signatures into model calibration objectives. Water Resour. Res. 51(5), 3796–3814 (2015). https://doi.org/10.1002/2014WR016520

    Article  Google Scholar 

  58. Tian, Y., Cheng, R., Zhang, X., Jin, Y.: PlatEMO: A MATLAB platform for evolutionary multi-objective optimization. IEEE Comput. Intell. Magaz. 12(4), 73–87 (2017). https://doi.org/10.1109/MCI.2017.2742868

    Article  Google Scholar 

  59. Tolson, B.A., Shoemaker, C.A.: Cannonsville Reservoir Watershed SWAT2000 model development, calibration and validation. J. Hydrol. 337(1–2), 68–86 (2007). https://doi.org/10.1016/j.jhydrol.2007.01.017

    Article  Google Scholar 

  60. Vrugt, J.A., Robinson, B.A.: Improved evolutionary optimization from genetically adaptive multimethod search. In: Proceedings of the National Academy of Sciences of the United States of America, pp. 708–711 (2007). https://doi.org/10.1073/pnas.0610471104

  61. Wagner, T., Emmerich, M., Deutz, A., Ponweiser, W.: On expected-improvement criteria for model-based multi-objective optimization. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), pp. 718–727 (2010). https://doi.org/10.1007/978-3-642-15844-5_72

  62. Wilcoxon, F., Katti, S.K., Wilcox, R.A.: Critical values and probability levels for the Wilcoxon rank sum test and the Wilcoxon signed rank test. American Cyanamid, NY (1963)

    MATH  Google Scholar 

  63. Wild, S.M., Shoemaker, C.: Global convergence of radial basis function trust region derivative-free algorithms. SIAM J. Optim. 21(3), 761–781 (2011). https://doi.org/10.1137/09074927X

    Article  MathSciNet  MATH  Google Scholar 

  64. Willems, P.: A time series tool to support the multi-criteria performance evaluation of rainfall-runoff models. Environ. Modell. Softw. 24(3), 311–321 (2009). https://doi.org/10.1016/j.envsoft.2008.09.005

    Article  Google Scholar 

  65. Ye, K.Q., Li, W., Sudjianto, A.: Algorithmic construction of optimal symmetric Latin hypercube designs. J. Statist. Plann. Inference 90, 145–159 (2000). https://doi.org/10.1016/s0378-3758(00)00105-1

    Article  MathSciNet  MATH  Google Scholar 

  66. Yu, G., Jin, Y., Olhofer, M.: References or Preferences—Rethinking Many-objective Evolutionary Optimization. In: 2019 IEEE Congress on Evolutionary Computation, CEC 2019, pp. 2410–2417 (2019). https://doi.org/10.1109/CEC.2019.8790106

  67. Žilinskas, A.: On the worst-case optimal multi-objective global optimization. Optim. Lett. (2013). https://doi.org/10.1007/s11590-012-0547-8

    Article  MathSciNet  MATH  Google Scholar 

  68. Žilinskas, A., Zhigljavsky, A.: Branch and probability bound methods in multi-objective optimization. Optim. Lett. 10, 341–353 (2016). https://doi.org/10.1007/s11590-014-0777-z

    Article  MathSciNet  MATH  Google Scholar 

  69. Zitzler, E., Knowles, J., Thiele, L.: Quality assessment of pareto set approximations. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), pp. 373–404 (2008). https://doi.org/10.1007/978-3-540-88908-3-14

  70. Zitzler, E., Thiele, L.: Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach. IEEE Trans. Evolution. Comput. 3(4), 257–271 (1999). https://doi.org/10.1109/4235.797969

    Article  Google Scholar 

Download references

Acknowledgements

This work was partially supported by Prof. Shoemaker’s NUS startup grant, by the National Research Foundation (NRF), Prime Minister’s Office, Singapore under its Campus for Research Excellence and Technological Enterprise (CREATE) programme, Grant Number R-706-001-102-281, National University of Singapore, and by a MOE-Singapore scholarship for Wenyu Wang. We thank the reviewers for their careful reading of the manuscript and their helpful comments.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Christine A. Shoemaker.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 1293 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, W., Akhtar, T. & Shoemaker, C.A. Integrating \(\varepsilon \)-dominance and RBF surrogate optimization for solving computationally expensive many-objective optimization problems. J Glob Optim 82, 965–992 (2022). https://doi.org/10.1007/s10898-021-01019-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10898-021-01019-w

Keywords

Navigation