Skip to main content
SpringerLink
Log in

A survey on handling computationally expensive multiobjective optimization problems using surrogates: non-nature inspired methods

  • REVIEW ARTICLE
  • Published:
Structural and Multidisciplinary Optimization Aims and scope Submit manuscript

Abstract

Computationally expensive multiobjective optimization problems arise, e.g. in many engineering applications, where several conflicting objectives are to be optimized simultaneously while satisfying constraints. In many cases, the lack of explicit mathematical formulas of the objectives and constraints may necessitate conducting computationally expensive and time-consuming experiments and/or simulations. As another challenge, these problems may have either convex or nonconvex or even disconnected Pareto frontier consisting of Pareto optimal solutions. Because of the existence of many such solutions, typically, a decision maker is required to select the most preferred one. In order to deal with the high computational cost, surrogate-based methods are commonly used in the literature. This paper surveys surrogate-based methods proposed in the literature, where the methods are independent of the underlying optimization algorithm and mitigate the computational burden to capture different types of Pareto frontiers. The methods considered are classified, discussed and then compared. These methods are divided into two frameworks: the sequential and the adaptive frameworks. Based on the comparison, we recommend the adaptive framework to tackle the aforementioned challenges.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  • Abaqus (2013). http://www.3ds.com/products-services/simulia/portfolio/abaqus/overview/. Accessed Dec 2013

  • Bhardwaj P, Dasgupta B, Deb K (2013) Modelling the Pareto-optimal set using b-spline basis functions for continuous multi-objective optimization problems. Eng Optim:1–27

  • Bornatico R, Pfeiffer M, Witzig A (2011) Untersuchung ausgewählter solarsysteme durch abtasten grosser parameterräume. In: Proceedings of the 21th OTTI solar thermal technology symposium. Bad Staffelstein, Germany

    Google Scholar 

  • Bornatico R, Hüssy J, Witzig A, Guzzella L (2013) Surrogate modeling for the fast optimization of energy systems. Energy 57: 653–662

    Article  Google Scholar 

  • Buhmann MD (2003) Radial basis functions: theory and implementations. Cambridge University Press, New York

    Book  Google Scholar 

  • Clarke SM, Griebsch JH, Simpson TW (2004) Analysis of support vector regression for approximation of complex engineering analyses. J Mech Des 127(6):1077–1087

    Article  Google Scholar 

  • Chen G, Han X, Liu G, Jiang C, Zhao Z (2012) An efficient multi-objective optimization method for black-box functions using sequential approximate technique. Appl Soft Comput 12(1):14–27

    Article  Google Scholar 

  • Deb K, Thiele L, Laumanns M, Zitzler E (2002) Scalable multi-objective optimization test problems. In: Proceedings of congress on evolutionary computation, vol 1. IEEE, Honolulu, USA, pp 825–830

  • Deb K, Pratap A, Agarwal S, Meyarivan T (2002) A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans Evol Comput 6:183–197

    Article  Google Scholar 

  • Durillo J, Nebro A, Luna F, Alba E (2008) Solving three-objective optimization problems using a new hybrid cellular genetic algorithm. In: Rudolph G, Jansen T, Lucas C, Poloni S, Beume N (eds) Parallel problem solving from nature – PPSN X, volume 5199 of Lecture notes in computer science. Springer Berlin Heidelberg, pp 661–670

  • Eskandari H, Geiger CD (2008) A fast Pareto genetic algorithm approach for solving expensive multiobjective optimization problems. J Heuristics 14:203–241

    Article  MATH  Google Scholar 

  • Eskelinen P, Miettinen K, Klamroth K, Hakanen J (2010) Pareto navigator for interactive nonlinear multiobjective optimization. OR Spectrum 32(1):211–227

    Article  MathSciNet  MATH  Google Scholar 

  • Friedman JH (1991) Multivariate adaptive regression splines. Ann Stat 19(1):1–67

    Article  MATH  Google Scholar 

  • Forrester AIJ, Sóbester A, Keane AJ (2008) Engineering design via surrogate modelling: a practical guide. Wiley, Chichester

    Book  Google Scholar 

  • Forrester AIJ, Keane AJ (2009) Recent advances in surrogate-based optimization. Prog Aerosp Sci 45:50–79

    Article  Google Scholar 

  • Gamito MN, Maddock SC (2009) Accurate multidimensional poisson-disk sampling. ACM Trans Graph 29 (1):1–19

    Article  Google Scholar 

  • Giunta A, Watson LT, Koehler J (1998) A comparison of approximation modeling techniques: polynomial versus interpolating models. In: Proceedings of 7th AIAA/USAF/NASA/ISSMO symposium on multidisciplinary analysis and optimization, vol 1. St. Louis, MO, pp 392–404. AIAA-98-4758

  • Gobbi M, Guarneri P, Scala L, Scotti L (2013) A local approximation based multi-objective optimization algorithm with applications. Optim Eng:1–23

  • Goel T, Vaidyanathan R, Haftka R, Shyy W, Queipo N, Tucker K (2004) Response surface approximation of Pareto optimal front in multi-objective optimization. In: Proceedings of the 10th AIAA/ISSMO multidisciplinary analysis and optimization conferences, Albany, NY, pp 2230–2245

  • Goel T, Vaidyanathan R, Haftka RT, Shyy W, Queipo NV, Tucker K (2007) Response surface approximation of Pareto optimal front in multi-objective optimization. Comput Methods Appl Mech Eng 196:879–893

    Article  MATH  Google Scholar 

  • Goldberg DE (1989) Genetic algorithms in search, optimization and machine learning. Addison-Wesley Longman Publishing Co., New York

    MATH  Google Scholar 

  • Hagan MT, Demuth HB, Beale M (1996) Neural network design. PWS, Boston

    Google Scholar 

  • Hakanen J, Miettinen K, Sahlstedt K (2011) Wastewater treatment: new insight provided by interactive multiobjective optimization. Decis Support Syst 51(2):328–337

    Article  Google Scholar 

  • Hartikainen M, Miettinen K, Wiecek MM (2012) PAINT: Pareto front interpolation for nonlinear multiobjective optimization. Comput Optim Appl 52(3):845–867

    Article  MathSciNet  MATH  Google Scholar 

  • Helton JC, Johnson JD, Sallaberry CJ, Storlie CB (2006) Survey of sampling-based methods for uncertainty and sensitivity analysis. Reliab Eng Syst Saf 91(10-11):1175–1209

    Article  Google Scholar 

  • Hickernell FJ, Lemieux C, Owen AB (2005) Control variates for quasi-monte carlo. Stat Sci 20(1):1–31

    Article  MathSciNet  MATH  Google Scholar 

  • Jakobsson S, Patriksson M, Rudholm J, Wojciechowski A (2010) A method for simulation based optimization using radial basis functions. Optim Eng 11(4):501–532

    Article  MathSciNet  MATH  Google Scholar 

  • Jakobsson S, Saif-Ul-Hasnain M, Rundqvist R, Edelvik F, Andersson B, Patriksson M, Ljungqvist M, Lortet D, Wallesten J (2010) Combustion engine optimization: a multiobjective approach. Optim Eng 11 (4):533–554

    Article  MathSciNet  MATH  Google Scholar 

  • Jin R, Chen W, Simpson TW (2001) Comparative studies of metamodelling techniques under multiple modelling criteria. Struct Multidiscip Optim 23(1):1–13

    Article  Google Scholar 

  • Jin Y (2011) Surrogate-assisted evolutionary computation: recent advances and future challenges. Swarm Evol Comput 1(2):61–70

    Article  Google Scholar 

  • Johnson ME, Moore LM, Ylvisaker D (1990) Minimax and maximin distance designs. J Stat Plan Inf 26(2):131–148

    Article  MathSciNet  Google Scholar 

  • Kleijnen JPC (2009) Kriging metamodeling in simulation: a review. Eur J Oper Res 192(3):707–716

    Article  MathSciNet  MATH  Google Scholar 

  • Kitayama S, Arakawa M, Yamazaki K (2011) Sequential approximate optimization using radial basis function network for engineering optimization. Optim Eng 12(4):535–557

    Article  MathSciNet  MATH  Google Scholar 

  • Kitayama S, Arakawa M, Yamazaki K (2011) Differential evolution as the global optimization technique and its application to structural optimization. Appl Soft Comput 11(4):3792–3803

    Article  Google Scholar 

  • Khokhar ZO, Vahabzadeh H, Ziai A, Wang GG, Menon C (2010) On the performance of the PSP method for mixed-variable multi-objective design optimization. J Mech Des 132(7):1–11

    Article  Google Scholar 

  • Kitayama S, Srirat J, Arakawa M, Yamazaki K (2013) Sequential approximate multi-objective optimization using radial basis function network. Struct Multidiscip Optim 48(3):501–515

    Article  MathSciNet  Google Scholar 

  • Knowles J (2006) ParEGO: a hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems. IEEE Trans Evol Comput 10(1):50–66

    Article  Google Scholar 

  • Knowles J, Nakayama H (2008) Meta-modeling in multiobjective optimization. In: Blaszczynski J, Jin Y, Shimoyama K, Slowinski R (eds) Multiobjective optimization: interactive and evolutionary approaches. Springer-Verlag Berlin Heidelberg, pp 245–284

  • Koziel S, Ciaurri DE, Leifsson L (2011) Surrogate-based methods. In: Koziel S, Yang X (eds) Studies in computational intelligence, vol 356. Springer Berlin Heidelberg, pp 33–59

  • Kursawe F (1991) A variant of evolution strategies for vector optimization. In: Schwefel HP, Männer R (eds) Parallel Problem Solving from Nature, vol 496. Springer Berlin Heidelberg, pp 193–197

  • Kutner MH, Nachtsheim CJ, Neter J, Li W (2005) Applied linear statistical models. McGraw Hill, Boston

    Google Scholar 

  • Li YF, Ng SH, Xie M, Goh TN (2010) A systematic comparison of metamodeling techniques for simulation optimization in decision support systems. Appl Soft Comput 10(4):1257–1273

    Article  Google Scholar 

  • Liao X, Li Q, Yang X, Zhang W, Li W (2008) Multiobjective optimization for crash safety design of vehicles using stepwise regression model. Struct Multidiscip Optim 35(6):561–569

    Article  Google Scholar 

  • Liu GP, Han X (2006) A micro multi-objective genetic algorithm for multi-objective optimizations. In: The 4th China–Japan–Korea joint symposium on optimization of structural and mechanical systems, Kunming, China

  • Liu GP, Han X, Jiang C (2008) A novel multi-objective optimization method based on an approximation model management technique. Comput Methods Appl Mech Eng 197:2719–2731

    Article  MATH  Google Scholar 

  • Livermore Software Technology Corporation (LSTC) (2013). http://www.lstc.com/products/ls-dyna. Accessed Dec 2013

  • Lotov AV, Bushenkov VA, Kamenev GK (2001) Feasible goals method. Search for smart decisions. Computing Center of RAS, Moscow

    Google Scholar 

  • Lotov AV, Bushenkov VA, Kamenev GK (2004) Interactive decision maps: approximation and visualization of Pareto frontier. Kluwer Academic Publishers, Massachusetts

    Book  Google Scholar 

  • Luque M, Ruiz F, Miettinen K (2011) Global formulation for interactive multiobjective optimization. OR Spectrum 33(1):27–48

    Article  MathSciNet  MATH  Google Scholar 

  • Madsen JI, Shyy W, Haftka RT (2000) Response surface techniques for diffuser shape optimization. AIAA J 3(9):1512–1518

    Article  Google Scholar 

  • Marler RT, Arora JS (2004) Survey of multi-objective optimization methods for engineering. Struct Multidiscip Optim 26(6):369–395

    Article  MathSciNet  MATH  Google Scholar 

  • Messac A, Ismail-Yahaya A, Mattson CA (2003) The normalized normal constraint method for generating the Pareto frontier. Struct Multidiscip Optim 25(2):86–98

    Article  MathSciNet  MATH  Google Scholar 

  • Messac A, Mattson CA (2004) Normal constraint method with guarantee of even representation of complete Pareto frontier. AIAA J 42(10):2101–2111

    Article  Google Scholar 

  • Messac A, Mullur AA (2008) A computationally efficient metamodeling approach for expensive multiobjective optimization. Optim Eng 9(1):37–67

    Article  MathSciNet  Google Scholar 

  • Miettinen K (1999) Nonlinear multiobjective optimization. Kluwer Academic Publishers, Norwell

    MATH  Google Scholar 

  • Miettinen K, Ruiz F, Wierzbicki AP (2008) Introduction to multiobjective optimization: interactive approaches. In: Branke J, Deb K, Miettinen K, Slowinski R (eds) Multiobjective optimization: interactive and evolutionary approaches. Springer-Verlag Berlin Heidelberg, pp 27–57

  • Miettinen K (2014) Survey of methods to visualize alternatives in multiple criteria decision making problems. OR Spectrum 36(1): 3–37

    Article  MathSciNet  MATH  Google Scholar 

  • Moldex3d: Plastic injection molding simulation software (2013). http://www.moldex3d.com/en/. Accessed Dec 2013

  • Monz M, Küfer KH, Bortfeld TR, Thieke C (2008) Pareto navigation—algorithmic foundation of interactive multi-criteria IMRT planning. Phys Med Biol 53(4):985–998

    Article  Google Scholar 

  • MSC Nastran-Multidisciplinary structural analysis (2013). http://www.mscsoftware.com/product/msc-nastran, Accessed Dec 2013

  • Mullur AA, Messac A (2005) Extended radial basis functions: more flexible and effective metamodeling. AIAA J 43(6):1306–1315

    Article  Google Scholar 

  • Nakayama H, Yun Y, Yoon M (2009) Sequential approximate multiobjective optimization using computational intelligence, 1st edn. Springer Publishing Company, Incorporated, Berlin

    MATH  Google Scholar 

  • Nebro AJ, Luna F, Alba E, Dorronsoro B, Durillo JJ, Beham A (2008) AbYSS: adapting scatter search to multiobjective optimization. IEEE Trans Evol Comput 12:439–457

    Article  Google Scholar 

  • Niederreiter H (1992) Random number generation and quasi-Monte Carlo methods, Philadelphia

  • Okabe T, Jin Y, Sendhoff B (2003) A critical survey of performance indices for multi-objective optimisation. In: The 2003 congress on Evolutionary computation, 2003. CEC ’03. vol 2, pp 878–885

  • Pascoletti A, Serafini P (1984) Scalarizing vector optimization problems. J Optim Theory Appl 42(4):499–524

    Article  MathSciNet  MATH  Google Scholar 

  • Polysun (2013). http://www.polysunonline.com/PsoPublic/app/home/access. Accessed Dec 2013

  • Queipo NV, Haftka RT, Shyy W, Goel T, Vaidyanathan R, Tucker KP (2005) Surrogate-based analysis and optimization. Prog Aerosp Sci 41(1):1–28

    Article  Google Scholar 

  • Reyes M, Coello CA (2005) Improving PSO-based multi-objective optimization using crowding, mutation and epsilon-dominance. In: Coello Coello C, Aguirre AH, Zitzler E (eds) Proceedings of 3rd international conference on evolutionary multi-criterion optimization, number 505–519 in EMO’05, Guanajuato, Mexico. Springer, Berlin

    Google Scholar 

  • Rezaveisi M, Sepehrnoori K, Johns RT (2014) Tie-simplex-based phase-behavior modeling in an IMPEC reservoir simulator. Soc Pet Eng 19(02):327–339

    Google Scholar 

  • Ruiz F, Luque M, Miettinen K (2012) Improving the computational efficiency in a global formulation (GLIDE) for interactive multiobjective optimization. OR Spectrum 197(1):47–70

    MathSciNet  Google Scholar 

  • Schaumann E, Balling R, Day K (1998) Genetic algorithms with multiple objectives. In: Proceedings of the 7th AIAA/USAF/ NASA/ISSMO symposium on multidisciplinary analysis and optimization, vol 3. Washington, DC, pp 2114–2123

  • Shan S, Wang GG (2004) An efficient Pareto set identification approach for multiobjective optimization on black-box functions. J Mech Des 127(5):866–874

    Article  MathSciNet  Google Scholar 

  • Shan S, Wang GG (2010) Survey of modeling and optimization strategies to solve high-dimensional design problems with computationally-expensive black-box functions. Struct Multidiscip Optim 41(2):219–241

    Article  MathSciNet  MATH  Google Scholar 

  • Simpson TW, Poplinski JD, Koch PN, Allen JK (2001) Metamodels for computer-based engineering design: survey and recommendations. Engineering with Computers 17(2):129–150

    Article  MATH  Google Scholar 

  • Simpson TW, Booker AJ, Ghosh D, Giunta AA, Koch PN, Yang RJ (2004) Approximation methods in multidisciplinary analysis and optimization: a panel discussion. Struct Multidiscip Optim 27(5):302–313

    Article  Google Scholar 

  • Simpson TW, Toropov V, Balabanov V, Viana F (2008) Design and analysis of computer experiments in multidisciplinary design optimization: a review of how far we have come - or not. In: Proceedings of the 12th AIAA/ISSMO multidisciplinary analysis and optimization conference, pages AIAA–2008–5802, British Colombia

  • Smola AJ, Schökopf B (2004) A tutorial on support vector regression. Stat Comput 14(3):199–222

    Article  MathSciNet  Google Scholar 

  • Star-CD (2013). http://www.cd-adapco.com/products/star-cd. Accessed Dec 2013

  • Steinhaus H (2011) Mathematical snapshots. Courier Dover Publications, New York

    Google Scholar 

  • Steuer RE (1986) Multiple criteria optimization: theory, computation and application. Wiley, New York

    MATH  Google Scholar 

  • Su R, Gui L, Fan Z (2011) Multi-objective optimization for bus body with strength and rollover safety constraints based on surrogate models. Struct Multidiscip Optim 44(3):431–441

    Article  MathSciNet  MATH  Google Scholar 

  • Su R, Wang X, Gui L, Fan Z (2011) Multi-objective topology and sizing optimization of truss structures based on adaptive multi-island search strategy. Struct Multidiscip Optim 43(2):275– 286

    Article  Google Scholar 

  • Tenne Yl, Goh CK (2010) Computational intelligence in expensive optimization problems, 1st edition. Springer Publishing Company, Berlin

    MATH  Google Scholar 

  • Wang L, Shan S, Wang GG (2004) Mode-pursuing sampling method for global optimization on expensive black-box functions. J Eng Optim 36(4):419–438

    Article  Google Scholar 

  • Wang GG, Shan S (2006) Review of metamodeling techniques in support of engineering design optimization. J Mech Des 129(4): 370–380

    Article  Google Scholar 

  • Wierzbicki AP (1986) On the completeness and constructiveness of parametric characterizations to vector optimization problems. OR Spectrum 8(2):73–87

    Article  MathSciNet  MATH  Google Scholar 

  • Wilson B, Cappelleri D, Simpson TW, Frecker M (2001) Efficient Pareto frontier exploration using surrogate approximations. Optim Eng 2(1):31–50

    Article  MathSciNet  Google Scholar 

  • Wu J, Azarm S (2000) Metrics for quality assessment of a multiobjective design optimization solution set. J Mech Des 123(1):18–25

    Article  Google Scholar 

  • Wagner T, Emmerich M, Deutz A, Ponweiser W (2010). In: Schaefer R, Cotta C, Kolodziej J, Rudolph G (eds) On expected-improvement criteria for model-based multi-objective optimization. Springer, Berlin

  • Yang BS, Yeun YS, Ruy WS (2002) Managing approximation models in multiobjective optimization. Struct Multidiscip Optim 24(2):141–156

    Article  Google Scholar 

  • Yun Y, Yoon M, Nakayama H (2009) Multi-objective optimization based on meta-modeling by using support vector regression. Optim Eng 10(2):167–181

    Article  MathSciNet  MATH  Google Scholar 

  • Zanakis SH, Solomon A, Wishart N, Dublish S (1998) Multi-attribute decision making: a simulation comparison of select methods. Eur J Oper Res 107(3):507–529

    Article  MATH  Google Scholar 

  • Zitzler E, Laumanns M, Thiele L (2001) SPEA2: Improving the strength Pareto evolutionary algorithm. Technical Report 103, Computer Engineering and Networks Laboratory (TIK), Department of Electrical Engineering, Swiss Federal Institute of Technology (ETH)

  • Zitzler E, Laumanns M, Thiele L (2002) SPEA2: Improving the strength Pareto evolutionary algorithm for multiobjective optimization. In: Proceedings of the conference on evolutionary methods for design optimization and control, CIMNE, Barcelona, Spain, pp 95–100

  • Zitzler E, Thiele L, Laumanns M, Fonseca CM, da Fonseca VG (2003) Performance assessment of multiobjective optimizers: an analysis and review. IEEE Trans Evol Comput 7(2):117–132

    Article  MATH  Google Scholar 

  • Zitzler E, Knowles J, Thiele L (2008) Quality assessment of pareto set approximations, vol 5252. Springer, Berlin, pp 373– 404

    Google Scholar 

  • Zhou J, Turng LS (2007) Adaptive multiobjective optimization of process conditions for injection molding using a Gaussian process approach. Adv Polym Technol 26(2):71–85

    Article  Google Scholar 

  • Zhou A, Qu BY, Li H, Zhao SZ, Suganthan PN, Zhang Q (2011) Multiobjective evolutionary algorithms: a survey of the state of the art. Swarm Evol Comput 1(1):32–49

    Article  Google Scholar 

Download references

Acknowledgments

Mohammad Tabatabaei thanks the COMAS doctoral program in computing and mathematical sciences and Karthik Sindhya thanks the TEKES -the Finnish Funding Agency for Technology and Innovation (the SIMPRO project) for the financial assistance.

Author information

Authors and Affiliations

  1. University of Jyvaskyla, Department of Mathematical Information Technology, P.O. Box 35 (Agora), FI-40014, University of Jyvaskyla, Finland

    Mohammad Tabatabaei, Jussi Hakanen, Markus Hartikainen, Kaisa Miettinen & Karthik Sindhya

Authors
  1. Mohammad Tabatabaei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jussi Hakanen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Markus Hartikainen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kaisa Miettinen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Karthik Sindhya
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohammad Tabatabaei.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tabatabaei, M., Hakanen, J., Hartikainen, M. et al. A survey on handling computationally expensive multiobjective optimization problems using surrogates: non-nature inspired methods. Struct Multidisc Optim 52, 1–25 (2015). https://doi.org/10.1007/s00158-015-1226-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00158-015-1226-z

Keywords

Search

Navigation