Abstract
Computationally expensive multiobjective optimization problems arise, e.g. in many engineering applications, where several conflicting objectives are to be optimized simultaneously while satisfying constraints. In many cases, the lack of explicit mathematical formulas of the objectives and constraints may necessitate conducting computationally expensive and time-consuming experiments and/or simulations. As another challenge, these problems may have either convex or nonconvex or even disconnected Pareto frontier consisting of Pareto optimal solutions. Because of the existence of many such solutions, typically, a decision maker is required to select the most preferred one. In order to deal with the high computational cost, surrogate-based methods are commonly used in the literature. This paper surveys surrogate-based methods proposed in the literature, where the methods are independent of the underlying optimization algorithm and mitigate the computational burden to capture different types of Pareto frontiers. The methods considered are classified, discussed and then compared. These methods are divided into two frameworks: the sequential and the adaptive frameworks. Based on the comparison, we recommend the adaptive framework to tackle the aforementioned challenges.
Access this article
We’re sorry, something doesn't seem to be working properly.
Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.
Similar content being viewed by others
References
Abaqus (2013). http://www.3ds.com/products-services/simulia/portfolio/abaqus/overview/. Accessed Dec 2013
Bhardwaj P, Dasgupta B, Deb K (2013) Modelling the Pareto-optimal set using b-spline basis functions for continuous multi-objective optimization problems. Eng Optim:1–27
Bornatico R, Pfeiffer M, Witzig A (2011) Untersuchung ausgewählter solarsysteme durch abtasten grosser parameterräume. In: Proceedings of the 21th OTTI solar thermal technology symposium. Bad Staffelstein, Germany
Bornatico R, Hüssy J, Witzig A, Guzzella L (2013) Surrogate modeling for the fast optimization of energy systems. Energy 57: 653–662
Buhmann MD (2003) Radial basis functions: theory and implementations. Cambridge University Press, New York
Clarke SM, Griebsch JH, Simpson TW (2004) Analysis of support vector regression for approximation of complex engineering analyses. J Mech Des 127(6):1077–1087
Chen G, Han X, Liu G, Jiang C, Zhao Z (2012) An efficient multi-objective optimization method for black-box functions using sequential approximate technique. Appl Soft Comput 12(1):14–27
Deb K, Thiele L, Laumanns M, Zitzler E (2002) Scalable multi-objective optimization test problems. In: Proceedings of congress on evolutionary computation, vol 1. IEEE, Honolulu, USA, pp 825–830
Deb K, Pratap A, Agarwal S, Meyarivan T (2002) A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans Evol Comput 6:183–197
Durillo J, Nebro A, Luna F, Alba E (2008) Solving three-objective optimization problems using a new hybrid cellular genetic algorithm. In: Rudolph G, Jansen T, Lucas C, Poloni S, Beume N (eds) Parallel problem solving from nature – PPSN X, volume 5199 of Lecture notes in computer science. Springer Berlin Heidelberg, pp 661–670
Eskandari H, Geiger CD (2008) A fast Pareto genetic algorithm approach for solving expensive multiobjective optimization problems. J Heuristics 14:203–241
Eskelinen P, Miettinen K, Klamroth K, Hakanen J (2010) Pareto navigator for interactive nonlinear multiobjective optimization. OR Spectrum 32(1):211–227
Friedman JH (1991) Multivariate adaptive regression splines. Ann Stat 19(1):1–67
Forrester AIJ, Sóbester A, Keane AJ (2008) Engineering design via surrogate modelling: a practical guide. Wiley, Chichester
Forrester AIJ, Keane AJ (2009) Recent advances in surrogate-based optimization. Prog Aerosp Sci 45:50–79
Gamito MN, Maddock SC (2009) Accurate multidimensional poisson-disk sampling. ACM Trans Graph 29 (1):1–19
Giunta A, Watson LT, Koehler J (1998) A comparison of approximation modeling techniques: polynomial versus interpolating models. In: Proceedings of 7th AIAA/USAF/NASA/ISSMO symposium on multidisciplinary analysis and optimization, vol 1. St. Louis, MO, pp 392–404. AIAA-98-4758
Gobbi M, Guarneri P, Scala L, Scotti L (2013) A local approximation based multi-objective optimization algorithm with applications. Optim Eng:1–23
Goel T, Vaidyanathan R, Haftka R, Shyy W, Queipo N, Tucker K (2004) Response surface approximation of Pareto optimal front in multi-objective optimization. In: Proceedings of the 10th AIAA/ISSMO multidisciplinary analysis and optimization conferences, Albany, NY, pp 2230–2245
Goel T, Vaidyanathan R, Haftka RT, Shyy W, Queipo NV, Tucker K (2007) Response surface approximation of Pareto optimal front in multi-objective optimization. Comput Methods Appl Mech Eng 196:879–893
Goldberg DE (1989) Genetic algorithms in search, optimization and machine learning. Addison-Wesley Longman Publishing Co., New York
Hagan MT, Demuth HB, Beale M (1996) Neural network design. PWS, Boston
Hakanen J, Miettinen K, Sahlstedt K (2011) Wastewater treatment: new insight provided by interactive multiobjective optimization. Decis Support Syst 51(2):328–337
Hartikainen M, Miettinen K, Wiecek MM (2012) PAINT: Pareto front interpolation for nonlinear multiobjective optimization. Comput Optim Appl 52(3):845–867
Helton JC, Johnson JD, Sallaberry CJ, Storlie CB (2006) Survey of sampling-based methods for uncertainty and sensitivity analysis. Reliab Eng Syst Saf 91(10-11):1175–1209
Hickernell FJ, Lemieux C, Owen AB (2005) Control variates for quasi-monte carlo. Stat Sci 20(1):1–31
Jakobsson S, Patriksson M, Rudholm J, Wojciechowski A (2010) A method for simulation based optimization using radial basis functions. Optim Eng 11(4):501–532
Jakobsson S, Saif-Ul-Hasnain M, Rundqvist R, Edelvik F, Andersson B, Patriksson M, Ljungqvist M, Lortet D, Wallesten J (2010) Combustion engine optimization: a multiobjective approach. Optim Eng 11 (4):533–554
Jin R, Chen W, Simpson TW (2001) Comparative studies of metamodelling techniques under multiple modelling criteria. Struct Multidiscip Optim 23(1):1–13
Jin Y (2011) Surrogate-assisted evolutionary computation: recent advances and future challenges. Swarm Evol Comput 1(2):61–70
Johnson ME, Moore LM, Ylvisaker D (1990) Minimax and maximin distance designs. J Stat Plan Inf 26(2):131–148
Kleijnen JPC (2009) Kriging metamodeling in simulation: a review. Eur J Oper Res 192(3):707–716
Kitayama S, Arakawa M, Yamazaki K (2011) Sequential approximate optimization using radial basis function network for engineering optimization. Optim Eng 12(4):535–557
Kitayama S, Arakawa M, Yamazaki K (2011) Differential evolution as the global optimization technique and its application to structural optimization. Appl Soft Comput 11(4):3792–3803
Khokhar ZO, Vahabzadeh H, Ziai A, Wang GG, Menon C (2010) On the performance of the PSP method for mixed-variable multi-objective design optimization. J Mech Des 132(7):1–11
Kitayama S, Srirat J, Arakawa M, Yamazaki K (2013) Sequential approximate multi-objective optimization using radial basis function network. Struct Multidiscip Optim 48(3):501–515
Knowles J (2006) ParEGO: a hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems. IEEE Trans Evol Comput 10(1):50–66
Knowles J, Nakayama H (2008) Meta-modeling in multiobjective optimization. In: Blaszczynski J, Jin Y, Shimoyama K, Slowinski R (eds) Multiobjective optimization: interactive and evolutionary approaches. Springer-Verlag Berlin Heidelberg, pp 245–284
Koziel S, Ciaurri DE, Leifsson L (2011) Surrogate-based methods. In: Koziel S, Yang X (eds) Studies in computational intelligence, vol 356. Springer Berlin Heidelberg, pp 33–59
Kursawe F (1991) A variant of evolution strategies for vector optimization. In: Schwefel HP, Männer R (eds) Parallel Problem Solving from Nature, vol 496. Springer Berlin Heidelberg, pp 193–197
Kutner MH, Nachtsheim CJ, Neter J, Li W (2005) Applied linear statistical models. McGraw Hill, Boston
Li YF, Ng SH, Xie M, Goh TN (2010) A systematic comparison of metamodeling techniques for simulation optimization in decision support systems. Appl Soft Comput 10(4):1257–1273
Liao X, Li Q, Yang X, Zhang W, Li W (2008) Multiobjective optimization for crash safety design of vehicles using stepwise regression model. Struct Multidiscip Optim 35(6):561–569
Liu GP, Han X (2006) A micro multi-objective genetic algorithm for multi-objective optimizations. In: The 4th China–Japan–Korea joint symposium on optimization of structural and mechanical systems, Kunming, China
Liu GP, Han X, Jiang C (2008) A novel multi-objective optimization method based on an approximation model management technique. Comput Methods Appl Mech Eng 197:2719–2731
Livermore Software Technology Corporation (LSTC) (2013). http://www.lstc.com/products/ls-dyna. Accessed Dec 2013
Lotov AV, Bushenkov VA, Kamenev GK (2001) Feasible goals method. Search for smart decisions. Computing Center of RAS, Moscow
Lotov AV, Bushenkov VA, Kamenev GK (2004) Interactive decision maps: approximation and visualization of Pareto frontier. Kluwer Academic Publishers, Massachusetts
Luque M, Ruiz F, Miettinen K (2011) Global formulation for interactive multiobjective optimization. OR Spectrum 33(1):27–48
Madsen JI, Shyy W, Haftka RT (2000) Response surface techniques for diffuser shape optimization. AIAA J 3(9):1512–1518
Marler RT, Arora JS (2004) Survey of multi-objective optimization methods for engineering. Struct Multidiscip Optim 26(6):369–395
Messac A, Ismail-Yahaya A, Mattson CA (2003) The normalized normal constraint method for generating the Pareto frontier. Struct Multidiscip Optim 25(2):86–98
Messac A, Mattson CA (2004) Normal constraint method with guarantee of even representation of complete Pareto frontier. AIAA J 42(10):2101–2111
Messac A, Mullur AA (2008) A computationally efficient metamodeling approach for expensive multiobjective optimization. Optim Eng 9(1):37–67
Miettinen K (1999) Nonlinear multiobjective optimization. Kluwer Academic Publishers, Norwell
Miettinen K, Ruiz F, Wierzbicki AP (2008) Introduction to multiobjective optimization: interactive approaches. In: Branke J, Deb K, Miettinen K, Slowinski R (eds) Multiobjective optimization: interactive and evolutionary approaches. Springer-Verlag Berlin Heidelberg, pp 27–57
Miettinen K (2014) Survey of methods to visualize alternatives in multiple criteria decision making problems. OR Spectrum 36(1): 3–37
Moldex3d: Plastic injection molding simulation software (2013). http://www.moldex3d.com/en/. Accessed Dec 2013
Monz M, Küfer KH, Bortfeld TR, Thieke C (2008) Pareto navigation—algorithmic foundation of interactive multi-criteria IMRT planning. Phys Med Biol 53(4):985–998
MSC Nastran-Multidisciplinary structural analysis (2013). http://www.mscsoftware.com/product/msc-nastran, Accessed Dec 2013
Mullur AA, Messac A (2005) Extended radial basis functions: more flexible and effective metamodeling. AIAA J 43(6):1306–1315
Nakayama H, Yun Y, Yoon M (2009) Sequential approximate multiobjective optimization using computational intelligence, 1st edn. Springer Publishing Company, Incorporated, Berlin
Nebro AJ, Luna F, Alba E, Dorronsoro B, Durillo JJ, Beham A (2008) AbYSS: adapting scatter search to multiobjective optimization. IEEE Trans Evol Comput 12:439–457
Niederreiter H (1992) Random number generation and quasi-Monte Carlo methods, Philadelphia
Okabe T, Jin Y, Sendhoff B (2003) A critical survey of performance indices for multi-objective optimisation. In: The 2003 congress on Evolutionary computation, 2003. CEC ’03. vol 2, pp 878–885
Pascoletti A, Serafini P (1984) Scalarizing vector optimization problems. J Optim Theory Appl 42(4):499–524
Polysun (2013). http://www.polysunonline.com/PsoPublic/app/home/access. Accessed Dec 2013
Queipo NV, Haftka RT, Shyy W, Goel T, Vaidyanathan R, Tucker KP (2005) Surrogate-based analysis and optimization. Prog Aerosp Sci 41(1):1–28
Reyes M, Coello CA (2005) Improving PSO-based multi-objective optimization using crowding, mutation and epsilon-dominance. In: Coello Coello C, Aguirre AH, Zitzler E (eds) Proceedings of 3rd international conference on evolutionary multi-criterion optimization, number 505–519 in EMO’05, Guanajuato, Mexico. Springer, Berlin
Rezaveisi M, Sepehrnoori K, Johns RT (2014) Tie-simplex-based phase-behavior modeling in an IMPEC reservoir simulator. Soc Pet Eng 19(02):327–339
Ruiz F, Luque M, Miettinen K (2012) Improving the computational efficiency in a global formulation (GLIDE) for interactive multiobjective optimization. OR Spectrum 197(1):47–70
Schaumann E, Balling R, Day K (1998) Genetic algorithms with multiple objectives. In: Proceedings of the 7th AIAA/USAF/ NASA/ISSMO symposium on multidisciplinary analysis and optimization, vol 3. Washington, DC, pp 2114–2123
Shan S, Wang GG (2004) An efficient Pareto set identification approach for multiobjective optimization on black-box functions. J Mech Des 127(5):866–874
Shan S, Wang GG (2010) Survey of modeling and optimization strategies to solve high-dimensional design problems with computationally-expensive black-box functions. Struct Multidiscip Optim 41(2):219–241
Simpson TW, Poplinski JD, Koch PN, Allen JK (2001) Metamodels for computer-based engineering design: survey and recommendations. Engineering with Computers 17(2):129–150
Simpson TW, Booker AJ, Ghosh D, Giunta AA, Koch PN, Yang RJ (2004) Approximation methods in multidisciplinary analysis and optimization: a panel discussion. Struct Multidiscip Optim 27(5):302–313
Simpson TW, Toropov V, Balabanov V, Viana F (2008) Design and analysis of computer experiments in multidisciplinary design optimization: a review of how far we have come - or not. In: Proceedings of the 12th AIAA/ISSMO multidisciplinary analysis and optimization conference, pages AIAA–2008–5802, British Colombia
Smola AJ, Schökopf B (2004) A tutorial on support vector regression. Stat Comput 14(3):199–222
Star-CD (2013). http://www.cd-adapco.com/products/star-cd. Accessed Dec 2013
Steinhaus H (2011) Mathematical snapshots. Courier Dover Publications, New York
Steuer RE (1986) Multiple criteria optimization: theory, computation and application. Wiley, New York
Su R, Gui L, Fan Z (2011) Multi-objective optimization for bus body with strength and rollover safety constraints based on surrogate models. Struct Multidiscip Optim 44(3):431–441
Su R, Wang X, Gui L, Fan Z (2011) Multi-objective topology and sizing optimization of truss structures based on adaptive multi-island search strategy. Struct Multidiscip Optim 43(2):275– 286
Tenne Yl, Goh CK (2010) Computational intelligence in expensive optimization problems, 1st edition. Springer Publishing Company, Berlin
Wang L, Shan S, Wang GG (2004) Mode-pursuing sampling method for global optimization on expensive black-box functions. J Eng Optim 36(4):419–438
Wang GG, Shan S (2006) Review of metamodeling techniques in support of engineering design optimization. J Mech Des 129(4): 370–380
Wierzbicki AP (1986) On the completeness and constructiveness of parametric characterizations to vector optimization problems. OR Spectrum 8(2):73–87
Wilson B, Cappelleri D, Simpson TW, Frecker M (2001) Efficient Pareto frontier exploration using surrogate approximations. Optim Eng 2(1):31–50
Wu J, Azarm S (2000) Metrics for quality assessment of a multiobjective design optimization solution set. J Mech Des 123(1):18–25
Wagner T, Emmerich M, Deutz A, Ponweiser W (2010). In: Schaefer R, Cotta C, Kolodziej J, Rudolph G (eds) On expected-improvement criteria for model-based multi-objective optimization. Springer, Berlin
Yang BS, Yeun YS, Ruy WS (2002) Managing approximation models in multiobjective optimization. Struct Multidiscip Optim 24(2):141–156
Yun Y, Yoon M, Nakayama H (2009) Multi-objective optimization based on meta-modeling by using support vector regression. Optim Eng 10(2):167–181
Zanakis SH, Solomon A, Wishart N, Dublish S (1998) Multi-attribute decision making: a simulation comparison of select methods. Eur J Oper Res 107(3):507–529
Zitzler E, Laumanns M, Thiele L (2001) SPEA2: Improving the strength Pareto evolutionary algorithm. Technical Report 103, Computer Engineering and Networks Laboratory (TIK), Department of Electrical Engineering, Swiss Federal Institute of Technology (ETH)
Zitzler E, Laumanns M, Thiele L (2002) SPEA2: Improving the strength Pareto evolutionary algorithm for multiobjective optimization. In: Proceedings of the conference on evolutionary methods for design optimization and control, CIMNE, Barcelona, Spain, pp 95–100
Zitzler E, Thiele L, Laumanns M, Fonseca CM, da Fonseca VG (2003) Performance assessment of multiobjective optimizers: an analysis and review. IEEE Trans Evol Comput 7(2):117–132
Zitzler E, Knowles J, Thiele L (2008) Quality assessment of pareto set approximations, vol 5252. Springer, Berlin, pp 373– 404
Zhou J, Turng LS (2007) Adaptive multiobjective optimization of process conditions for injection molding using a Gaussian process approach. Adv Polym Technol 26(2):71–85
Zhou A, Qu BY, Li H, Zhao SZ, Suganthan PN, Zhang Q (2011) Multiobjective evolutionary algorithms: a survey of the state of the art. Swarm Evol Comput 1(1):32–49
Acknowledgments
Mohammad Tabatabaei thanks the COMAS doctoral program in computing and mathematical sciences and Karthik Sindhya thanks the TEKES -the Finnish Funding Agency for Technology and Innovation (the SIMPRO project) for the financial assistance.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Tabatabaei, M., Hakanen, J., Hartikainen, M. et al. A survey on handling computationally expensive multiobjective optimization problems using surrogates: non-nature inspired methods. Struct Multidisc Optim 52, 1–25 (2015). https://doi.org/10.1007/s00158-015-1226-z
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00158-015-1226-z