A survey on handling computationally expensive multiobjective optimization problems using surrogates: non-nature inspired methods

  • Mohammad Tabatabaei
  • Jussi Hakanen
  • Markus Hartikainen
  • Kaisa Miettinen
  • Karthik Sindhya
REVIEW ARTICLE

Abstract

Computationally expensive multiobjective optimization problems arise, e.g. in many engineering applications, where several conflicting objectives are to be optimized simultaneously while satisfying constraints. In many cases, the lack of explicit mathematical formulas of the objectives and constraints may necessitate conducting computationally expensive and time-consuming experiments and/or simulations. As another challenge, these problems may have either convex or nonconvex or even disconnected Pareto frontier consisting of Pareto optimal solutions. Because of the existence of many such solutions, typically, a decision maker is required to select the most preferred one. In order to deal with the high computational cost, surrogate-based methods are commonly used in the literature. This paper surveys surrogate-based methods proposed in the literature, where the methods are independent of the underlying optimization algorithm and mitigate the computational burden to capture different types of Pareto frontiers. The methods considered are classified, discussed and then compared. These methods are divided into two frameworks: the sequential and the adaptive frameworks. Based on the comparison, we recommend the adaptive framework to tackle the aforementioned challenges.

Keywords

Multicriteria Decision Making (MCDM) Black-box function Computational cost Metamodeling technique Sampling technique Pareto frontier 

Notes

Acknowledgments

Mohammad Tabatabaei thanks the COMAS doctoral program in computing and mathematical sciences and Karthik Sindhya thanks the TEKES -the Finnish Funding Agency for Technology and Innovation (the SIMPRO project) for the financial assistance.

References

  1. Bhardwaj P, Dasgupta B, Deb K (2013) Modelling the Pareto-optimal set using b-spline basis functions for continuous multi-objective optimization problems. Eng Optim:1–27Google Scholar
  2. Bornatico R, Pfeiffer M, Witzig A (2011) Untersuchung ausgewählter solarsysteme durch abtasten grosser parameterräume. In: Proceedings of the 21th OTTI solar thermal technology symposium. Bad Staffelstein, GermanyGoogle Scholar
  3. Bornatico R, Hüssy J, Witzig A, Guzzella L (2013) Surrogate modeling for the fast optimization of energy systems. Energy 57: 653–662CrossRefGoogle Scholar
  4. Buhmann MD (2003) Radial basis functions: theory and implementations. Cambridge University Press, New YorkCrossRefGoogle Scholar
  5. Clarke SM, Griebsch JH, Simpson TW (2004) Analysis of support vector regression for approximation of complex engineering analyses. J Mech Des 127(6):1077–1087CrossRefGoogle Scholar
  6. Chen G, Han X, Liu G, Jiang C, Zhao Z (2012) An efficient multi-objective optimization method for black-box functions using sequential approximate technique. Appl Soft Comput 12(1):14–27CrossRefGoogle Scholar
  7. Deb K, Thiele L, Laumanns M, Zitzler E (2002) Scalable multi-objective optimization test problems. In: Proceedings of congress on evolutionary computation, vol 1. IEEE, Honolulu, USA, pp 825–830Google Scholar
  8. Deb K, Pratap A, Agarwal S, Meyarivan T (2002) A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans Evol Comput 6:183–197CrossRefGoogle Scholar
  9. Durillo J, Nebro A, Luna F, Alba E (2008) Solving three-objective optimization problems using a new hybrid cellular genetic algorithm. In: Rudolph G, Jansen T, Lucas C, Poloni S, Beume N (eds) Parallel problem solving from nature – PPSN X, volume 5199 of Lecture notes in computer science. Springer Berlin Heidelberg, pp 661–670Google Scholar
  10. Eskandari H, Geiger CD (2008) A fast Pareto genetic algorithm approach for solving expensive multiobjective optimization problems. J Heuristics 14:203–241CrossRefMATHGoogle Scholar
  11. Eskelinen P, Miettinen K, Klamroth K, Hakanen J (2010) Pareto navigator for interactive nonlinear multiobjective optimization. OR Spectrum 32(1):211–227MathSciNetCrossRefMATHGoogle Scholar
  12. Friedman JH (1991) Multivariate adaptive regression splines. Ann Stat 19(1):1–67CrossRefMATHGoogle Scholar
  13. Forrester AIJ, Sóbester A, Keane AJ (2008) Engineering design via surrogate modelling: a practical guide. Wiley, ChichesterCrossRefGoogle Scholar
  14. Forrester AIJ, Keane AJ (2009) Recent advances in surrogate-based optimization. Prog Aerosp Sci 45:50–79CrossRefGoogle Scholar
  15. Gamito MN, Maddock SC (2009) Accurate multidimensional poisson-disk sampling. ACM Trans Graph 29 (1):1–19CrossRefGoogle Scholar
  16. Giunta A, Watson LT, Koehler J (1998) A comparison of approximation modeling techniques: polynomial versus interpolating models. In: Proceedings of 7th AIAA/USAF/NASA/ISSMO symposium on multidisciplinary analysis and optimization, vol 1. St. Louis, MO, pp 392–404. AIAA-98-4758Google Scholar
  17. Gobbi M, Guarneri P, Scala L, Scotti L (2013) A local approximation based multi-objective optimization algorithm with applications. Optim Eng:1–23Google Scholar
  18. Goel T, Vaidyanathan R, Haftka R, Shyy W, Queipo N, Tucker K (2004) Response surface approximation of Pareto optimal front in multi-objective optimization. In: Proceedings of the 10th AIAA/ISSMO multidisciplinary analysis and optimization conferences, Albany, NY, pp 2230–2245Google Scholar
  19. Goel T, Vaidyanathan R, Haftka RT, Shyy W, Queipo NV, Tucker K (2007) Response surface approximation of Pareto optimal front in multi-objective optimization. Comput Methods Appl Mech Eng 196:879–893CrossRefMATHGoogle Scholar
  20. Goldberg DE (1989) Genetic algorithms in search, optimization and machine learning. Addison-Wesley Longman Publishing Co., New YorkMATHGoogle Scholar
  21. Hagan MT, Demuth HB, Beale M (1996) Neural network design. PWS, BostonGoogle Scholar
  22. Hakanen J, Miettinen K, Sahlstedt K (2011) Wastewater treatment: new insight provided by interactive multiobjective optimization. Decis Support Syst 51(2):328–337CrossRefGoogle Scholar
  23. Hartikainen M, Miettinen K, Wiecek MM (2012) PAINT: Pareto front interpolation for nonlinear multiobjective optimization. Comput Optim Appl 52(3):845–867MathSciNetCrossRefMATHGoogle Scholar
  24. Helton JC, Johnson JD, Sallaberry CJ, Storlie CB (2006) Survey of sampling-based methods for uncertainty and sensitivity analysis. Reliab Eng Syst Saf 91(10-11):1175–1209CrossRefGoogle Scholar
  25. Hickernell FJ, Lemieux C, Owen AB (2005) Control variates for quasi-monte carlo. Stat Sci 20(1):1–31MathSciNetCrossRefMATHGoogle Scholar
  26. Jakobsson S, Patriksson M, Rudholm J, Wojciechowski A (2010) A method for simulation based optimization using radial basis functions. Optim Eng 11(4):501–532MathSciNetCrossRefMATHGoogle Scholar
  27. Jakobsson S, Saif-Ul-Hasnain M, Rundqvist R, Edelvik F, Andersson B, Patriksson M, Ljungqvist M, Lortet D, Wallesten J (2010) Combustion engine optimization: a multiobjective approach. Optim Eng 11 (4):533–554MathSciNetCrossRefMATHGoogle Scholar
  28. Jin R, Chen W, Simpson TW (2001) Comparative studies of metamodelling techniques under multiple modelling criteria. Struct Multidiscip Optim 23(1):1–13CrossRefGoogle Scholar
  29. Jin Y (2011) Surrogate-assisted evolutionary computation: recent advances and future challenges. Swarm Evol Comput 1(2):61–70CrossRefGoogle Scholar
  30. Johnson ME, Moore LM, Ylvisaker D (1990) Minimax and maximin distance designs. J Stat Plan Inf 26(2):131–148MathSciNetCrossRefGoogle Scholar
  31. Kleijnen JPC (2009) Kriging metamodeling in simulation: a review. Eur J Oper Res 192(3):707–716MathSciNetCrossRefMATHGoogle Scholar
  32. Kitayama S, Arakawa M, Yamazaki K (2011) Sequential approximate optimization using radial basis function network for engineering optimization. Optim Eng 12(4):535–557MathSciNetCrossRefMATHGoogle Scholar
  33. Kitayama S, Arakawa M, Yamazaki K (2011) Differential evolution as the global optimization technique and its application to structural optimization. Appl Soft Comput 11(4):3792–3803CrossRefGoogle Scholar
  34. Khokhar ZO, Vahabzadeh H, Ziai A, Wang GG, Menon C (2010) On the performance of the PSP method for mixed-variable multi-objective design optimization. J Mech Des 132(7):1–11CrossRefGoogle Scholar
  35. Kitayama S, Srirat J, Arakawa M, Yamazaki K (2013) Sequential approximate multi-objective optimization using radial basis function network. Struct Multidiscip Optim 48(3):501–515MathSciNetCrossRefGoogle Scholar
  36. Knowles J (2006) ParEGO: a hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems. IEEE Trans Evol Comput 10(1):50–66CrossRefGoogle Scholar
  37. Knowles J, Nakayama H (2008) Meta-modeling in multiobjective optimization. In: Blaszczynski J, Jin Y, Shimoyama K, Slowinski R (eds) Multiobjective optimization: interactive and evolutionary approaches. Springer-Verlag Berlin Heidelberg, pp 245–284Google Scholar
  38. Koziel S, Ciaurri DE, Leifsson L (2011) Surrogate-based methods. In: Koziel S, Yang X (eds) Studies in computational intelligence, vol 356. Springer Berlin Heidelberg, pp 33–59Google Scholar
  39. Kursawe F (1991) A variant of evolution strategies for vector optimization. In: Schwefel HP, Männer R (eds) Parallel Problem Solving from Nature, vol 496. Springer Berlin Heidelberg, pp 193–197Google Scholar
  40. Kutner MH, Nachtsheim CJ, Neter J, Li W (2005) Applied linear statistical models. McGraw Hill, BostonGoogle Scholar
  41. Li YF, Ng SH, Xie M, Goh TN (2010) A systematic comparison of metamodeling techniques for simulation optimization in decision support systems. Appl Soft Comput 10(4):1257–1273CrossRefGoogle Scholar
  42. Liao X, Li Q, Yang X, Zhang W, Li W (2008) Multiobjective optimization for crash safety design of vehicles using stepwise regression model. Struct Multidiscip Optim 35(6):561–569CrossRefGoogle Scholar
  43. Liu GP, Han X (2006) A micro multi-objective genetic algorithm for multi-objective optimizations. In: The 4th China–Japan–Korea joint symposium on optimization of structural and mechanical systems, Kunming, ChinaGoogle Scholar
  44. Liu GP, Han X, Jiang C (2008) A novel multi-objective optimization method based on an approximation model management technique. Comput Methods Appl Mech Eng 197:2719–2731CrossRefMATHGoogle Scholar
  45. Livermore Software Technology Corporation (LSTC) (2013). http://www.lstc.com/products/ls-dyna. Accessed Dec 2013
  46. Lotov AV, Bushenkov VA, Kamenev GK (2001) Feasible goals method. Search for smart decisions. Computing Center of RAS, MoscowGoogle Scholar
  47. Lotov AV, Bushenkov VA, Kamenev GK (2004) Interactive decision maps: approximation and visualization of Pareto frontier. Kluwer Academic Publishers, MassachusettsCrossRefGoogle Scholar
  48. Luque M, Ruiz F, Miettinen K (2011) Global formulation for interactive multiobjective optimization. OR Spectrum 33(1):27–48MathSciNetCrossRefMATHGoogle Scholar
  49. Madsen JI, Shyy W, Haftka RT (2000) Response surface techniques for diffuser shape optimization. AIAA J 3(9):1512–1518CrossRefGoogle Scholar
  50. Marler RT, Arora JS (2004) Survey of multi-objective optimization methods for engineering. Struct Multidiscip Optim 26(6):369–395MathSciNetCrossRefMATHGoogle Scholar
  51. Messac A, Ismail-Yahaya A, Mattson CA (2003) The normalized normal constraint method for generating the Pareto frontier. Struct Multidiscip Optim 25(2):86–98MathSciNetCrossRefMATHGoogle Scholar
  52. Messac A, Mattson CA (2004) Normal constraint method with guarantee of even representation of complete Pareto frontier. AIAA J 42(10):2101–2111CrossRefGoogle Scholar
  53. Messac A, Mullur AA (2008) A computationally efficient metamodeling approach for expensive multiobjective optimization. Optim Eng 9(1):37–67MathSciNetCrossRefGoogle Scholar
  54. Miettinen K (1999) Nonlinear multiobjective optimization. Kluwer Academic Publishers, NorwellMATHGoogle Scholar
  55. Miettinen K, Ruiz F, Wierzbicki AP (2008) Introduction to multiobjective optimization: interactive approaches. In: Branke J, Deb K, Miettinen K, Slowinski R (eds) Multiobjective optimization: interactive and evolutionary approaches. Springer-Verlag Berlin Heidelberg, pp 27–57Google Scholar
  56. Miettinen K (2014) Survey of methods to visualize alternatives in multiple criteria decision making problems. OR Spectrum 36(1): 3–37MathSciNetCrossRefMATHGoogle Scholar
  57. Moldex3d: Plastic injection molding simulation software (2013). http://www.moldex3d.com/en/. Accessed Dec 2013
  58. Monz M, Küfer KH, Bortfeld TR, Thieke C (2008) Pareto navigation—algorithmic foundation of interactive multi-criteria IMRT planning. Phys Med Biol 53(4):985–998CrossRefGoogle Scholar
  59. MSC Nastran-Multidisciplinary structural analysis (2013). http://www.mscsoftware.com/product/msc-nastran, Accessed Dec 2013
  60. Mullur AA, Messac A (2005) Extended radial basis functions: more flexible and effective metamodeling. AIAA J 43(6):1306–1315CrossRefGoogle Scholar
  61. Nakayama H, Yun Y, Yoon M (2009) Sequential approximate multiobjective optimization using computational intelligence, 1st edn. Springer Publishing Company, Incorporated, BerlinMATHGoogle Scholar
  62. Nebro AJ, Luna F, Alba E, Dorronsoro B, Durillo JJ, Beham A (2008) AbYSS: adapting scatter search to multiobjective optimization. IEEE Trans Evol Comput 12:439–457CrossRefGoogle Scholar
  63. Niederreiter H (1992) Random number generation and quasi-Monte Carlo methods, PhiladelphiaGoogle Scholar
  64. Okabe T, Jin Y, Sendhoff B (2003) A critical survey of performance indices for multi-objective optimisation. In: The 2003 congress on Evolutionary computation, 2003. CEC ’03. vol 2, pp 878–885Google Scholar
  65. Pascoletti A, Serafini P (1984) Scalarizing vector optimization problems. J Optim Theory Appl 42(4):499–524MathSciNetCrossRefMATHGoogle Scholar
  66. Queipo NV, Haftka RT, Shyy W, Goel T, Vaidyanathan R, Tucker KP (2005) Surrogate-based analysis and optimization. Prog Aerosp Sci 41(1):1–28CrossRefGoogle Scholar
  67. Reyes M, Coello CA (2005) Improving PSO-based multi-objective optimization using crowding, mutation and epsilon-dominance. In: Coello Coello C, Aguirre AH, Zitzler E (eds) Proceedings of 3rd international conference on evolutionary multi-criterion optimization, number 505–519 in EMO’05, Guanajuato, Mexico. Springer, BerlinGoogle Scholar
  68. Rezaveisi M, Sepehrnoori K, Johns RT (2014) Tie-simplex-based phase-behavior modeling in an IMPEC reservoir simulator. Soc Pet Eng 19(02):327–339Google Scholar
  69. Ruiz F, Luque M, Miettinen K (2012) Improving the computational efficiency in a global formulation (GLIDE) for interactive multiobjective optimization. OR Spectrum 197(1):47–70MathSciNetGoogle Scholar
  70. Schaumann E, Balling R, Day K (1998) Genetic algorithms with multiple objectives. In: Proceedings of the 7th AIAA/USAF/ NASA/ISSMO symposium on multidisciplinary analysis and optimization, vol 3. Washington, DC, pp 2114–2123Google Scholar
  71. Shan S, Wang GG (2004) An efficient Pareto set identification approach for multiobjective optimization on black-box functions. J Mech Des 127(5):866–874MathSciNetCrossRefGoogle Scholar
  72. Shan S, Wang GG (2010) Survey of modeling and optimization strategies to solve high-dimensional design problems with computationally-expensive black-box functions. Struct Multidiscip Optim 41(2):219–241MathSciNetCrossRefMATHGoogle Scholar
  73. Simpson TW, Poplinski JD, Koch PN, Allen JK (2001) Metamodels for computer-based engineering design: survey and recommendations. Engineering with Computers 17(2):129–150CrossRefMATHGoogle Scholar
  74. Simpson TW, Booker AJ, Ghosh D, Giunta AA, Koch PN, Yang RJ (2004) Approximation methods in multidisciplinary analysis and optimization: a panel discussion. Struct Multidiscip Optim 27(5):302–313CrossRefGoogle Scholar
  75. Simpson TW, Toropov V, Balabanov V, Viana F (2008) Design and analysis of computer experiments in multidisciplinary design optimization: a review of how far we have come - or not. In: Proceedings of the 12th AIAA/ISSMO multidisciplinary analysis and optimization conference, pages AIAA–2008–5802, British ColombiaGoogle Scholar
  76. Smola AJ, Schökopf B (2004) A tutorial on support vector regression. Stat Comput 14(3):199–222MathSciNetCrossRefGoogle Scholar
  77. Star-CD (2013). http://www.cd-adapco.com/products/star-cd. Accessed Dec 2013
  78. Steinhaus H (2011) Mathematical snapshots. Courier Dover Publications, New YorkGoogle Scholar
  79. Steuer RE (1986) Multiple criteria optimization: theory, computation and application. Wiley, New YorkMATHGoogle Scholar
  80. Su R, Gui L, Fan Z (2011) Multi-objective optimization for bus body with strength and rollover safety constraints based on surrogate models. Struct Multidiscip Optim 44(3):431–441MathSciNetCrossRefMATHGoogle Scholar
  81. Su R, Wang X, Gui L, Fan Z (2011) Multi-objective topology and sizing optimization of truss structures based on adaptive multi-island search strategy. Struct Multidiscip Optim 43(2):275– 286CrossRefGoogle Scholar
  82. Tenne Yl, Goh CK (2010) Computational intelligence in expensive optimization problems, 1st edition. Springer Publishing Company, BerlinMATHGoogle Scholar
  83. Wang L, Shan S, Wang GG (2004) Mode-pursuing sampling method for global optimization on expensive black-box functions. J Eng Optim 36(4):419–438CrossRefGoogle Scholar
  84. Wang GG, Shan S (2006) Review of metamodeling techniques in support of engineering design optimization. J Mech Des 129(4): 370–380CrossRefGoogle Scholar
  85. Wierzbicki AP (1986) On the completeness and constructiveness of parametric characterizations to vector optimization problems. OR Spectrum 8(2):73–87MathSciNetCrossRefMATHGoogle Scholar
  86. Wilson B, Cappelleri D, Simpson TW, Frecker M (2001) Efficient Pareto frontier exploration using surrogate approximations. Optim Eng 2(1):31–50MathSciNetCrossRefGoogle Scholar
  87. Wu J, Azarm S (2000) Metrics for quality assessment of a multiobjective design optimization solution set. J Mech Des 123(1):18–25CrossRefGoogle Scholar
  88. Wagner T, Emmerich M, Deutz A, Ponweiser W (2010). In: Schaefer R, Cotta C, Kolodziej J, Rudolph G (eds) On expected-improvement criteria for model-based multi-objective optimization. Springer, BerlinGoogle Scholar
  89. Yang BS, Yeun YS, Ruy WS (2002) Managing approximation models in multiobjective optimization. Struct Multidiscip Optim 24(2):141–156CrossRefGoogle Scholar
  90. Yun Y, Yoon M, Nakayama H (2009) Multi-objective optimization based on meta-modeling by using support vector regression. Optim Eng 10(2):167–181MathSciNetCrossRefMATHGoogle Scholar
  91. Zanakis SH, Solomon A, Wishart N, Dublish S (1998) Multi-attribute decision making: a simulation comparison of select methods. Eur J Oper Res 107(3):507–529CrossRefMATHGoogle Scholar
  92. Zitzler E, Laumanns M, Thiele L (2001) SPEA2: Improving the strength Pareto evolutionary algorithm. Technical Report 103, Computer Engineering and Networks Laboratory (TIK), Department of Electrical Engineering, Swiss Federal Institute of Technology (ETH)Google Scholar
  93. Zitzler E, Laumanns M, Thiele L (2002) SPEA2: Improving the strength Pareto evolutionary algorithm for multiobjective optimization. In: Proceedings of the conference on evolutionary methods for design optimization and control, CIMNE, Barcelona, Spain, pp 95–100Google Scholar
  94. Zitzler E, Thiele L, Laumanns M, Fonseca CM, da Fonseca VG (2003) Performance assessment of multiobjective optimizers: an analysis and review. IEEE Trans Evol Comput 7(2):117–132CrossRefMATHGoogle Scholar
  95. Zitzler E, Knowles J, Thiele L (2008) Quality assessment of pareto set approximations, vol 5252. Springer, Berlin, pp 373– 404Google Scholar
  96. Zhou J, Turng LS (2007) Adaptive multiobjective optimization of process conditions for injection molding using a Gaussian process approach. Adv Polym Technol 26(2):71–85CrossRefGoogle Scholar
  97. Zhou A, Qu BY, Li H, Zhao SZ, Suganthan PN, Zhang Q (2011) Multiobjective evolutionary algorithms: a survey of the state of the art. Swarm Evol Comput 1(1):32–49CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  • Mohammad Tabatabaei
    • 1
  • Jussi Hakanen
    • 1
  • Markus Hartikainen
    • 1
  • Kaisa Miettinen
    • 1
  • Karthik Sindhya
    • 1
  1. 1.University of JyvaskylaDepartment of Mathematical Information TechnologyUniversity of JyvaskylaFinland

Personalised recommendations