Structural and Multidisciplinary Optimization

, Volume 57, Issue 3, pp 1329–1343 | Cite as

Employing partial metamodels for optimization with scarce samples

  • Di Wu
  • Kambiz H. Hajikolaei
  • G. Gary Wang


To deal with high-dimensional, computationally expensive and black-box optimization (HEB) problems, a Partial Metamodel-based Optimization (PMO) method using Radial Basis Function-High Dimensional Model Representation (RBF-HDMR) along with a moving cut-center strategy is developed. To reduce the exponentially increasing cost of building an accurate metamodel for high dimensional problems, partial RBF-HDMR models of selected design variables are constructed at every iteration in the proposed strategy based on sensitivity analysis. After every iteration, the cut center of RBF-HDMR is moved to the most recent optimum point in order to pursue the optimum. Numerical tests show that the PMO method in general performs better than optimization with a complete RBF-HDMR for high-dimensional problems in terms of both effectiveness and efficiency. To improve the performance of the PMO method, a trust region based PMO (TR-PMO) is developed. When the allowed number of function calls is scarce, TR-PMO has advantages over compared metamodel-based optimization methods. The proposed method was then successfully applied to an airfoil design problem. The use of a partial metamodel for the purpose of optimization shows promises and may lead to development of other novel algorithms.


High dimension HDMR Metamodeling Sensitivity analysis Optimization 



Funding from Natural Science and Engineering Research Council (NSERC) of Canada (Grant No: RGPIN 04291-2014) is gratefully appreciated.


  1. Adorio E, Diliman U (2005)MVF–multivariate test functions library in c for unconstrained global optimization, Available at:
  2. Alexandrov NM, Dennis JE, Lewis RM, Torczon V (1998) A trust-region framework for managing the use of approximation models in optimization. Struct Optim 15(1):16–23CrossRefGoogle Scholar
  3. Allison JT, Kokkolaras M, Papalambros PY (2009) Optimal Partitioning and Coordination Decisions in Decomposition-Based Design Optimization. J Mech Des 131(8):81008CrossRefGoogle Scholar
  4. Bates RA, Buck RJ, Riccomagno E, Wynn HP (1996) Experimental Design and Observation for Large Systems. J R Stat Soc Ser B 58(1):77–94MathSciNetzbMATHGoogle Scholar
  5. Bloebaum C, Hajela P, Sobieszczanski-Sobieski J (1992) Non-Hierarchic System Decomposition in Structural Optimization. Eng Optim 19(3):171–186CrossRefGoogle Scholar
  6. Cai X, Qiu H, Gao L, Yang P, Shao X (2016) An enhanced RBF-HDMR integrated with an adaptive sampling method for approximating high dimensional problems in engineering design. Struct Multidiscip Optim 53(6):1209–1229CrossRefGoogle Scholar
  7. Cheng GH, Younis A, Haji Hajikolaei K, Gary Wang G (2015) Trust Region Based Mode Pursuing Sampling Method for Global Optimization of High Dimensional Design Problems. J Mech Des 137(2):21407CrossRefGoogle Scholar
  8. Cressie N (1988) Spatial prediction and ordinary kriging. Math Geol 20(4):405–421MathSciNetCrossRefzbMATHGoogle Scholar
  9. Duan X, Wang GG, Kang X, Niu Q, Naterer G, Peng Q (2009) Performance study of mode-pursuing sampling method. Eng Optim 41(1):1–21CrossRefGoogle Scholar
  10. Fang H, Horstemeyer MF (2006) Global response approximation with radial basis functions. Eng Optim 38(4):407–424MathSciNetCrossRefGoogle Scholar
  11. Friedman JH (1991) Multivariate Adaptive Regression Splines. Ann Stat 19(1):1–67MathSciNetCrossRefzbMATHGoogle Scholar
  12. Goldberg DE (1989) Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley Longman Publishing Co., Inc., BostonzbMATHGoogle Scholar
  13. Haji Hajikolaei K, Cheng GH, Wang GG (2015) Optimization on Metamodeling-Supported Iterative Decomposition. J Mech Des 138(2):21401CrossRefGoogle Scholar
  14. Hajikolaei KH, Wang GG (2013) High Dimensional Model Representation with Principal Component Analysis. J Mech Des 36(1):11003CrossRefGoogle Scholar
  15. Hajikolaei KH, Pirmoradi Z, Cheng GH, Wang GG (2014) Decomposition for Large Scale Global Optimization Based on Quantified Variable Correlations Uncovered by Metamodeling. Eng Optim 47(4):429–452CrossRefGoogle Scholar
  16. Hajikolaei KH, Cheng GH, Wang GG (2016) Optimization on Metamodeling- Supported Iterative Decomposition. ASME J Mech Des 138:21401CrossRefGoogle Scholar
  17. Huang Z, Qiu H, Zhao M, Cai X, Gao L (2015) An adaptive SVR-HDMR model for approximating high dimensional problems. Eng Comput Int J Comput Eng Softw 32(3):643–667Google Scholar
  18. Jones DR, Law C (1993) Lipschitzian Optimization Without the Lipschitz Constant. J Optim Theory Appl 79(1):157–181MathSciNetCrossRefzbMATHGoogle Scholar
  19. Jones DR, Schonlau M, Welch WJ (1998) Efficient Global Optimization of Expensive Black-Box Functions. J Glob Optim 13(4):455–492MathSciNetCrossRefzbMATHGoogle Scholar
  20. Kaya H, Kaplan M, Saygın H (2004) A recursive algorithm for finding HDMR terms for sensitivity analysis. Comput Phys Commun 158(2):106–112CrossRefzbMATHGoogle Scholar
  21. Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Proceedings of the IEEE international conference on neural networks, Perth, Western Australia, pp 1942–1948Google Scholar
  22. Kim HM, Michelena NF, Papalambros PY, Jiang T (2003) Target Cascading in Optimal System Design. J Mech Des 125(3):474CrossRefGoogle Scholar
  23. Kirkpatrick S, Gelatt CD, Vecchi MP (1983) Optimization by Simulated Annealing. Science 220(4598):671–680MathSciNetCrossRefzbMATHGoogle Scholar
  24. Kulfan B, Bussoletti J (2006) ‘Fundamental’ parameteric geometry representations for aircraft component shapes. 11th aiaa/issmo multidiscip. Anal. Optim. Conf., 1, pp 547–591Google Scholar
  25. Liu Y, Yao X, Zhao Q, Higuchi T (2001) Scaling Up Fast Evolutionary Programming with Cooperative Coevolution. In: Proceedings of the 2001 Congress on Evolutionary Computation, Seoul, pp 1101–1108Google Scholar
  26. Michelena N, Papalambros P, Park H, Kulkarni D (1999) Hierarchical Overlapping Coordination for Large-Scale Optimization by Decomposition. AIAA J 37(7):890–896CrossRefGoogle Scholar
  27. Papadrakakis M, Lagaros ND, Tsompanakis Y (1998) Structural optimization using evolution strategies and neural networks. Comput Methods Appl Mech Eng 156(1–4):309–333CrossRefzbMATHGoogle Scholar
  28. Potter M, De Jong K (1994) A Cooperative Coevolutionary Approach to Function Optimization. In: Proceedings of the Third Conference on Parallel Problem Solving from Nature, Springer-Verlag, pp 249–257Google Scholar
  29. Rabitz H, Alis Ö, Alış ÖF (1999) General foundations of high-dimensional model representations. J Math Chem 25(2–3):197–233MathSciNetCrossRefzbMATHGoogle Scholar
  30. Rassokhin DN, Lobanov VS, Drive S (2000) Nonlinear Mapping of Massive Data Sets by Fuzzy Clustering and Neural Networks. J Comput Chem 22(4):373–386CrossRefGoogle Scholar
  31. Schittkowski K (1987) More Test Examples for Nonlinear Programming Codes. Lecture Notes in Economics and Mathematical Systems. Springer-Verlag, BerlinGoogle Scholar
  32. Shan S, Wang GG (2009) Development of Adaptive RBF-HDMR Model for Approximating High Dimensional Problems. Volume 5: 35th Design Automation Conference, Parts A and B, ASME, pp 727–740Google Scholar
  33. Shan S, Wang GG (2010a) Survey of modeling and optimization strategies to solve high-dimensional design problems with computationally-expensive black-box functions. Struct Multidiscip Optim 41(2):219–241MathSciNetCrossRefzbMATHGoogle Scholar
  34. Shan S, Wang GG (2010b) Metamodeling for High Dimensional Simulation-Based Design Problems. J Mech Des 132:51009CrossRefGoogle Scholar
  35. Shi Y, Teng H, Li Z (2005) Cooperative Co-evolutionary Differential Evolution for Function Optimization. In: Proceedings of the First International Conference on Natural Computation, Springer-Verlag, pp 1080–1088Google Scholar
  36. Sobol′ I (1993) Sensitivity estimates for nonlinear mathematical models. Math Model Comput Exp 1(4):407–414MathSciNetzbMATHGoogle Scholar
  37. Somorjai RL, Dolenko B, Demko A, Mandelzweig M, Nikulin AE, Baumgartner R, Pizzi NJ (2004) Mapping high-dimensional data onto a relative distance plane--an exact method for visualizing and characterizing high-dimensional patterns. J Biomed Inform 37(5):366–379CrossRefGoogle Scholar
  38. Srivastava A, Hacker K, Lewis K, Simpson TW (2004) A method for using legacy data for metamodel-based design of large-scale systems. Struct Multidiscip Optim 28(2):146–155Google Scholar
  39. Wang GG, Shan S, (2004) Design Space Reduction for Multi-objective Optimization and Robust Design Optimization Problems. SAE Trans J Mater Manuf, pp 101–110.
  40. Wang L, Shan S, Wang GG (2004) Mode-pursuing sampling method for global optimization on expensive black-box functions. Eng Optim 36(4):419–438CrossRefGoogle Scholar
  41. Wang H, Tang L, Li GY (2011) Adaptive MLS-HDMR metamodeling techniques for high dimensional problems. Expert Syst Appl 38(11):14117–14126Google Scholar
  42. Winer EH, Bloebaum CL (2002) Development of visual design steering as an aid in large-scale multidisciplinary design optimization. Part I : method development. Struct Multidiscip Optim 23(6):412–424CrossRefGoogle Scholar
  43. XFOIL (2013) [Online]. Available at:
  44. Yang Z, Tang K, Yao X (2008) Large scale evolutionary optimization using cooperative coevolution. Inf Sci (Ny) 178(15):2985–2999MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany 2017

Authors and Affiliations

  1. 1.Product Design and Optimization Laboratory (PDOL)Simon Fraser UniversitySurreyCanada

Personalised recommendations