Skip to main content
SpringerLink
Log in

Cross-validation based single response adaptive design of experiments for Kriging metamodeling of deterministic computer simulations

  • Research Paper
  • Published:
Structural and Multidisciplinary Optimization Aims and scope Submit manuscript

Abstract

A new approach for single response adaptive design of deterministic computer experiments is presented. The approach is called SFCVT, for Space-Filling Cross-Validation Tradeoff. SFCVT uses metamodeling to obtain an estimate of cross-validation errors, which are maximized subject to a constraint on space filling to determine sample points in the design space. The proposed method is compared, using a test suite of forty four numerical examples, with three DOE methods from the literature. The numerical test examples can be classified into symmetric and asymmetric functions. Symmetric examples refer to functions for which the extreme points are located symmetrically in the design space and asymmetric examples are those for which the extreme regions are not located in a symmetric fashion in the design space. Based upon the comparison results for the numerical examples, it is shown that SFCVT performs better than an existing adaptive and a non-adaptive DOE method for asymmetric multimodal functions with high nonlinearity near the boundary, and is comparable for symmetric multimodal functions and other test problems. The proposed approach is integrated with a multi-scale heat exchanger optimization tool to reduce the computational effort involved in the design of novel air-to-water heat exchangers. The resulting designs are shown to be significantly more compact than mainstream heat exchanger designs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Notes

  1. \(\begin{array}{rll}\textrm {RMSE}&=&\left ( {\frac {1}{n}\displaystyle \sum\limits_{i=1}^n {\left [ {\hat {y}\left ( {x_i } \right )-y\left ( {x_i } \right )} \right ]^{2}} } \right )^{1/2} \\ \textrm {MAE}&=&\max \left | {\hat {y}\left ( {x_i } \right )-y\left ( {x_i } \right )} \right | \end{array}\)

References

  • Abdelaziz O (2009) Development of multi-scale, multi-physics, analysis capability and its application to novel heat exchanger design and optimization. Doctor of Philosophy dissertation, Department of Mechanical Engineering. University of Maryland

  • Abdelaziz O, Aute V, Radermacher R, Azarm S (2010) Approximation assisted optimization novel compact heat exchanger designs. HVAC&R Res 16(5):707–728

    Article  Google Scholar 

  • Armstrong M (1998) Basic linear geostatistics. Springer Verlag, Berlin

    Book  MATH  Google Scholar 

  • Bakker TMD (2000) Design optimization with Kriging models WBBM report series 47. Delft University Press, Netherlands

    Google Scholar 

  • Busby D, Farmer CL, Iske A (2007) Hierarchical nonlinear approximation for experimental design and statistical data fitting. SIAM J Sci Comput 29(1):49–69

    Article  MathSciNet  MATH  Google Scholar 

  • Congdon C, Carlsen D, Simpson TW, Martin JD (2008) Improving the performance of visual steering commands for multi-dimensional trade space exploration. In: Proceedings of the ASME international DETC and CIE conference, New York, pp 3–6

  • Cox DD, John S (1997) SDO: a sequential method for global optimization. In: Alexandrov N, Hussaini MY (eds) Multidisciplinary design optimization: state of the art. SIAM, Philadelphia, pp 315–329

    Google Scholar 

  • Cressie NAC (1993) Statistics for spatial data. Wiley series in probability and mathematical statistics. Wiley, New York

    Google Scholar 

  • Deb K (2001) Multi-objective optimization using evolutionary algorithms. Wiley, New York

    MATH  Google Scholar 

  • Fang KT, Lin DKJ, Winkler P, Zhang Y (2000) Uniform design: theory and application. Technometrics 39(3):237–248

    Article  Google Scholar 

  • Farhang-Mehr A, Azarm S (2005) Bayesian meta-modeling of engineering design simulations: a sequential approach with adaptation to irregularities in the response behavior. Int J Numer Methods Eng 62:2104–2126

    Article  MATH  Google Scholar 

  • Giunta AA, Wojtkiewicz SF, Eldred MS (2003) Overview of modern design of experiments methods for computational simulations. 41st Aerospace Sciences Meeting and Exhibit, AIAA 2003–649: 1–17, 6–9 January, Reno, Nevada

  • Goel T, Haftka RT, Queipo NV, Shyy W (2006) Performance estimate and simultaneous application of multiple surrogates. In: The proceedings of the 11th AIAA/ISSMO multidisciplinary analysis and optimization conference. Portsmouth, VA

  • Hamad H (2006) A new metric for measuring metamodels quality-of-fit for deterministic simulations. In: Proceedings of the winter simulation conference, vol 3, issue no 6, pp 882–888

  • Hedar AR (2005) Test problems for unconstrained and constrained global optimization, global optimization methods and codes. System Optimization Lab, Kyoto University

    Google Scholar 

  • Jiang H, Aute V, Radermacher R (2006) CoilDesigner: a general purpose simulation and design tool for air to refrigerant heat exchangers. Int J Refrig 29(4):601–610

    Article  Google Scholar 

  • Jin R, Chen W, Simpson TW (2001) Comparative studies of metamodeling techniques under multiple modeling criteria. Struct Multidiscip Optim 23(1):1–13

    Article  Google Scholar 

  • Jin R, Chen W, Sudjianto A (2002) On sequential sampling for global metamodeling in engineering design. In: CD-ROM proceedings of ASME IDETC, design automation conference, Montreal, Canada

  • Johnson ME, Moore LM, Ylvisaker D (1990) Minimax and maximin distance designs. J Stat Plan Infer 26(2):131–148

    Article  MathSciNet  Google Scholar 

  • Jones DR (2001) A taxonomy of global optimization methods based on response surfaces. J Glob Optim 21:345–383

    Article  MATH  Google Scholar 

  • Jones DR, Schonlau M, Welch WJ (1998) Efficient global optimization of expensive black Box functions. J Glob Optim 13:455–492

    Article  MathSciNet  MATH  Google Scholar 

  • Joseph VR, Hung Y, Sudjianto A (2008) Blind Kriging: a new method for developing metamodels. J Mech Des 130:031102-1–031102-8

    Article  Google Scholar 

  • Kalagnanam JR, Diwekar UM (1997) An efficient sampling technique for off-line quality control. Technometrics 39(3):308–319

    Article  MATH  Google Scholar 

  • Kleijnen JPC, van Beers WCM (2004) Application-driven sequential designs for simulation experiments: Kriging metamodeling. J Oper Res Soc 55(8):876–883

    Article  MATH  Google Scholar 

  • Knowles J (2005) ParEGO: a hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems. IEEE Trans Evol Comput 10(1):50–66

    Article  Google Scholar 

  • Koehler JR, Owen AB (1996) Computer experiments. Handbook of statistics, vol 13. Elsevier Science, New York

    Google Scholar 

  • Kuczera R, Mourelatos Z (2009) On estimating the reliability of multiple failure region problems using approximate metamodels. J Mech Des 131(12):121003-1–121003-11

    Article  Google Scholar 

  • Li G, Aute V, Azarm S (2010) An accumulative error based adaptive design of experiments for offline metamodeling. Struct Multidiscip Optim 40(1–6):137–155

    Article  Google Scholar 

  • Lin Y, Chen VCP, Tsui K-L, Mistree F, Allen JK (2004a) A sequential exploratory experimental design method: development of appropriate empirical models in design. In: Proceedings of the ASME design engineering technical conference, Salt Lake City, Utah. Paper No. DETC 2004–57527

  • Lin Y, Mistree F, Allen JK, Tsui K, Chen VCP (2004b) Sequential metamodeling in engineering design. In: Proceedings of the 10th AIAA/ISSMO multidisciplinary analysis and optimization conference, Albany, NY

  • Lophaven SN, Nielsen HB, Soøndergaard J (2002) Aspects of the Matlab Toolbox DACE. IMM-TR2002-13, Technical University of Denmark, DK

  • MATLAB® (2007) The language of technical computing. User manual, The MathWorks Inc

  • McKay MD, Beckman RJ, Conover WJ (1979) A comparison of three methods for selecting values of input variables in the analysis of response from a computer code. Technometrics 21(2):239–245

    MathSciNet  MATH  Google Scholar 

  • Meckesheimer M, Booker AJ, Barton RR, Simpson TW (2002) Computationally inexpensive metamodel assessment strategies. AIAA J 40(10):2053–2060

    Article  Google Scholar 

  • Myers RH, Montgomery DC (2002) Response surface methodology: process and product optimization using designed experiments. Wiley, New York

    Google Scholar 

  • Pettersson M, Olvander J (2007) Adaptive complex method for efficient design optimization. In: ASME 2007 international design engineering technical conferences and computers and information in engineering conference (IDETC/CIE2007), Las Vegas, Nevada, USA, 4–7 September 2007

  • Picheny V, Ginsbourger D, Roustant O, Haftka R, Kim N-H (2010) Adaptive design of experiments for accurate approximation of a target region. J Mech Des 132(7):071008-1–071008-9

    Article  Google Scholar 

  • Rohsenow WM, Choi HY (1961) Heat, mass and momentum transfer. Prentice Hall, New York

    Google Scholar 

  • Romero DA, Amon CH, Finger S (2006) On adaptive sampling for single and multi-response bayesian surrogate models. In: Proceedings of the ASME international design engineering technical conference, Philadelphia

  • Sacks J, Welch WJ, Mitchell TJ, Wynn HP (1989) Design and analysis of computer experiments. Stat Sci 4(4):409–435

    Article  MathSciNet  MATH  Google Scholar 

  • Sasena MJ (2002) Adaptive experimental design applied to an ergonomics testing procedure. In: Proceedings of the ASME international design engineering technical conference, Montreal, Canada

  • Sasena MJ, Papalambros PY, Goovaerts P (2000) Metamodeling sampling criteria in global optimization framework. In: Proceedings of the 8th AIAA/USAF/ISSMO symposium on multidisciplinary analysis and optimization, Long Beach CA

  • Shah RK, London AL (1978) Laminar flow forced convection in ducts: a source book for compact heat exchanger analytical data. Academic Press, New York

    Google Scholar 

  • Shao T, Krishnamurty S (2008) A clustering-based surrogate model updating approach to simulation-based engineering design. J Mech Des 130:041101-1–041101-13

    Article  Google Scholar 

  • Shewry MC, Wynn HP (1987) Maximum entropy sampling. J Appl Stat 14:165–170

    Article  Google Scholar 

  • Taguchi G (1987) System of experimental design, edited by Don Clausing. American Supplier Institute, Dearborn

    Google Scholar 

  • Van Veldhuizen DA, Lamont GB (1998) Multiobjective evolutionary algorithms research: a history and analysis. Technical report TR-98–03, Air Force Institute of Technology, Wright-Patterson AFB

  • Wang CC, Chi K, Chang C (2000) Heat transfer and friction characteristics of plain fin and tube heat exchangers, part II: correlation. Int J Heat Mass Transf 43:2693–2700

    Article  Google Scholar 

  • Wang GG, Shan S (2007) Review of metamodeling techniques in support of engineering design optimization. J Mech Des 129(4):369–463

    Article  MathSciNet  Google Scholar 

  • Watson AG, Barnes RJ (1995) Infill sampling criteria to locate extremes. Math Geol 27(5):589–608

    Article  Google Scholar 

Download references

Acknowledgments

The work presented in this paper was supported in part through a contract from the Office of Naval Research, contract number N000140710468. Such support does not constitute an endorsement by the funding agency of the opinions expressed in the paper.

Author information

Authors and Affiliations

  1. Mechanical Engineering Department, University of Maryland, College Park, MD, 20742, USA

    V. Aute, K. Saleh, S. Azarm & R. Radermacher

  2. Oak Ridge National Lab, Oak Ridge, TN, 37831, USA

    O. Abdelaziz

Authors
  1. V. Aute
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. K. Saleh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. O. Abdelaziz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. S. Azarm
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. R. Radermacher
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to V. Aute.

Additional information

A previous version of this paper was presented at: 12th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, Victoria, British Columbia, Canada, Sep 10–12, 2008; Aute, V., Abdelaziz, O., Azarm, S., Radermacher, R. 2008, “Cross-validation Based Single Response Adaptive Design of Experiments”

Appendix

Appendix

In order to evaluate the general applicability of SFCVT, a suite of test problems was compiled. A list of the test problems used in this paper is given in Table 5.

Table 5 Test problems
Full size table

Rights and permissions

Reprints and permissions

About this article

Cite this article

Aute, V., Saleh, K., Abdelaziz, O. et al. Cross-validation based single response adaptive design of experiments for Kriging metamodeling of deterministic computer simulations. Struct Multidisc Optim 48, 581–605 (2013). https://doi.org/10.1007/s00158-013-0918-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00158-013-0918-5

Keywords

Search

Navigation