Physical Experiments and Computer Experiments

  • Thomas J. Santner
  • Brian J. Williams
  • William I. Notz
Part of the Springer Series in Statistics book series (SSS)


Experiments have long been used to study the relationship between a set of inputs to a physical system and the resulting output. Termed physical experiments in this text, there is a growing trend to replace or supplement the physical system used in such an experiment with a deterministic simulator.


  1. Berk R, Bickel P, Campbell K, Fovell R, Keller-McNulty S, Kelly E, Linn R, Park B, Perelson A, Rouphail N, Sacks J, Schoenberg F (2002) Workshop on statistical approaches for the evaluation of complex computer models. Stat Sci 17(2):173–192MathSciNetCrossRefGoogle Scholar
  2. Bernardo MC, Buck RJ, Liu L, Nazaret WA, Sacks J, Welch WJ (1992) Integrated circuit design optimization using a sequential strategy. IEEE Trans Comput Aided Des 11:361–372CrossRefGoogle Scholar
  3. Birk DM (1997) An introduction to mathematical fire modeling. Technomic Publishing, LancasterGoogle Scholar
  4. Bliznyuk N, Ruppert D, Shoemaker C, Regis R, Wild S, Mugunthan P (2008) Bayesian calibration and uncertainty analysis for computationally expensive models using optimization and radial basis function approximation. J Comput Graph Stat 17:270–294MathSciNetCrossRefGoogle Scholar
  5. Booker AJ, Dennis JE, Frank PD, Serafini DB, Torczon V (1997) Optimization using surrogate objectives on a helicopter test example. Technical report SSGTECH-97-027, Boeing technical reportGoogle Scholar
  6. Butler NA (2001) Optimal and orthogonal Latin hypercube designs for computer experiments. Biometrika 88:847–857MathSciNetCrossRefGoogle Scholar
  7. Chang PB (1998) Robust design and analysis of femoral components for total hip arthroplasty. PhD thesis, Sibley School of Mechanical and Aerospace Engineering, Cornell University, Ithaca, NYGoogle Scholar
  8. Chang PB, Williams BJ, Bawa Bhalla KS, Belknap TW, Santner TJ, Notz WI, Bartel DL (2001) Robust design and analysis of total joint replacements: finite element model experiments with environmental variables. J Biomech Eng 123:239–246CrossRefGoogle Scholar
  9. Cooper LY (1980) Estimating safe available egress time from fires. Technical report 80-2172, National Bureau of Standards, WashingtonGoogle Scholar
  10. Cooper LY (1997) VENTCF2: an algorithm and associated FORTRAN 77 subroutine for calculating flow through a horizontal ceiling/floor vent in a zone-type compartmental fire model. Fire Safe J 28:253–287CrossRefGoogle Scholar
  11. Cooper LY, Stroup DW (1985) ASET—a computer program for calculating available safe egress time. Fire Safe J 9:29–45CrossRefGoogle Scholar
  12. Crick MJ, Hofer E, Jones JA, Haywood SM (1988) Uncertainty analysis of the food chain and atmospheric dispersion modules of MARC. Technical report NRPBR184, National Radiological Protection BoardGoogle Scholar
  13. Dandekar R, Kirkendall N (1993) Latin hypercube sampling for sensitivity and uncertainty analysis. In: American Statistical Association proceedings of the Section on Physical and Engineering Sciences. American Statistical Association, Alexandria, VA, pp 26–31Google Scholar
  14. Dixon LCW, Szego GP (1978) The global optimisation problem: an introduction. In: Dixon LCW, Szego GP (eds) Towards global optimisation, vol 2. North Holland, Amsterdam, pp 1–15Google Scholar
  15. Gramacy RB, Gray GA, Le Digabel S, Lee HKH, Ranjan P, Wells G, Wild SM (2016) Modeling an augmented Lagrangian for blackbox constrained optimization (with discussion). Technometrics 58(1):1–29MathSciNetCrossRefGoogle Scholar
  16. Handcock MS (1991) On cascading Latin hypercube designs and additive models for experiments. Commun Stat Theory Methods 20(2):417–439MathSciNetCrossRefGoogle Scholar
  17. Harper WV, Gupta SK (1983) Sensitivity/uncertainty analysis of a borehole scenario comparing Latin hypercube sampling and deterministic sensitivity approaches. Technical report BMI/ONWI-516, Office of Nuclear Waste Isolation, Battelle Memorial Institute, Columbus, OHGoogle Scholar
  18. Helton JC (1993) Uncertainty and sensitivity analysis techniques for use in performance assessment for radioactive waste disposal. Reliab Eng Syst Saf 42:327–367CrossRefGoogle Scholar
  19. Huber PJ (1981) Robust statistics. Wiley, New York, NYCrossRefGoogle Scholar
  20. Janssens ML (2000) Introduction to mathematical fire modeling, 2nd edn. Technomic Publishing Company, LancasterGoogle Scholar
  21. Jones DR, Schonlau M, Welch WJ (1998) Efficient global optimization of expensive black-box functions. J Glob Optim 13:455–492MathSciNetCrossRefGoogle Scholar
  22. Kennedy MC, O’Hagan A (2000) Predicting the output from a complex computer code when fast approximations are available. Biometrika 87:1–13MathSciNetCrossRefGoogle Scholar
  23. Kotzar GM, Davy DT, Berilla J, Goldberg VM (1995) Torsional loads in the early postoperative period following total hip replacement. J Orthop Res 13:945–955CrossRefGoogle Scholar
  24. Lehman J, Santner TJ, Notz WI (2004) Design of computer experiments to determine robust control variables. Stat Sinica 14:571–580zbMATHGoogle Scholar
  25. Lempert R, Williams BJ, Hendrickson J (2002) Using sensitivity analysis to support robust adaptive planning. Technical report, RANDGoogle Scholar
  26. Loh WL (1996) On Latin hypercube sampling. Ann Stat 24:2058–2080MathSciNetCrossRefGoogle Scholar
  27. Lynn RR (1997) Transport model for prediction of wildfire behavior. Technical report LA-13334-T, Los Alamos National LaboratoryGoogle Scholar
  28. McKay MD, Beckman RJ, Conover WJ (1979) A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21:239–245MathSciNetzbMATHGoogle Scholar
  29. Mitchell TJ, Morris MD, Ylvisaker D (1994) Asymptotically optimum experimental designs for prediction of deterministic functions given derivative information. J Stat Plann Inf 41:377–389MathSciNetCrossRefGoogle Scholar
  30. Mockus A (1998) Estimating dependencies from spatial averages. J Comput Graph Stat 7:501–513Google Scholar
  31. Mockus J, Eddy W, Mockus A, Mockus L, Reklaitis G (1997) Bayesian heuristic approach to discrete and global optimization: algorithms, visualization, software, and applications. Kluwer Academic, New York, NYCrossRefGoogle Scholar
  32. Montgomery GP, Truss LT (2001) Combining a statistical design of experiments with formability simulations to predict the formability of pockets in sheet metal parts. Technical report 2001-01-1130, Society of Automotive EngineersGoogle Scholar
  33. Morris MD, Mitchell TJ, Ylvisaker D (1993) Bayesian design and analysis of computer experiments: use of derivatives in surface prediction. Technometrics 35:243–255MathSciNetCrossRefGoogle Scholar
  34. O’Hagan A, Haylock RG (1997) Bayesian uncertainty analysis and radiological protection. In: Barnett V, Turkman KF (eds) Statistics for the environment, vol 3. Wiley, New York, NY, pp 109–128Google Scholar
  35. O’Hagan A, Kennedy MC, Oakley JE (1999) Uncertainty analysis and other inference tools for complex computer codes. In: Bernardo JM, Berger JO, Dawid AP, Smith AFM (eds) Bayesian statistics, vol 6. Oxford University Press, Oxford, pp 503–524Google Scholar
  36. Ong K, Santner T, Bartel D (2008) Robust design for acetabular cup stability accounting for patient and surgical variability. J Biomech Eng 130(031001):1–11Google Scholar
  37. Owen AB (1992a) A central limit theorem for Latin hypercube sampling. J R Stat Soc Ser B Methodol 54:541–551MathSciNetzbMATHGoogle Scholar
  38. Owen AB (1994) Controlling correlations in Latin hypercube samples. J Am Stat Assoc 89:1517–1522CrossRefGoogle Scholar
  39. Pebesma EJ, Heuvelink GBM (1999) Latin hypercube sampling of Gaussian random fields. Technometrics 41:303–312CrossRefGoogle Scholar
  40. Picheny V, Ginsbourger D, Richet Y, Caplin G (2013) Quantile-based optimization of noisy computer experiments with tunable precision. Technometrics 55(1):2–13MathSciNetCrossRefGoogle Scholar
  41. Preston DL, Tonks DL, Wallace DC (2003) Model of plastic deformation for extreme loading conditions. J Appl Phys 93:211–220CrossRefGoogle Scholar
  42. Qian PZ, Seepersad CC, Joseph VR, Allen JK, Wu CFJ (2006) Building surrogate models with details and approximate simulations. ASME J Mech Des 128:668–677CrossRefGoogle Scholar
  43. Qian PZG (2009) Nested Latin hypercube designs. Biometrika 96:957–970MathSciNetCrossRefGoogle Scholar
  44. Qian PZG (2012) Sliced Latin hypercube designs. J Am Stat Assoc 107:393–399MathSciNetCrossRefGoogle Scholar
  45. Qian PZG, Wu CFJ (2008) Bayesian hierarchical modeling for integrating low-accuracy and high-accuracy experiments. Technometrics 50(2):192–204MathSciNetCrossRefGoogle Scholar
  46. Regis RG, Shoemaker CA (2005) Constrained global optimization of expensive black box functions using radial basis functions. J Glob Optim 31:153–171MathSciNetCrossRefGoogle Scholar
  47. Sahama AR, Diamond NT (2001) Sample size considerations and augmentation of computer experiments. J Stat Comput Simul 68(4):307–319MathSciNetCrossRefGoogle Scholar
  48. Sahama TR, Diamond NT (2008) Computer experiment – a case study for modelling and simulation of manufacturing systems. Asian Int J Sci Technol Prod Manuf 1(2):97–103Google Scholar
  49. Stein ML (1987) Large sample properties of simulations using Latin hypercube sampling. Technometrics 29:143–151MathSciNetCrossRefGoogle Scholar
  50. Tang B (1993) Orthogonal array-based Latin hypercubes. J Am Stat Assoc 88:1392–1397MathSciNetCrossRefGoogle Scholar
  51. Tang B (1994) A theorem for selecting OA-based Latin hypercubes using a distance criterion. Commun Stat Theory Methods 23:2047–2058MathSciNetCrossRefGoogle Scholar
  52. Trosset MW, Padula AD (2000) Designing and analyzing computational experiments for global optimization. Technical report 00-25, Department of Computational & Applied Mathematics, Rice UniversityGoogle Scholar
  53. Trunin RF (1998) Shock compression of condensed materials. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  54. Vazquez E, Bect J (2010) Pointwise consistency of the kriging predictor with known mean and covariance functions. In: Giovagnoli A, Atkinson AC, Torsney B, May C (eds) mODa 9 - advances in model-oriented design and analysis. Physica-Verlag, Heidelberg, pp 221–228CrossRefGoogle Scholar
  55. Villarreal-Marroquín MG, Chen PH, Mulyana R, Santner TJ, Dean AM, Castro JM (2017) Multiobjective optimization of injection molding using a calibrated predictor based on physical and simulated data. Polym Eng Sci 57(3):248–257CrossRefGoogle Scholar
  56. Walton W (1985) ASET-B: a room fire program for personal computers. Technical Report 85-3144-1, National Bureau of Standards, WashingtonGoogle Scholar
  57. Worley BA (1987) Deterministic uncertainty analysis. Technical report ORNL-6428, National Technical Information Service, 5285 Port Royal Road, Springfield, VA 22161Google Scholar
  58. Ye KQ (1998) Orthogonal column Latin hypercubes and their application in computer experiments. J Am Stat Assoc 93:1430–1439MathSciNetCrossRefGoogle Scholar
  59. Ye KQ, Li W, Sudjianto A (2000) Algorithmic construction of optimal symmetric Latin hypercube designs. J Stat Plann Inf 90(1):145–159MathSciNetCrossRefGoogle Scholar
  60. Zhou H (ed) (2013) Computer modeling for injection molding: simulation, optimization, and control. Wiley, Hoboken, NJGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  • Thomas J. Santner
    • 1
  • Brian J. Williams
    • 2
  • William I. Notz
    • 1
  1. 1.Department of StatisticsThe Ohio State UniversityColumbusUSA
  2. 2.Statistical Sciences GroupLos Alamos National LaboratoryLos AlamosUSA

Personalised recommendations