Screening pp 308-327 | Cite as

Screening the Input Variables to a Computer Model Via Analysis of Variance and Visualization

  • Matthias Schonlau
  • William J. Welch


A nexperiment involving a complex computer model or code may have tens or even hundreds of input variables and, hence, the identification of the more important variables (screening) is often crucial. Methods are described for decomposing a complex input—output relationship into effects. Effects are more easily understood because each is due to only one or a small number of input variables. They can be assessed for importance either visually or via a functional analysis of variance. Effects are estimated from flexible approximations to the input—output relationships of the computer model. This allows complex nonlinear and interaction relationships to be identified. The methodology is demonstrated on a computer model of the relationship between environmental policy and the world economy.


Marginal Effect Computer Experiment Human Development Index Corrected Effect Output Relationship 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. In Second International Symposium on Information Theory, pages 267–281. Akademia Kiadó, Budapest.Google Scholar
  2. Aslett, R., Buck, R.J., Duvall, S.G., Sacks, J., and Welch, W.J. (1998). Circuit optimization via sequential computer experiments: Design of an output buffer. Journal of the Royal Statistical Society, C, 47, 31–48.zbMATHCrossRefGoogle Scholar
  3. Bernardo, M.C., Buck, R., Liu, L., Nazaret, W.A., Sacks, J., and Welch, W.J. (1992). Integrated circuit design optimization using a sequential strategy. IEEE Transactions on Computer-Aided Design, 11, 361–372.CrossRefGoogle Scholar
  4. Bettonvil, B. and Kleijnen, J.P.C. (1996). Searching for important factors in simulation models with many factors: Sequential bifurcation. European Journal of Operational Research, 96, 180–194.CrossRefGoogle Scholar
  5. Chapman, W.L., Welch, W.J., Bowman, K.P., Sacks, J., and Walsh, J.E. (1994). Arctic sea ice variability: Model sensitivities and a multidecadal simulation. Journal of Geophysical Research C, 99, 919–935.CrossRefGoogle Scholar
  6. Currin, C., Mitchell, T., Morris, M., and Ylvisaker, D. (1991). Bayesian prediction of deterministic functions, with applications to the design and analysis of computer experiments. Journal of the American Statistical Association, 86, 953–963.MathSciNetCrossRefGoogle Scholar
  7. Gough, W.A. and Welch, W.J. (1994). Parameter space exploration of an ocean general circulation model using an isopycnal mixing parameterization. Journal of Marine Research, 52, 773–796.CrossRefGoogle Scholar
  8. Gu, C. and Wahba, G. (1993). Smoothing spline ANOVA with componentwise Bayesian “confidence intervals”. Journal of Computational and Graphical Statistics, 2, 97–117.MathSciNetCrossRefGoogle Scholar
  9. Herbert, R.D. and Leeves, G.D. (1998). Troubles in Wonderland. Complexity International, 6. Scholar
  10. Hoeffding, W. (1948). A class of statistics with asymptotically normal distribution. The Annals of Mathematical Statistics, 19, 293–325.zbMATHMathSciNetGoogle Scholar
  11. Iman, R.L. and Conover, W.J. (1980). Small sample sensitivity analysis techniques for computer models, with an application to risk assessment. Communications in Statistics A—Theory and Methods, 9, 1749–1842.MathSciNetGoogle Scholar
  12. Jones, D.R., Schonlau, M., and Welch, W.J. (1998). Efficient global optimization of expensive black-box functions. Journal of Global Optimization, 13, 455–492.zbMATHMathSciNetCrossRefGoogle Scholar
  13. Koehler, J.R. and Owen, A.B. (1996). Computer experiments. In Handbook of Statistics, Volume 13. Editors: S. Ghosh and C.R. Rao. Elsevier, Amsterdam.Google Scholar
  14. Lempert, R.J., Popper, S.W., and Bankes, S.C. (2003). Shaping the Next One Hundred Years: New Methods for Quantitative, Long-term Policy Analysis. RAND, Santa Monica, CA. Scholar
  15. McKay, M.D., Conover, W.J., and Beckman, R.J. (1979). A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics, 21, 239–245.zbMATHMathSciNetCrossRefGoogle Scholar
  16. Morris, M.D. (1991). Factorial sampling plans for preliminary computer experiments. Technometrics, 33, 161–174.CrossRefGoogle Scholar
  17. Mrawira, D., Welch, W.J., Schonlau, M., and Haas, R. (1999). Sensitivity analysis of computer models: World Bank HDM-III model. Journal of Transportation Engineering, 125, 421–428.CrossRefGoogle Scholar
  18. R Development Core Team (2005). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna. http://www.R-project.orgGoogle Scholar
  19. Robert, C.P. and Casella, G. (2004). Monte Carlo Statistical Methods, second edition. Springer, New York.Google Scholar
  20. Sacks, J., Welch, W.J., Mitchell, T.J., and Wynn, H.P. (1989). Design and analysis of computer experiments (with discussion). Statistical Science, 4, 409–435.zbMATHMathSciNetGoogle Scholar
  21. Santner, T.J., Williams, B.J., and Notz, W.I. (2003). The Design and Analysis of Computer Experiments. Springer Verlag, New York.Google Scholar
  22. Schonlau, M. (1997). Computer experiments and global optmization. PhD thesis, University of Waterloo, Waterloo, Ontario.Google Scholar
  23. Welch, W.J., Buck, R.J., Sacks, J., Wynn, H.P., Mitchell, T.J., and Morris, M.D. (1992). Screening, predicting, and computer experiments. Technometrics, 34, 15–25.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, Inc. 2006

Authors and Affiliations

  • Matthias Schonlau
    • 1
  • William J. Welch
    • 2
  1. 1.RAND CorporationPittsburghUSA
  2. 2.Department of StatisticsUniversity of British ColumbiaVancouverCanada

Personalised recommendations