Design and Analysis of Computational Experiments: Overview
This chapter presents an overview of the design and analysis of computational experiments with optimization algorithms. It covers classic designs and their corresponding (meta)models; namely, Resolution-III designs including fractional factorial two-level designs for first-order polynomial models, Resolution-IV and Resolution-V designs for two-factor interactions, and designs including central composite designs for second-degree polynomials. It also reviews factor screening in experiments with very many factors, focusing on the sequential bifurcation method. Furthermore, it reviews Kriging models and their designs. Finally, it discusses experiments aimed at the optimization of the parameters of a given optimization algorithm, allowing multiple random experimental outputs. This optimization may use either generalized response surface methodology or Kriging combined with mathematical programming; the discussion also covers Taguchian robust optimization.
KeywordsResponse Surface Methodology Robust Optimization Latin Hypercube Sampling Kriging Model Classic Design
Unable to display preview. Download preview PDF.
Comments by two anonymous referees led to a much improved version of the present chapter.
- Angün E, Kleijnen J (2009) An asymptotic test of optimality conditions in multiresponse simulation-based optimization, working paperGoogle Scholar
- Ankenman B, Nelson B, Staum J (2009) Stochastic kriging for simulation metamodeling. Operations Research (accepted)Google Scholar
- Bartz-Beielstein T (2006) Experimental research in evolutionary computation—The new experimentalism. Natural Computing Series, SpringerGoogle Scholar
- Bartz-Beielstein T, Preuss M (2010) The future of experimental research. In: Bartz-Beielstein T, Chiarandini M, Paquete L, Preuss M (eds) Empirical methods for the analysis of optimization algorithms, Springer, pp 17–46Google Scholar
- Cressie N (1993) Statistics for spatial data: revised edition. Wiley, New YorkGoogle Scholar
- Dellino G, Kleijnen J, Meloni C (2009) Robust optimization in simulation: Taguchi and response surface methodology. In: Rossini M, Hill R, Johansson B, Dunkin A, Ingalls R (eds) Proceedings of the 2009 Winter Simulation Conference, (accepted)Google Scholar
- Kleijnen J, Van Beers W (2009) Monotonicity-preserving bootstrapped Kriging metamodels for expensive simulations, working paperGoogle Scholar
- Kleijnen J, Bettonvil B, Persson F (2006a) Screening for the important factors in large discrete-event simulation: sequential bifurcation and its applications. In: Dean A, Lewis S (eds) Screening: Methods for experimentation in industry, drug discovery, and genetics, Springer, pp 287–307Google Scholar
- Law A (2007) Simulation modeling and analysis, 4th edn. McGraw-Hill, BostonGoogle Scholar
- Lophaven S, Nielsen H, Søndergaard J (2002) DACE—A Matlab Kriging Toolbox. Tech. Rep. IMM-REP-2002-12, Informatics and Mathematical Modelling, Technical University of Denmark, Copenhagen, DenmarkGoogle Scholar
- Montgomery D (2009) Design and analysis of experiments, 7th edn. Wiley, Hoboken, New JerseyGoogle Scholar
- Ridge E, Kudenko D (2007) Screening the parameters affecting heuristic performance. In: Lipson H (ed) GECCO, ACM, p 180, URL http://doi.acm. org/10.1145/1276958.1276994
- Ridge E, Kudenko D (2010) Sequential experiment designs for screening and tuning parameters of stochastic heuristics. In: Bartz-Beielstein T, Chiarandini M, Paquete L, Preuss M (eds) Empirical Methods for the Analysis of Optimization Algorithms, Springer, pp 265–287Google Scholar
- Taguchi G (1987) System of experimental designs. Krauss International, NYGoogle Scholar
- Xu J, Yang F, Wan H (2007) Controlled sequential bifurcation for software reliability study. In: Henderson S, Biller B, Hsieh MH, Shortle J, Tew J, Barton R (eds) Proceedings of the 2007 Winter Simulation Conference, pp 281–288Google Scholar
- Yin J, Ng S, Ng K (2008) Kriging model with modified nugget effect. In: Proceedings of the 2008 IEEE International Conference on Industrial Engineering and Engineering Management, pp 1714–1718Google Scholar