Skip to main content
SpringerLink
Log in

Second-order nearly orthogonal Latin hypercubes for exploring stochastic simulations

  • Original Article
  • Published:
Journal of Simulation

Abstract

This paper presents new Latin hypercube designs with minimal correlations between all main, quadratic, and two-way interaction effects for a full second-order model. These new designs facilitate exploratory analysis of stochastic simulation models in which there is considerable a priori uncertainty about the forms of the responses. We focus on understanding the underlying complexities of simulated systems by exploring the input variables’ effects on the behavior of simulation responses. These new designs allow us to determine the driving factors, detect interactions between input variables, identify points of diminishing or increasing rates of return, and find thresholds or change points in localized areas. Our proposed designs enable analysts to fit many diverse metamodels to multiple outputs with a single set of runs. Creating these designs is computationally intensive; therefore, several have been cataloged and made available online to experimenters.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5

Similar content being viewed by others

References

  • Ankenman B, Nelson B and Staum J (2010). Stochastic kriging for simulation metamodeling. Operations Research 58(2): 371–382.

    Article  Google Scholar 

  • Atkinson AC, Donev AN and Tobias RD (2007). Optimum Experimental Designs, with SAS. Oxford: Oxford University Press.

    Google Scholar 

  • Bankes S (1993). Exploratory modeling for policy analysis. Operations Research 41(3): 435–449.

    Article  Google Scholar 

  • Barton RR (1998). Simulation metamodels. In: Medeiros DJ, Watson EF, Carson JS, and Manivannan, MS (eds). Proceedings of the 1998 Winter Simulation Conference, Vol. 1, IEEE. Piscataway, NJ, pp 167–174.

  • Beers V and Kleijnen JPC (2003). Kriging for interpolation in random simulation. Journal of the Operational Research Society 54(3): 255–262.

    Article  Google Scholar 

  • Beers V and Kleijnen JPC (2004). Kriging interpolation in simulation: A survey. In Proceedings of the 2004 Winter Simulation Conference. Washington, D.C., Vol. 1, pp 113–112.

  • Box GEP and Behnken DW (1960). Some new three level designs for the study of quantitative variables. Technometrics 2(4): 455–475.

    Article  Google Scholar 

  • Box GEP and Draper NR (1987). Empirical Model-Building and Response Surfaces. Wiley: New York, NY.

    Google Scholar 

  • Challenor P (2013). Experimental design for the validation of kriging metamodels in computer experiments. Journal of Simulation 7(4): 290–296.

    Article  Google Scholar 

  • Cioppa TM and Lucas TW (2007). Efficient nearly orthogonal and space-filling Latin hypercubes. Technometrics 49(1): 45–55.

    Article  Google Scholar 

  • Fang KT (1980). The uniform design: Application of number-theoretic methods in experimental design. Acta Mathematicale Applicatae Sinica 3: 363–372.

    Google Scholar 

  • Fang KT, Lin DKJ, Winker P and Zhang Y (2000). Uniform design: Theory and application. Technometrics 42(3): 237–248.

    Article  Google Scholar 

  • Florian A (1992). An efficient sampling scheme: Updated Latin hypercube sampling. Probabilistic Engineering Mechanics 7(2): 123–130.

    Article  Google Scholar 

  • Goldberg DE (1989). Genetic Algorithms in Search, Optimization, and Machine Learning. 1st edn, Addison-Wesley Longman Publishing Co: Boston, MA.

    Google Scholar 

  • Goldfarb HB, Borror CM, Montgomery DC and Anderson-Cook C (2005). Using genetic algorithms to generate mixture-process experimental designs involving control and noise variables. Journal of Quality Technology 37(1): 60–74.

    Google Scholar 

  • Heredia-Langner A, Carlyle WM, Montgomery DC and Borror CM (2004). Model-robust optimal designs: A genetic algorithm approach. Journal of Quality Technology 35(1): 263–279.

    Google Scholar 

  • Heredia-Langner A, Carlyle WM, Montgomery DC, Borror CM and Runger GC (2003). Genetic algorithms for the construction of D-optimal designs. Journal of Quality Technology 35(1): 28–46.

    Google Scholar 

  • Hernandez AS, Lucas TW and Carlyle WM (2012). Constructing nearly orthogonal Latin hypercubes for any nonsaturated run-variable combination. ACM Transactions on Modeling and Computer Simulation 22(20): 4.

    Google Scholar 

  • Hickernell FJ (1998). A generalized discrepancy and quadrature error bound. Mathematics of Computation 67(221): 299–322.

    Article  Google Scholar 

  • Holland JH (1975). Adaptation in Natural and Artificial Systems. The University of Michigan Press: Ann Arbor.

    Google Scholar 

  • Jin R, Chen W and Sudjianto A (2005). An efficient algorithm for constructing optimal design of computer experiments. Journal of Statistical Planning and Inference 134(1): 268–287.

    Article  Google Scholar 

  • Johnson ME, Moore LM and Ylvisaker D (1990). Minimax and maxmin distance design. Journal of Statistical Planning and Inference 26(2): 131–148.

    Article  Google Scholar 

  • Jones B and Nachtsheim CJ (2011). A class of three-level designs for definitive screening in the presence of second-order effects. Journal of Quality Technology 43(1): 1–15.

    Google Scholar 

  • Joseph VR and Hung Y (2008). Orthogonal-maximin Latin hypercube designs. Statistica Sinica 18(1): 171–186.

    Google Scholar 

  • Kim L and Loh H (2003). Classification trees and bivariate linear discriminant node models. Journal of Graphical and Statistics 12(3): 512–530.

    Article  Google Scholar 

  • Kleijnen JPC (2005). An overview of the design and analysis of simulation experiments for sensitivity analysis. European Journal of Operations Research 164(2): 287–300.

    Article  Google Scholar 

  • Kleijnen JPC (2008a). Simulation experiments in practice: Statistical design and regression analysis. Journal of Simulation 2(1): 19–27.

    Article  Google Scholar 

  • Kleijnen JPC (2008b). Design of experiments: Overview. In Proceedings of the 2008 Winter Simulation Conference. Miami, FL.

  • Kleijnen JPC, Sanchez SM, Lucas TW and Cioppa TM (2005). A user’s guide to the brave new world of designing simulation experiments. INFORMS Journal on Computing 17(3): 263–289.

    Article  Google Scholar 

  • Koehler JR and Owen AB (1996). Computer experiments. Handbook of Statistics 13: 261–308.

    Article  Google Scholar 

  • Law A (2007). Simulation Modeling and Analysis. 4th edn, McGraw-Hill: New York.

    Google Scholar 

  • Liefvendahl M and Stocki R (2006). A study on algorithms for optimization of Latin hypercubes. Journal of Statistical Planning and Inference 136(9): 3231–3247.

    Article  Google Scholar 

  • Lucas TW, Kelton WD, Sanchez SM and Sanchez PJ (2015). Changing the paradigm: Simulation, often the method of first resort. Naval Research Logistics 62: 293–303.

    Article  Google Scholar 

  • MacCalman AD (2013). Flexible space-filling designs for complex system simulations. Doctoral dissertation. Naval Postgraduate School: Monterey, CA.

  • McKay MD, Beckman RJ and Conover WJ (1979). A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21(2): 239–245.

    Google Scholar 

  • Michalewicz Z and Fogel DB (2010). How to Solve it: Modern Heuristics. Springer-Verlag Berlin: Heidelberg, NY.

    Google Scholar 

  • Moon H, Santner TJ and Dean A (2011). Algorithms for generating maximin Latin hypercube and orthogonal designs. Journal of Statistical Theory and Practice 5(1): 81–98.

    Article  Google Scholar 

  • Morris MD and Mitchell TJ (1995). Exploratory designs for computer experiments. Journal of Statistical Planning and Inference 43(3): 381–402.

    Article  Google Scholar 

  • Myers RH, Montgomery DC and Anderson-Cook CM (2009). Response Surface Methodology: Process and Product Optimization using Designed Experiments. 3rd edn, Wiley, Hoboken, NJ.

    Google Scholar 

  • National Research Council (2006). Defense Modeling, Simulation, and Analysis: Meeting the Challenge Committee on Modeling and Simulation for Defense Transformation. The National Academies Press: Washington, DC.

  • Owen AB (1994). Controlling correlations. In latin hypercube samples. Journal of the American Statistical Association: Theory and Methods 89(428): 1517–1522.

    Article  Google Scholar 

  • Pang FM, Liu Q and Lin DKJ (2009). A construction method for orthogonal Latin hypercube designs with prime power levels. Statist. Sinica 19(4): 1721–1728.

    Google Scholar 

  • Ryan TP (2007). Modern Experimental Design. 1st edn, Wiley-Interscience: Hoboken, NJ.

    Book  Google Scholar 

  • Sacks JW, Welch J, Mitchell TJ and Wynn HP (1989). Design and analysis of computer experiments (includes comments and rejoinder). Statistical Science 4(4): 409–435.

    Article  Google Scholar 

  • Sall J (2007). JMP Start Statistics: A Guide to Statistics and Data Analysis Using JMP, Fourth Edition. 4th edn, SAS Publishing, Cary, NC.

    Google Scholar 

  • Sanchez SM (2008). Better than a petaflop: The power of efficient experimental design. In Proceedings of the 40th Conference on Winter Simulation, pp 73–84.

  • Sanchez SM, Lucas TW, Sanchez PJ, Nannini CJ and Wan H (2012). Designs for large-scale simulation experiments, with application to defense and homeland security. In: Hinkelmann K (ed) The Design and Analysis of Computer Experiments. Volume 3: Special Designs and Applications. Wiley: Hoboken, NJ, pp 413–441.

    Chapter  Google Scholar 

  • Santner TJ, Williams BJ and Notz WI (2003). The Design and Analysis of Computer Experiments. Springer-Verlag, New York, NY.

    Book  Google Scholar 

  • Shewry MC and Wynn HP (1987). Maximum entropy sampling. Journal of Applied Statistics 14(7): 165–170.

    Article  Google Scholar 

  • Siggelkow N (2002). Misperceiving interactions among complements and substitutes: Organizational consequences. Management Science 48(7): 900–916.

    Article  Google Scholar 

  • Steinberg DM and Lin DKJ (2006). A construction method for orthogonal Latin hypercube designs. Biometrika 93(2): 279–288.

    Article  Google Scholar 

  • Sun FS, Liu MQ and Lin DKJ (2009). Construction of orthogonal Latin hypercube designs. Biometrika 96(4): 971–974.

    Article  Google Scholar 

  • Vieira JrH (2008). Optimizing stochastic functions using a genetic algorithm: An aeronautic military application. In: Siarry P and Michalewicz Z (eds) Advances in Metaheuristics for Hard Optimization. Springer-Verlag: Berlin, pp 353–363.

    Chapter  Google Scholar 

  • Vieira JrH, Sanchez SM, Kienitz KH and Belderrain MCN (2011). Generating and improving orthogonal designs by using mixed integer programming. European Journal of Operational Research 215(3): 629–638.

    Article  Google Scholar 

  • Vieira JrH, Sanchez SM, Kienitz KH and Belderrain MCN (2013). Efficient, nearly orthogonal-and-balanced, mixed designs: An effective way to conduct trade-off analysis via simulation. Journal of Simulation 7(4): 264–275.

    Article  Google Scholar 

  • Yang JF, Lin CD, Qian PZG and Lin DKJ (2013). Construction of sliced orthogonal Latin hypercube Designs. Statitsica Sinica 23(3): 1117–1130.

    Google Scholar 

  • Ye KQ (1998). Orthogonal column Latin hypercubes and their application in computer experiments. Journal of the American Statistical Association: Theory and Methods 93(444): 1430–1439.

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank our colleagues at the Naval Postgraduate School for their insightful comments. In addition, the authors are grateful to the referees, whose feedback improved the content and clarity of this paper. The research was made possible by a grant from the Office of Naval Research (N0001412WX20823).

Author information

Authors and Affiliations

  1. United States Military Academy, West Point, NY, USA

    A D MacCalman

  2. Technological Institute of Aeronautics, São José dos Campos, Brazil

    H Vieira

  3. Naval Postgraduate School, Monterey, USA

    T Lucas

Authors
  1. A D MacCalman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. H Vieira
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. T Lucas
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to A D MacCalman.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

MacCalman, A., Vieira, H. & Lucas, T. Second-order nearly orthogonal Latin hypercubes for exploring stochastic simulations. J Simulation 11, 137–150 (2017). https://doi.org/10.1057/jos.2016.8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1057/jos.2016.8

Keywords

Search

Navigation