Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Numerical studies of space-filling designs: optimization of Latin Hypercube Samples and subprojection properties


Quantitative assessment of the uncertainties tainting the results of computer simulations is nowadays a major topic of interest in both industrial and scientific communities. One of the key issues in such studies is to get information about the output when the numerical simulations are expensive to run. This paper considers the problem of exploring the whole space of variations of the computer model input variables in the context of a large dimensional exploration space. Various properties of space-filling designs are justified: interpoint-distance, discrepancy, minimum spanning tree criteria. A specific class of design, the optimized Latin Hypercube Sample, is considered. Several optimization algorithms, coming from the literature, are studied in terms of convergence speed, robustness to subprojection and space-filling properties of the resulting design. Some recommendations for building such designs are given. Finally, another contribution of this paper is the deep analysis of the space-filling properties of the design 2D-subprojections.

This is a preview of subscription content, log in to check access.

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10
Figure 11
Figure 12
Figure 13
Figure 14
Figure 15
Figure 16


  1. 1.

    In the following, LHS may refer to Latin Hypercube Sampling as well.

  2. 2.

    Which includes boolean ones, that is properties like ‘having a Latin Hypercube structure’.

  3. 3.

    Modelling MEsocosm structure and functioning for representing LOtic DYnamic ecosystems

  4. 4.

    In the following, a ‘point’ corresponds to an ‘experiment’ (at least a subset of experimental conditions).

  5. 5.

    The range is the support of the distribution.

  6. 6.

    A permutation π of {1, …, N} is a bijective function from {1, …, N} to {1, …, N}.


  1. Auder B, de Crécy A, Iooss B and Marquès M (2012). Screening and metamodeling of computer experiments with functional outputs. Application to thermal-hydraulic computations. Reliability Engineering and System Safety 107: 122–131.

  2. Cannamela C, Garnier J and Iooss B (2008). Controlled stratification for quantile estimation. Annals of Applied Statistics 2 (4): 1554–1580.

  3. Cheng R and Davenport T (1989). The problem of dimensionality in stratified sampling. Management Science 35 (11): 1278–1296.

  4. Ciric C, Ciffroy P and Charles S (2012). Use of sensitivity analysis to discriminate non-influential and influential parameters within an aquatic ecosystem model. Ecological Modelling 246: 119–130.

  5. de Crécy A et al (2008). Uncertainty and sensitivity analysis of the LOFT L2-5 test: Results of the BEMUSE programme. Nuclear Engineering and Design 238 (12): 3561–3578.

  6. de Rocquigny E, Devictor N and Tarantola S (eds). (2008). Uncertainty in Industrial Practice. Wiley: Chichester, England.

  7. Dussert C, Rasigni G, Rasigni M and Palmari J (1986). Minimal spanning tree: A new approach for studying order and disorder. Physical Review B 34 (5): 3528–3531.

  8. Fang KT (2001). Wrap-around L2-discrepancy of random sampling, Latin hypercube and uniform designs. Journal of Complexity 17 (4): 608–624.

  9. Fang KT, Li R and Sudjianto A (2006). Design and Modeling for Computer Experiments. Chapman & Hall/CRC: Boca Raton, FL.

  10. Fang KT and Lin DK (2003). Uniform experimental designs and their applications in industry. Handbook of Statistics 22: 131–178.

  11. Franco J (2008). Planification d’expériences numériques en phase exploratoire pour la simulation des phénomènes complexes Thèse de l’Ecole Nationale Supérieure des Mines de Saint-Etienne, France.

  12. Franco J, Bay X, Corre B and Dupuy D (2008). Strauss processes: A new space-filling design for computer experiments. In: Proceedings of Joint Meeting of the Statistical Society of Canada and the Société Française de Statistique, Ottawa, Canada.

  13. Franco J, Vasseur O, Corre B and Sergent M (2009). Minimum spanning tree: A new approach to assess the quality of the design of computer experiments. Chemometrics and Intelligent Laboratory Systems 97 (2): 164–169.

  14. Gentle J (2003). Random Number generation and Monte Carlo methods. Springer: New York.

  15. Hickernell F (1998a). A generalized discrepancy and quadrature error bound. Mathematics of Computation 67 (4): 299–322.

  16. Hickernell F (1998b). Lattice rules: How well do they measure up? In: Hellekalek P and Larcher G (eds). Random and Quasi-random point sets. Springer-Verlag: Berlin/New-York, pp 106–166.

  17. Iooss B, Boussouf L, Feuillard V and Marrel A (2010). Numerical studies of the metamodel fitting and validation processes. International Journal of Advances in Systems and Measurements 3 (1&2): 11–21.

  18. Iooss B et al (2012). Some new insights in derivative-based global sensitivity measures. In: Proceedings of the PSAM11 ESREL 2012 Conference, Curran Association, Helsinki, Finland, pp 1094–1104.

  19. Jin R, Chen W and Sudjianto A (2005). An efficient algorithm for constructing optimal design of computer experiments. Journal of Statistical Planning and Inference 134 (1): 268–287.

  20. Johnson M, Moore L and Ylvisaker D (1990). Minimax and maximin distance design. Journal of Statistical Planning and Inference 26 (2): 131–148.

  21. Kleijnen J and Sargent R (2000). A methodology for fitting and validating metamodels in simulation. European Journal of Operational Research 120 (1): 14–29.

  22. Koehler J and Owen A (1996). Computer experiments. In: Ghosh S and Rao C (eds). Design and Analysis of Experiments, volume 13 of Handbook of Statistics. Elsevier.

  23. Kucherenko S, Rodriguez-Fernandez M, Pantelides C and Shah N (2009). Monte carlo evaluation of derivative-based global sensitivity measures. Reliability Engineering and System Safety 94 (7): 1135–1148.

  24. Lamboni M, Iooss B, Popelin AL and Gamboa F (2013). Derivative-based global sensitivity measures: General links with sobol’ indices and numerical tests. Mathematics and Computers in Simulation 87: 45–54.

  25. Lemieux C (2009). Monte Carlo and Quasi-Monte Carlo Sampling. Springer: New York.

  26. Loeppky J, Sacks J and Welch W (2009). Choosing the sample size of a computer experiment: A practical guide. Technometrics 51 (4): 366–376.

  27. Marrel A (2008). Mise en oeuvre et exploitation du métamodèle processus gaussien pour l’analyse de modèles numériques—Application à un code de transport hydrogéologique Thèse de l’INSA Toulouse, France.

  28. Marrel A, Iooss B, Laurent B and Roustant O (2009). Calculations of the Sobol indices for the Gaussian process metamodel. Reliability Engineering and System Safety 94 (3): 742–751.

  29. Marrel A, Iooss B, Van Dorpe F and Volkova E (2008). An efficient methodology for modeling complex computer codes with Gaussian processes. Computational Statistics and Data Analysis 52 (10): 4731–4744.

  30. Marrel A and Marquès M (2012). Sensitivity analysis of safety factor predictions for nuclear component behaviour under accidental conditions. In: Proceedings of the PSAM11 ESREL 2012 Conference, Curran Association, Helsinki, Finland, pp 1134–1143.

  31. McKay M, Beckman R and Conover W (1979). A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21 (2): 239–245.

  32. Morris M and Mitchell T (1995). Exploratory designs for computationnal experiments. Journal of Statistical Planning and Inference 43 (3): 381–402.

  33. Niederreiter H (1987). Low-discrepancy and low-dispersion sequences. Journal of Number Theory 30 (1): 51–70.

  34. Park JS (1994). Optimal Latin-hypercube designs for computer experiments. Journal of Statistical Planning and Inference 39 (1): 95–111.

  35. Pronzato L and Müller W (2012). Design of computer experiments: Space filling and beyond. Statistics and Computing 22 (3): 681–701.

  36. Pujol G (2009). Simplex-based screening designs for estimating metamodels. Reliability Engineering and System Safety 94 (7): 1156–1160.

  37. Sacks J, Welch W, Mitchell T and Wynn H (1989). Design and analysis of computer experiments. Statistical Science 4 (4): 409–435.

  38. Saltelli A, Ratto M, Tarantola S and Campolongo F (2006). Sensitivity analysis practices: Strategies for model-based inference. Reliability Engineering and System Safety 91 (10&11): 1109–1125.

  39. Santner T, Williams B and Notz W (2003). The Design and Analysis of Computer Experiments. Springer: New York.

  40. Schwery M and Wynn H (1987). Maximum entropy sampling. Journal of Applied Statistics 14 (2): 165–170.

  41. Simpson T, Lin D and Chen W (2001). Sampling strategies for computer experiments: Design and analysis. International Journal of Reliability and Applications 2 (3): 209–240.

  42. Van Dam E, Husslage B, den Hertog D and Melissen H (2007). Maximin Latin hypercube designs in two dimensions. Operations Research 55 (1): 158–169.

  43. Varet S, Lefebvre S, Durand G, Roblin A and Cohen S (2012). Effective discrepancy and numerical experiments. Computer Physics Communications 183 (12): 2535–2541.

  44. Viana F, Venter G and Balabanov V (2010). An algorithm for fast optimal Latin hypercube design of experiments. International Journal for Numerical Methods in Engineering 82 (2): 135–156.

Download references


Part of this work has been backed by French National Research Agency (ANR) through the COSINUS programme (project COSTA BRAVA noANR-09-COSI-015). All our numerical tests were performed within the R statistical software environment with the DiceDesign package. We thank Luc Pronzato for helpful discussions and Catalina Ciric for providing the prey-predator model example. Finally, we are grateful to both of the reviewers for their valuable comments and their help with the English.

Author information

Correspondence to B Iooss.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Damblin, G., Couplet, M. & Iooss, B. Numerical studies of space-filling designs: optimization of Latin Hypercube Samples and subprojection properties. J Simulation 7, 276–289 (2013). https://doi.org/10.1057/jos.2013.16

Download citation


  • discrepancy
  • optimal design
  • Latin Hypercube Sampling
  • computer experiment