Skip to main content
SpringerLink
Log in

Optimal design of experiments for hypothesis testing on ordered treatments via intersection-union tests

  • Regular Article
  • Published:
Statistical Papers Aims and scope Submit manuscript

Abstract

We find experimental plans for hypothesis testing when a prior ordering of experimental groups or treatments is expected. Despite the practical interest of the topic, namely in dose finding, algorithms for systematically calculating good plans are still elusive. Here, we consider the intersection-union principle for constructing optimal experimental designs for testing hypotheses about ordered treatments. We propose an optimization-based formulation to handle the problem when the power of the test is to be maximized. This formulation yields a complex objective function which we handle with a surrogate-based optimizer. The algorithm proposed is demonstrated for several ordering relations. The relationship between designs maximizing power for the intersection-union test (IUT) and optimality criteria used for linear regression models is analyzed; we demonstrate that IUT-based designs are well approximated by C-optimal designs and maximum entropy sampling designs while D\(_{\text {A}}\)-optimal designs are equivalent to balanced designs. Theoretical and numerical results supporting these relations are presented.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  • Abelson RP, Tukey JW (1963) Efficient utilization of non-numerical information in quantitative analysis general theory and the case of simple order. Ann Math Stat 34(4):1347–1369

    MathSciNet  MATH  Google Scholar 

  • Alizadeh R, Allen JK, Mistree F (2020) Managing computational complexity using surrogate models: a critical review. Res Eng Design 31:275–298

    Google Scholar 

  • Anstreicher KM, Fampa M, Lee J, Williams J (2001) Maximum-entropy remote sampling. Discret Appl Math 108(3):211–226. https://doi.org/10.1016/S0166-218X(00)00217-1

    Article  MathSciNet  MATH  Google Scholar 

  • Antognini AB, Frieri R, Novelli M, Zagoraiou M (2021) Optimal designs for testing the efficacy of heterogeneous experimental groups. Electron J Stat 15(1):3217–3248. https://doi.org/10.1214/21-EJS1864

    Article  MathSciNet  MATH  Google Scholar 

  • Atkinson AC, Donev AN, Tobias RD (2007) Optimum experimental designs, with SAS. Oxford University Press, Oxford

    MATH  Google Scholar 

  • Bartholomew DJ (1959) A test of homogeneity for ordered alternatives. Biometrika 46:36–48

    MathSciNet  MATH  Google Scholar 

  • Bartholomew DJ (1959) A test of homogeneity for ordered alternatives. II. Biometrika 46:328–335

    MathSciNet  MATH  Google Scholar 

  • Bechhofer RE (1969) Optimal allocation of observations when comparing several treatments with a control. In: Krishnaiah PR (ed) Multivariate analysis. II. Academic Press, Boca Raton, pp 673–685

    Google Scholar 

  • Bechhofer RE, Nocturne DJM (1972) Optimal allocation of observations when comparing several treatments with a control, II: 2-sided comparisons. Technometrics 14(2):423–436

    MathSciNet  MATH  Google Scholar 

  • Bechhofer R, Turnbull B (1971) Optimal allocation of observations when comparing several treatments with a control (III): globally best one-sided intervals for unequal variances. In: Gupta SS, Yackel J (eds) Statistical decision theory and related topics. Academic Press, Boca Raton, pp 41–78

    Google Scholar 

  • Berger RL (1982) Multiparameter hypothesis testing and acceptance sampling. Technometrics 24(4):295–300

    MathSciNet  MATH  Google Scholar 

  • Berger RL, Hsu JC (1996) Bioequivalence trials, intersection-union tests and equivalence confidence sets. Stat Sci 11(4):283–302

    MathSciNet  MATH  Google Scholar 

  • Bhosekar A, Ierapetritou M (2018) Advances in surrogate based modeling, feasibility analysis, and optimization: A review. Comput Chem Eng 108:250–267

    Google Scholar 

  • Bretz F (1999) Powerful modifications of Williams’ test on trend. Ph.D. thesis, University of Hannover

  • Buhmann MD (2009) Radial basis functions—theory and implementations, vol 12. Cambridge University Press, Cambridge

    MATH  Google Scholar 

  • Cohen J (1988) Statistical power analysis for the behavioral sciences, 2nd edn. Lawrence Erlbaum Associates Publishers, New York

    MATH  Google Scholar 

  • Cover TM, Thomas JA (2006) Elements of information theory 2nd edition (Wiley series in telecommunications and signal processing). Wiley, Hoboken

    Google Scholar 

  • Currin C, Mitchell T, Morris M, Ylvisaker D (1991) Bayesian prediction of deterministic functions, with applications to the design and analysis of computer experiments. J Am Stat Assoc 86(416):953–963. https://doi.org/10.1080/01621459.1991.10475138

    Article  MathSciNet  Google Scholar 

  • Davidov O, Herman A (2012) Ordinal dominance curve based inference for stochastically ordered distributions. J R Stat Soc Ser B 74(5):825–847

    MathSciNet  MATH  Google Scholar 

  • Davidov O, Fokianos K, Iliopoulos G (2014) Semiparametric inference for the two-way layout under order restrictions. Scand J Stat 41(3):622–638

    MathSciNet  MATH  Google Scholar 

  • Drezner Z (1994) Computation of the trivariate normal integral. Math Comput 62(205):289–294

    MathSciNet  MATH  Google Scholar 

  • Duarte BPM, Granjo JFO, Wong WK (2020) Optimal exact designs of experiments via mixed integer nonlinear programming. Stat Comput 30:93–112

    MathSciNet  MATH  Google Scholar 

  • Dunnett CW (1955) A multiple comparison procedure for comparing several treatments with a control. J Am Stat Assoc 50(272):1096–1121

    MATH  Google Scholar 

  • Dunnett CW (1964) New tables for multiple comparisons with a control. Biometrics 20:482–491

    MATH  Google Scholar 

  • Dunnett CW, Sobel M (1954) A bivariate generalization of student’s t-distribution, with tables for certain special cases. Biometrika 41(1–2):153–169. https://doi.org/10.1093/biomet/41.1-2.153

    Article  MathSciNet  MATH  Google Scholar 

  • Eriksson D, Bindel D, Shoemaker CA (2019) pySOT and POAP: An event-driven asynchronous framework for surrogate optimization

  • Farnan L, Ivanova A, Peddada SD (2014) Linear mixed effects models under inequality constraints with applications. PLoS ONE 9(1):8

    Google Scholar 

  • GAMS Development Corporation (2013) GAMS—A User’s Guide, GAMS Release 24.2.1. GAMS Development Corporation, Washington

    Google Scholar 

  • Genz A (2004) Numerical computation of rectangular bivariate and trivariate normal and t probabilities. Stat Comput 14:251–260

    MathSciNet  Google Scholar 

  • Genz A, Bretz F (2002) Comparison of methods for the computation of multivariate t probabilities. J Comput Graph Stat 11(4):950–971

    MathSciNet  Google Scholar 

  • Gleser LJ (1973) On a theory of intersection-union tests. Inst Math Stat Bull 2:233

    Google Scholar 

  • Gutmann HM (2001) A radial basis function method for global optimization. J Glob Optim 19:201–227

    MathSciNet  MATH  Google Scholar 

  • Higham NJ (1988) Computing a nearest symmetric positive semidefinite matrix. Linear Algebra Appl 103:103–118. https://doi.org/10.1016/0024-3795(88)90223-6

    Article  MathSciNet  MATH  Google Scholar 

  • Hirotsu C, Herzberg AM (1987) Optimal allocation of observations for inference on \(k\) ordered normal population means. Austral J Stat 29(2):151–165

    MathSciNet  MATH  Google Scholar 

  • Hwang JTG, Peddada SD (1994) Confidence interval estimation subject to order restrictions. Ann Stat 22(1):67–93

    MathSciNet  MATH  Google Scholar 

  • Jin R, Chen W, Sudjianto A (2005) An efficient algorithm for constructing optimal design of computer experiments. J Stat Plann Inference 134(1):268–287. https://doi.org/10.1016/j.jspi.2004.02.014

    Article  MathSciNet  MATH  Google Scholar 

  • Kim SH, Boukouvala F (2020) Surrogate-based optimization for mixed-integer nonlinear problems. Comput Chem Eng 140:106847

    Google Scholar 

  • Koehler JR, Owen AB (1996) Computer experiments. In: Gosh S, Rao CR (eds) Handbook of statistics, vol. 13, design and analysis of experiments. Elsevier, Amsterdam, pp 261–308

    Google Scholar 

  • Le Digabel S (2011) Algorithm 909: NOMAD: nonlinear optimization with the MADS algorithm. ACM Trans Math Softw 37(4):1–15

    MathSciNet  MATH  Google Scholar 

  • Lee RE, Spurrier JD (1995) Successive comparisons between ordered treatments. J Stat Plan Inference 43(3):323–330. https://doi.org/10.1016/0378-3758(95)91803-B

    Article  MathSciNet  MATH  Google Scholar 

  • Lehmann EL (1952) Testing multiparameter hypotheses. Ann Math Stat 23(4):541–552

    MathSciNet  MATH  Google Scholar 

  • Martin JD, Simpson TW (2005) Use of kriging models to approximate deterministic computer models. AIAA J 43(4):853–863

    Google Scholar 

  • Müller J (2014) MATSuMoTo: the Matlab surrogate model toolbox for computationally expensive black-box global optimization problems. http://arxiv.org/abs/1404.4261 (1404.4261v1)

  • Müller J (2016) MISO: mixed-integer surrogate optimization framework. Optim Eng 17:177–203

    MathSciNet  MATH  Google Scholar 

  • Müller J, Day M (2019) Surrogate optimization of computationally expensive black-box problems with hidden constraints. INFORMS J Comput 31(4):689–702

    MathSciNet  MATH  Google Scholar 

  • Müller J, Woodbury JD (2017) GOSAC: global optimization with surrogate approximation of constraints. J Global Optim 69:117–136

    MathSciNet  MATH  Google Scholar 

  • Müller J, Shoemaker CA, Piché R (2013) SO-MI: a surrogate model algorithm for computationally expensive nonlinear mixed-integer black-box global optimization problems. Comput Oper Res 40(5):1383–1400

    MathSciNet  MATH  Google Scholar 

  • Overstall AM, Woods DC (2017) Bayesian design of experiments using approximate coordinate exchange. Technometrics 59(4):458–470. https://doi.org/10.1080/00401706.2016.1251495

    Article  MathSciNet  Google Scholar 

  • Powell MJD (1992) The theory of radial basis function approximation in 1990. In: Light WA (ed) Advances in numerical analysis II: wavelets, subdivision, and radial functions. Oxford University Press, Oxford, pp 105–210

    Google Scholar 

  • Pukelsheim F (1993) Optimal design of experiments. SIAM, Philadelphia

    MATH  Google Scholar 

  • Regis RG, Shoemaker CA (2007) A stochastic radial basis function method for the global optimization of expensive functions. INFORMS J Comput 19(4):497–509

    MathSciNet  MATH  Google Scholar 

  • Regis RG, Shoemaker CA (2013) Combining radial basis function surrogates and dynamic coordinate search in high-dimensional expensive black-box optimization. Eng Optim 45(5):529–555. https://doi.org/10.1080/0305215X.2012.687731

    Article  MathSciNet  Google Scholar 

  • Rosa S (2018) Optimal designs for treatment comparisons represented by graphs. AStA Adv Stat Anal 102(4):479–503

    MathSciNet  MATH  Google Scholar 

  • Sahinidis N (2014) BARON 14.3.1: global optimization of mixed-integer nonlinear programs, User’s Manual. The Optimization Firm LLC, Pittsburgh

    Google Scholar 

  • Saikali KG, Berger RL (2002) More powerful tests for the sign testing problem. J Stat Plann Inference 107(1):187–205

    MathSciNet  MATH  Google Scholar 

  • Sebastiani P, Wynn HP (2000) Maximum entropy sampling and optimal Bayesian experimental design. J R Stat Soc Series B (Statistical Methodology) 62(1):145–157. https://doi.org/10.1111/1467-9868.00225

    Article  MathSciNet  MATH  Google Scholar 

  • Shewry MC, Wynn HP (1987) Maximum entropy sampling. J Appl Stat 14(2):165–170. https://doi.org/10.1080/02664768700000020

    Article  Google Scholar 

  • Sibson R (1974) D\(_{\text{A}}\)-optimality and duality. In: Gani, J., Sarkadi, K., Vincze, I. (eds.) Progress in Statistics, Vol.2 – Proc. 9th European Meeting of Statisticians, Budapest, pp 677–692. North-Holland, Amsterdam

  • Silvey SD (1980) Optimal design. Chapman & Hall, London

    MATH  Google Scholar 

  • Singh SP, Davidov O (2019) On the design of experiments with ordered treatments. J R Stat Soc B 81(5):881–900

    MathSciNet  MATH  Google Scholar 

  • Singh SP, Davidov O (2021) On efficient exact experimental designs for ordered treatments. Comput Stat Data Anal 164:107305. https://doi.org/10.1016/j.csda.2021.107305

    Article  MathSciNet  MATH  Google Scholar 

  • Singh B, Schell MJ, Wright FT (1993) The power functions of the likelihood ratio tests for a simple tree ordering in normal means: unequal weights. Commun Stat 22(2):425–449

    MathSciNet  MATH  Google Scholar 

  • Singh B, Halabi S, Schell MJ (2008) Sample size selection in clinical trials when population means are subject to a partial order: one-sided ordered alternatives. J Appl Stat 35(5):583–600

    MathSciNet  MATH  Google Scholar 

  • Tamhane AC (1996) Multiple comparisons. In: Gosh S, Rao CR (eds) Handbook of statistics, vol. 13, design and analysis of experiments. Elsevier, Amsterdam, pp 587–630

    Google Scholar 

  • Vanbrabant L, Van De Schoot R, Rosseel Y (2015) Constrained statistical inference: sample-size tables for ANOVA and regression. Front Psychol 5:1565

    Google Scholar 

  • Waite TW, Woods DC (2015) Designs for generalized linear models with random block effects via information matrix approximations. Biometrika 102(3):677–693. https://doi.org/10.1093/biomet/asv005

    Article  MathSciNet  MATH  Google Scholar 

  • Xiong C, Yu K, Gao F, Yan Y, Zhang Z (2005) Power and sample size for clinical trials when efficacy is required in multiple endpoints: application to an Alzheimer’s treatment trial. Clin Trials 2(5):387–393

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Chemical and Biological Engineering, Rua Pedro Nunes, Quinta da Nora, Instituto Politécnico de Coimbra, Instituto Superior de Engenharia de Coimbra, 3030-199, Coimbra, Portugal

    Belmiro P. M. Duarte

  2. CIEPQPF, Department of Chemical Engineering, University of Coimbra, Rua Sílvio Lima-Pólo II, 3030-790, Coimbra, Portugal

    Belmiro P. M. Duarte & Marco S. Reis

  3. Department of Statistics, London School of Economics, London, WC2A 2AE, UK

    Anthony C. Atkinson

  4. Department of Mathematics and Statistics, Indian Institute of Technology Kanpur, Kanpur, 208016, India

    Satya P. Singh

Authors
  1. Belmiro P. M. Duarte
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anthony C. Atkinson
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Satya P. Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Marco S. Reis
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

BPMD: Research, Conceptualization, Methodology, Writing original draft preparation. ACA: Research, Validation, Reviewing and editing. SPS: Validation, Reviewing and editing. MSR: Validation, Reviewing and editing.

Corresponding author

Correspondence to Belmiro P. M. Duarte.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix: Optimal designs for simple order relation

Appendix: Optimal designs for simple order relation

Here we present the optimal designs for tree ordering resulting from varying N and \(\delta \).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Duarte, B.P.M., Atkinson, A.C., Singh, S.P. et al. Optimal design of experiments for hypothesis testing on ordered treatments via intersection-union tests. Stat Papers 64, 587–615 (2023). https://doi.org/10.1007/s00362-022-01334-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00362-022-01334-8

Keywords

Mathematics Subject Classification

Search

Navigation