Skip to main content
SpringerLink
Log in

Optimal exact designs of experiments via Mixed Integer Nonlinear Programming

  • Published:
Statistics and Computing Aims and scope Submit manuscript

Abstract

Optimal exact designs are problematic to find and study because there is no unified theory for determining them and studying their properties. Each has its own challenges and when a method exists to confirm the design optimality, it is invariably applicable to the particular problem only. We propose a systematic approach to construct optimal exact designs by incorporating the Cholesky decomposition of the Fisher Information Matrix in a Mixed Integer Nonlinear Programming formulation. As examples, we apply the methodology to find D- and A-optimal exact designs for linear and nonlinear models using global or local optimizers. Our examples include design problems with constraints on the locations or the number of replicates at the optimal design points.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  • Atkinson, A., Donev, A.: The construction of exact \(D\)-optimum experimental designs with application to blocking response surface designs. Biometrika 76(3), 515–526 (1989)

    MathSciNet  MATH  Google Scholar 

  • Atkinson, A., Donev, A., Tobias, R.: Optimum Experimental Designs, with SAS. Oxford University Press, Oxford (2007)

    MATH  Google Scholar 

  • Berman, S.: An extension of the arc sine law. Ann. Math. Stat. 33(2), 681–684 (1962)

    MathSciNet  MATH  Google Scholar 

  • Boer, E., Hendrix, E.: Global optimization problems in optimal design of experiments in regression models. J. Glob. Optim. 18, 385–398 (2000)

    MathSciNet  MATH  Google Scholar 

  • Box, G., Hunter, W.: The experimental study of physical mechanisms. Technometrics 7(1), 23–42 (1965)

    Google Scholar 

  • Chang, F.C., Imhof, L., Sun, Y.Y.: Exact \(D\)-optimal designs for first-order trigonometric regression models on a partial circle. Metrika Int. J. Theor. Appl. Stat. 76(6), 857–872 (2013)

    MathSciNet  MATH  Google Scholar 

  • Chang, F.C., Yeh, Y.R.: Exact \(A\)-optimal designs for quadratic regression. Stat. Sin. 8, 527–534 (1998)

    MathSciNet  MATH  Google Scholar 

  • Chernoff, H.: Locally optimal designs for estimating parameters. Ann. Math. Stat. 24, 586–602 (1953)

    MathSciNet  MATH  Google Scholar 

  • Coale, A., McNeil, D.: The distribution by age of the frequency of first marriage in a female cohort. J. Am. Stat. Assoc. 67(340), 743–749 (1972)

    Google Scholar 

  • Cook, R., Nachtsheim, C.: Comparison of algorithms for constructing \(D\)-optimal design. Technometrics 22(3), 315–324 (1980)

    MATH  Google Scholar 

  • Donev, A.: Crossover designs with correlated observations. J. Biopharm. Stat. 8(2), 249–262 (1998). PMID: 9598421

    MATH  Google Scholar 

  • Drud, A.: CONOPT: a GRG code for large sparse dynamic nonlinear optimization problems. Math. Program. 31, 153–191 (1985)

    MathSciNet  MATH  Google Scholar 

  • Du Croz, J., Higham, N.: Stability of methods for matrix inversion. IMA J. Numer. Anal. 12, 1–19 (1992)

    MathSciNet  MATH  Google Scholar 

  • Duarte, B., Wong, W.: Finding Bayesian optimal designs for nonlinear models: a semidefinite programming-based approach. Int. Stat. Rev. 83(2), 239–262 (2015)

    MathSciNet  Google Scholar 

  • Duarte, B., Wong, W., Atkinson, A.: A semi-infinite programming based algorithm for determining \(T\)-optimum designs for model discrimination. J. Multivar. Anal. 135, 11–24 (2015)

    MathSciNet  MATH  Google Scholar 

  • Duarte, B., Wong, W., Dette, H.: Adaptive grid semidefinite programming for finding optimal designs. Stat. Comput. 28(2), 441–460 (2018)

    MathSciNet  MATH  Google Scholar 

  • Duran, M., Grossmann, I.: An outer-approximation algorithm for a class of mixed-integer nonlinear programs. Math. Program. 36(3), 307–339 (1986)

    MathSciNet  MATH  Google Scholar 

  • Esteban-Bravo, M., Leszkiewicz, A., Vidal-Sanz, J.: Exact optimal experimental designs with constraints. Stat. Comput. 27(3), 845–863 (2017)

    MathSciNet  MATH  Google Scholar 

  • Fedorov, V.: Theory of Optimal Experiments. Academic Press, New York (1972)

    Google Scholar 

  • Fedorov, V., Leonov, S.: Optimal Design for Nonlinear Response Models. Chapman and Hall/CRC Press, Boca Raton (2014)

    MATH  Google Scholar 

  • Fletcher, R., Leyffer, S.: Numerical experience with lower bounds for MIQP branch-and-bound. SIAM J. Optim. 8(2), 604–616 (1998)

    MathSciNet  MATH  Google Scholar 

  • Floudas, C.: Mixed-integer nonlinear optimization. In: Pardalos, P., Resende, M. (eds.) Handbook of Applied Optimization, pp. 451–475. Oxford University Press, Oxford (2002)

    Google Scholar 

  • Gaffke, N.: On \(D\)-optimality of exact linear regression designs with minimum support. J. Stat. Plan. Inference 15, 189–204 (1987)

    MathSciNet  MATH  Google Scholar 

  • GAMS Development Corporation: GAMS—a user’s guide, GAMS release 24.2.1. GAMS Development Corporation, Washington, DC, USA (2013a)

  • GAMS Development Corporation: GAMS—the solver manuals, GAMS release 24.2.1. GAMS Development Corporation, Washington, DC, USA (2013b)

  • Golub, G.H., van Loan, C.F.: Matrix Computations, 4th edn. JHU Press, Baltimore (2013)

    MATH  Google Scholar 

  • Goos, P., Donev, A.: Blocking response surface designs. Comput. Stat. Data Anal. 51(2), 1075–1088 (2006)

    MathSciNet  MATH  Google Scholar 

  • Goos, P., Vandebroek, M.: \(D\)-optimal response surface designs in the presence of random block effects. Comput. Stat. Data Anal. 37(4), 433–453 (2001)

    MathSciNet  MATH  Google Scholar 

  • Goos, P., Vandebroek, M.: \(D\)-optimal split-plot designs with given numbers and sizes of whole plots. Technometrics 45(3), 235–245 (2003)

    MathSciNet  Google Scholar 

  • Gotwalt, C., Jones, B., Steinberg, D.: Fast computation of designs robust to parameter uncertainty for nonlinear settings. Technometrics 51(1), 88–95 (2009)

    MathSciNet  Google Scholar 

  • Gribik, P., Kortanek, K.: Equivalence theorems and cutting plane algorithms for a class of experimental design problems. SIAM J. Appl. Math. 32, 232–259 (1977)

    MathSciNet  MATH  Google Scholar 

  • Harman, R., Filová, L.: Computing efficient exact designs of experiments using integer quadratic programming. Comput. Stat. Data Anal. 71, 1159–1167 (2014)

    MathSciNet  MATH  Google Scholar 

  • Harman, R., Jurík, T.: Computing \(c\)-optimal experimental designs using the simplex method of linear programming. Comput. Stat. Data Anal. 53(2), 247–254 (2008)

    MathSciNet  MATH  Google Scholar 

  • Hill, A.: The possible effects of the aggregation of the molecules of haemoglobin on its dissociation curves. J. Physiol. 40(Suppl.), 4–7 (1910)

    Google Scholar 

  • Imhof, L.: \(A\)-optimum exact designs for quadratic regression. J. Math. Anal. Appl. 228, 157–165 (1998)

    MathSciNet  MATH  Google Scholar 

  • Imhof, L., Lopez-Fidalgo, J., Wong, W.K.: Efficiencies of rounded optimal approximate designs for small samples. Stat. Neerlandica 55(3), 301–318 (2001)

    MathSciNet  MATH  Google Scholar 

  • Johnson, M., Nachtsheim, C.: Some guidelines for constructing exact \(D\)-optimal designs on convex design spaces. Technometrics 25, 271–277 (1983)

    MathSciNet  MATH  Google Scholar 

  • Kiefer, J.: General equivalence theory for optimum design (approximate theory). Ann. Stat. 2, 849–879 (1974)

    MathSciNet  MATH  Google Scholar 

  • Laird, A.: Dynamics of tumor growth. Br. J. Cancer 18(3), 490–502 (1964)

    Google Scholar 

  • Lastusilta, T., Bussieck, M., Westerlund, T.: Comparison of some high-performance MINLP solvers. Chem. Eng. Trans. 11, 125–130 (2007)

    Google Scholar 

  • Leszkiewicz, A.: Three essays on conjoint analysis: optimal design and estimation of endogenous consideration sets. Ph.D. thesis, Universidad Carlos III de Madrid (2014)

  • López-Fidalgo, J., Tommasi, C., Trandafir, P.: Optimal designs for discriminating between some extensions of the Michaelis–Menten model. J. Stat. Plan. Inference 138(12), 3797–3804 (2008)

    MathSciNet  MATH  Google Scholar 

  • Mandal, A., Wong, W.K., Yu, Y.: Algorithmic searches for optimal designs. In: Dean, A., Morris, M., Stufken, J., Bingham, D. (eds.) Handbook of Design and Analysis of Experiments, pp. 755–786. CRC Press, Boca Ratton (2015)

    Google Scholar 

  • Meyer, R., Nachtsheim, C.: The coordinate-exchange algorithm for constructing exact optimal experimental designs. Technometrics 37, 60–69 (1995)

    MathSciNet  MATH  Google Scholar 

  • Mitchell, T.: An algorithm for the construction of \(D\)-optimal experimental designs. Technometrics 20, 203–210 (1974)

    MathSciNet  MATH  Google Scholar 

  • Mitchell, T., Miller Jr., F.: Use of Design Repair to Construct Designs for Special Linear Models. Technical Report, pp. 130–131, Oak Ridge National Laboratory (1970)

  • Molchanov, I., Zuyev, S.: Steepest descent algorithm in a space of measures. Stat. Comput. 12, 115–123 (2002)

    MathSciNet  Google Scholar 

  • Nuñez Ares, J., Goos, P.: An integer linear programming approach to find trend-robust run orders of experimental designs. J. Qual. Technol. 51(1), 37–50 (2019). https://doi.org/10.1080/00224065.2018.1545496

    Article  Google Scholar 

  • Palhazi Cuervo, D., Goos, P., Sörensen, K.: Optimal design of large-scale screening experiments: a critical look at the coordinate-exchange algorithm. Stat. Comput. 26(1), 15–28 (2016)

    MathSciNet  MATH  Google Scholar 

  • Pantelides, C.: The consistent initialization of differential-algebraic systems. SIAM J. Sci. Stat. Comput. 9(2), 213–231 (1988)

    MathSciNet  MATH  Google Scholar 

  • Pukelsheim, F.: Optimal Design of Experiments. SIAM, Philadelphia (1993)

    MATH  Google Scholar 

  • Rasch, D., Hendrix, E., Boer, E.: Replication-free optimal designs in regression analysis. Comput. Stat. 12, 19–52 (1997)

    MATH  Google Scholar 

  • Sagnol, G.: Computing optimal designs of multiresponse experiments reduces to second-order cone programming. J. Stat. Plan. Inference 141(5), 1684–1708 (2011)

    MathSciNet  MATH  Google Scholar 

  • Sagnol, G., Harman, R.: Computing exact \(D\)-optimal designs by mixed integer second order cone programming. Ann. Stat. 43(5), 2198–2224 (2015)

    MathSciNet  MATH  Google Scholar 

  • Sahinidis, N.: BARON 14.3.1: global optimization of mixed-integer nonlinear programs, user’s manual. The Optimization Firm, LLC, Pittsburgh (2014)

  • Sartono, B., Goos, P., Schoen, E.: Constructing general orthogonal fractional factorial split-plot designs. Technometrics 57(4), 488–502 (2015a)

    MathSciNet  Google Scholar 

  • Sartono, B., Schoen, E., Goos, P.: Blocking orthogonal designs with mixed integer linear programming. Technometrics 57(3), 428–439 (2015b)

    MathSciNet  Google Scholar 

  • SAS Institute Inc.: \(\text{JMP}^{\textregistered }\) 13 User Guide, 2nd edn. SAS Institute Inc, Cary (2017)

  • Tawarlamani, M., Sahinidis, N.: Convexification and Global Optimization in Continuous and Mixed Integer Nonlinear Programming, 1st edn. Kluwer Academic Publishers, Dordrecht (2002)

    Google Scholar 

  • Vandenberghe, L., Boyd, S.: Applications of semidefinite programming. Appl. Numer. Math. 29, 283–299 (1999)

    MathSciNet  MATH  Google Scholar 

  • Vo-Thanh, N., Jans, R., Schoen, E., Goos, P.: Symmetry breaking in mixed integer linear programming formulations for blocking two-level orthogonal experimental designs. Comput. Oper. Res. 97, 96–110 (2018)

    MathSciNet  MATH  Google Scholar 

  • Welch, W.: Branch-and-bound search for experimental designs based on \(D\)-optimality and other criteria. Technometrics 24(1), 41–48 (1982)

    MathSciNet  MATH  Google Scholar 

  • Westerlund, T., Pettersson, F.: An extended cutting plane method for solving convex MINLP problems. Comput. Chem. Eng. 19, 131–136 (1995)

    Google Scholar 

  • Wynn, H.: The sequential generation of \(D\)-optimum experimental designs. Ann. Math. Stat. 41(5), 1655–1664 (1970)

    MathSciNet  MATH  Google Scholar 

  • Yang, J., Mandal, A., Majumdar, D.: Optimal design for two-level factorial experiments with binary response. Stat. Sin. 22(2), 885–907 (2012)

    MathSciNet  MATH  Google Scholar 

  • Yang, J., Mandal, A., Majumdar, D.: Optimal designs for \(2^k\) factorial experiments with binary response. Stat. Sin. 26, 381–411 (2016)

    MATH  Google Scholar 

Download references

Acknowledgements

The research of Wong is partially supported by a grant from the National Institute of General Medical Sciences of the National Institutes of Health under Award Number R01GM107639. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. The authors acknowledge two anonymous reviewers that contributed undoubtedly to improve the quality of the paper.

Author information

Authors and Affiliations

  1. Department of Chemical and Biological Engineering, Instituto Politécnico de Coimbra, Instituto Superior de Engenharia de Coimbra, Rua Pedro Nunes, Quinta da Nora, 3030–199, Coimbra, Portugal

    Belmiro P. M. Duarte

  2. Department of Chemical Engineering, CIEPQPF, University of Coimbra, Rua Sílvio Lima – Pólo II, 3030-790, Coimbra, Portugal

    Belmiro P. M. Duarte & José F. O. Granjo

  3. Department of Biostatistics, Fielding School of Public Health, UCLA, 10833 Le Conte Ave., Los Angeles, CA, 90095–1772, USA

    Weng Kee Wong

Authors
  1. Belmiro P. M. Duarte
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. José F. O. Granjo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Weng Kee Wong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Belmiro P. M. Duarte.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 107 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Duarte, B.P.M., Granjo, J.F.O. & Wong, W.K. Optimal exact designs of experiments via Mixed Integer Nonlinear Programming. Stat Comput 30, 93–112 (2020). https://doi.org/10.1007/s11222-019-09867-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11222-019-09867-z

Keywords

Mathematics Subject Classification

Search

Navigation