Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Adaptive grid semidefinite programming for finding optimal designs

Abstract

We find optimal designs for linear models using a novel algorithm that iteratively combines a semidefinite programming (SDP) approach with adaptive grid techniques. The proposed algorithm is also adapted to find locally optimal designs for nonlinear models. The search space is first discretized, and SDP is applied to find the optimal design based on the initial grid. The points in the next grid set are points that maximize the dispersion function of the SDP-generated optimal design using nonlinear programming. The procedure is repeated until a user-specified stopping rule is reached. The proposed algorithm is broadly applicable, and we demonstrate its flexibility using (i) models with one or more variables and (ii) differentiable design criteria, such as A-, D-optimality, and non-differentiable criterion like E-optimality, including the mathematically more challenging case when the minimum eigenvalue of the information matrix of the optimal design has geometric multiplicity larger than 1. Our algorithm is computationally efficient because it is based on mathematical programming tools and so optimality is assured at each stage; it also exploits the convexity of the problems whenever possible. Using several linear and nonlinear models with one or more factors, we show the proposed algorithm can efficiently find optimal designs.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

References

  1. Andersen, E., Jensen, B., Jensen, J., Sandvik, R., Worsøe, U.: Mosek version 6. Tech. rep., Technical Report TR-2009-3, MOSEK (2009)

  2. Arthanari, T.S., Dodge, Y.: Mathematical Programming in Statistics. A Wiley-Interscience publication. Wiley, Hoboken (1993)

  3. Atkinson, A.C., Donev, A.N.: The construction of exact \(D\)-optimum experimental designs with application to blocking response surface designs. Biometrika 76(3), 515–526 (1989)

  4. Atkinson, A.C., Donev, A.N., Tobias, R.D.: Optimum Experimental Designs, with SAS. Oxford University Press, Oxford (2007)

  5. Ben-Tal, A., Nemirovski, A.S.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. Society for Industrial and Applied Mathematics, Philadelphia (2001)

  6. Berger, M.J.: Adaptive mesh refinement for hyperbolic partial differential equations. Ph.D. Dissertation, Department of Computer Science, Stanford University, Stanford, CA, USA (1982)

  7. Berger, M.P.F., Wong, W.K.: An Introduction to Optimal Designs for Social and Biomedical Research. Wiley, Chichester (2009)

  8. Bischof, C.H., Bücker, H.M., Lang, B., Rasch, A., Vehreschild, A.: Combining source transformation and operator overloading techniques to compute derivatives for Matlab programs. In: Proceedings of the Second IEEE International Workshop on Source Code Analysis and Manipulation (SCAM 2002), pp. 65–72. IEEE Computer Society, Los Alamitos, CA, USA (2002)

  9. Box, G.E.P., Hunter, W.G.: The experimental study of physical mechanisms. Technometrics 7(1), 23–42 (1965)

  10. Boyd, S., Vandenberghe, L.: Convex Optimization. University Press, Cambridge (2004)

  11. Byrd, R.H., Hribar, M.E., Nocedal, J.: An interior point algorithm for large-scale nonlinear programming. SIAM J. Optim. 9(4), 877–900 (1999)

  12. Chaloner, K., Larntz, K.: Optimal Bayesian design applied to logistic regression experiments. J. Stat. Plan. Inference 59, 191–208 (1989)

  13. Chen, R.B., Chang, S.P., Wang, W., Tung, H.C., Wong, W.K.: Minimax optimal designs via particle swarm optimization methods. Stat. Comput. 25(5), 975–988 (2015)

  14. Coale, A., McNeil, D.: The distribution by age of the frequency of first marriage in a female cohort. J. Am. Stat. Assoc. 67(340), 743–749 (1972)

  15. Coleman, T.F., Li, Y.: On the convergence of reflective Newton methods for large-scale nonlinear minimization subject to bounds. Math. Program. 67(2), 189–224 (1994)

  16. Cook, R., Nachtsheim, C.: Comparison of algorithms for constructing \(D\)-optimal design. Technometrics 22(3), 315–324 (1980)

  17. Dette, H., Grigoriev, Y.: E-optimal designs for second-order response surface models. Ann. Stat. 42(4), 1635–1656 (2014)

  18. Dette, H., Pepelyshev, A., Zhigljavsky, A.A.: Improving updating rules in multiplicative algorithms for computing \(D\)-optimal designs. Comput. Stat. Data Anal. 53(2), 312–320 (2008)

  19. Dette, H., Studden, W.J.: Geometry of \(E\)-optimality. Ann. Stat. 21(1), 416–433 (1993)

  20. Drud, A.: CONOPT: a GRG code for large sparse dynamic nonlinear optimization problems. Math. Program. 31, 153–191 (1985)

  21. Drud, A.: CONOPT—a large-scale GRG code. ORSA J. Comput. 6(2), 207–216 (1994)

  22. Duarte, B.P., Wong, W.K.: A semi-infinite programming based algorithm for finding minimax optimal designs for nonlinear models. Stat. Comput. 24(6), 1063–1080 (2014)

  23. Duarte, B.P.M., Wong, W.K.: Finding Bayesian optimal designs for nonlinear models: a semidefinite programming-based approach. Int. Stat. Rev. 83(2), 239–262 (2015)

  24. Duarte, B.P., Wong, W.K., Atkinson, A.C.: A semi-infinite programming based algorithm for determining T-optimum designs for model discrimination. J. Multivar. Anal. 135, 11–24 (2015)

  25. Fedorov, V.V.: Theory of Optimal Experiments. Academic Press, Cambridge (1972)

  26. Fedorov, V.V., Leonov, S.L.: Optimal Design for Nonlinear Response Models. Chapman and Hall/CRC Press, Boca Raton (2014)

  27. Fellman, J.: An empirical study of a class of iterative searches for optimal designs. J. Stat. Plan. Inference 21, 85–92 (1989)

  28. Filová, L., Trnovská, M., Harman, R.: Computing maximin efficient experimental designs using the methods of semidefinite programming. Metrika 64(1), 109–119 (2011)

  29. Gaivoronski, A.: Linearization methods for optimization of functionals which depend on probability measures. In: Prékopa, A., Wets, R.J.B. (eds.) Stochastic Programming 84 Part II, Mathematical Programming Studies, vol. 28, pp. 157–181. Springer, Berlin Heidelberg (1986)

  30. Gill, P.E., Murray, W., Saunders, M.A.: SNOPT: an SQP algorithm for large-scale constrained optimization. SIAM Rev. 47(1), 99–131 (2005)

  31. Goos, P., Jones, B.: Optimal Design of Experiments: A Case Study Approach. Wiley, New York (2011)

  32. Grant, M., Boyd, S., Ye, Y.: cvx Users guide for cvx version 1.22. CVX Research, Inc., 1104 Claire Ave., Austin, TX 78703-2502 (2012)

  33. Harman, R., Jurík, T.: Computing c-optimal experimental designs using the simplex method of linear programming. Comput. Stat. Data Anal. 53(2), 247–254 (2008)

  34. Harman, R., Pronzato, L.: Improvements on removing non-optimal support points in \(D\)-optimum design algorithms. Stat. Probab. Lett. 77, 90–94 (2007)

  35. Heredia-Langner, A., Montgomery, D.C., Carlyle, W.M., Borror, C.M.: Model-robust optimal designs: a genetic algorithm approach. J. Qual. Technol. 36, 263–279 (2004)

  36. Hill, A.: The possible effects of the aggregation of the molecules of haemoglobin on its dissociation curves. J Physiol 40(Suppl), 4–7 (1910)

  37. Kiefer, J.C.: Optimum experimental designs. J. R. Stat. Soc. Ser. B 21, 272–319 (1959)

  38. Kiefer, J.: General equivalence theory for optimum design (approximate theory). Ann. Stat. 2, 849–879 (1974)

  39. Kiefer, J., Wolfowitz, J.: The equivalence of two extremum problem. Can. J. Math. 12, 363–366 (1960)

  40. López-Fidalgo, J., Tommasi, C., Trandafir, P.C.: Optimal designs for discriminating between some extensions of the Michaelis–Menten model. J. Stat. Plan. Inference 138(12), 3797–3804 (2008)

  41. Mandal, S., Torsney, B.: Construction of optimal designs using a clustering approach. J. Stat. Plan. Inference 136, 1120–1134 (2006)

  42. Mandal, A., Wong, W.K., Yu, Y.: Algorithmic searches for optimal designs. Handbook of Design and Analysis of Experiments, chap 21, pp. 755–786. CRC Press, New York (2015)

  43. Masoudi, E., Holling, H., Wong, W.K.: Application of imperialist competitive algorithm to find minimax and standardized maximin optimal designs. Comput. Stat. Data Anal. (in press). doi:10.1016/j.csda.2016.06.014

  44. Melas, V.: Functional Approach to Optimal Experimental Design, Lecture Notes in Statistics. Springer (2006)

  45. Meyer, R.K., Nachtsheim, C.J.: The coordinate-exchange algorithm for constructing exact optimal experimental designs. Technometrics 37, 60–69 (1995)

  46. Mitchell, T.J.: An algorithm for the construction of \(D\)-optimal experimental designs. Technometrics 16, 203–210 (1974)

  47. Mitchell, T.J., Miller Jr., F.L.: Use of design repair to construct designs for special linear models. Technical report, Oak Ridge National Laboratory, 130–131 (1970)

  48. Molchanov, I., Zuyev, S.: Steepest descent algorithm in a space of measures. Stat. Comput. 12, 115–123 (2002)

  49. Papp, D.: Optimal designs for rational function regression. J. Am. Stat. Assoc. 107, 400–411 (2012)

  50. Pázman, A.: Foundations of Optimum Experimental Design (Mathematics and its Applications). Springer, Netherlands (1986)

  51. Peraire, J., Vahdati, M., Morgan, K., Zienkiewicz, O.: Adaptive remeshing for compressible flow computations. J. Comput. Phys. 72(2), 449–466 (1987)

  52. Pronzato, L.: Removing non-optimal support points in \(D\)-optimum design algorithms. Stat. Probab. Lett. 63(3), 223–228 (2003)

  53. Pronzato, L.: Optimal experimental design and some related control problems. Automatica 44, 303–325 (2008)

  54. Pronzato, L., Zhigljavsky, A.A.: Algorithmic construction of optimal designs on compact sets for concave and differentiable criteria. J. Stat. Plan. Inference 154, 141–155 (2014)

  55. Pukelsheim, F.: Optimal Design of Experiments. SIAM, Philadelphia (1993)

  56. Pukelsheim, F., Torsney, B.: Optimal weights for experimental designs on linearly independent support points. Ann. Stat. 19(3), 1614–1625 (1991)

  57. Qi, H.: A semidefinite programming study of the Elfving theorem. J. Stat. Plan. Inference 141, 3117–3130 (2011)

  58. Ruszczyński, A.P.: Nonlinear optimization, No. vol. 13. In: Nonlinear Optimization. Princeton University Press, Princeton (2006)

  59. Sagnol, G.: Computing optimal designs of multiresponse experiments reduces to second-order cone programming. J. Stat. Plan. Inference 141(5), 1684–1708 (2011)

  60. Sagnol, G.: On the semidefinite representation of real functions applied to symmetric matrices. Linear Algebra Appl. 439(10), 2829–2843 (2013)

  61. Sagnol, G., Harman, R.: Computing exact \(D\)-optimal designs by mixed integer second order cone programming. Ann. Stat. 43(5), 2198–2224 (2015)

  62. Silvey, S.D.: Optimal Design. Chapman and Hall, London (1980)

  63. Sturm, J.: Using SeDuMi 1.02, a Matlab toolbox for optimization oversymmetric cones. Optim Methods Softw. 11, 625–653 (1999)

  64. Titterington, D.M.: Algorithms for computing \(D\)-optimal design on finite design spaces. In: Proceedings of the 1976 Conference on Information Science and Systems, 3, John Hopkins University, 213–216 (1976)

  65. Torsney, B., Mandal, S.: Two classes of multiplicative algorithms for constructing optimizing distributions. Comput. Stat. Data Anal. 51(3), 1591–1601 (2006)

  66. Torsney, B., Martín-Martín, R.: Multiplicative algorithms for computing optimum designs. J. Stat. Plan. Inference 139(12), 3947–3961 (2009)

  67. Vandenberghe, L., Boyd, S.: Semidefinite programming. SIAM Rev. 8, 49–95 (1996)

  68. Vandenberghe, L., Boyd, S.: Applications of semidefinite programming. Appl. Numer. Math. 29, 283–299 (1999)

  69. Wächter, A., Biegler, T.L.: On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Math. Program. 106(1), 25–57 (2005)

  70. Welch, W.J.: Algorithmic complexity: three NP-hard problems in computational statistics. J. Stat. Comput. Simul. 15(1), 17–25 (1982)

  71. Whittle, P.: Some general points in the theory of optimal experimental design. J. R. Stat. Soc. Ser. B 35, 123–130 (1973)

  72. Woods, D.C.: Robust designs for binary data: applications of simulated annealing. J. Stat. Comput. Simul. 80(1), 29–41 (2010)

  73. Wu, C.F.: Some algorithmic aspects of the theory of optimal designs. Ann. Stat. 6(6), 1286–1301 (1978)

  74. Wu, C.F., Wynn, H.P.: The convergence of general step-length algorithms for regular optimum design criteria. Ann. Stat. 6(6), 1273–1285 (1978)

  75. Wynn, H.P.: The sequential generation of \(D\)-optimum experimental designs. Ann. Math. Stat. 41(5), 1655–1664 (1970)

  76. Wynn, H.P.: Results in the theory and construction of \(D\)-optimum experimental designs. J. R. Stat. Soc. Ser. B 34, 133–147 (1972)

  77. Yang, M., Biedermann, S., Tang, E.: On optimal designs for nonlinear models: a general and efficient algorithm. J. Am. Stat. Assoc. 108(504), 1411–1420 (2013)

  78. Yu, Y.: \(D-\)optimal designs via a cocktail algorithm. Stat. Comput. 21(4), 475–481 (2010a)

  79. Yu, Y.: Monotonic convergence of a general algorithm for computing optimal designs. Ann. Stat. 38(3), 1593–1606 (2010b)

  80. Yu, Y.: Strict monotonicity and convergence rate of Titterington’s algorithm for computing \(D\)-optimal designs. Comput. Stat. Data. Anal. 54, 1419–1425 (2010c)

Download references

Acknowledgements

Dette’s work was partially supported by the Collaborative Research Center “Statistical modeling of nonlinear dynamic processes” (SFB 823, Teilprojekt C2) of the German Research Foundation (DFG). Dette and Wong were also partially supported by a Grant from the National Institute of General Medical Sciences of the National Institutes of Health under Award Number R01GM107639. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Author information

Correspondence to Belmiro P. M. Duarte.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Duarte, B.P.M., Wong, W.K. & Dette, H. Adaptive grid semidefinite programming for finding optimal designs. Stat Comput 28, 441–460 (2018). https://doi.org/10.1007/s11222-017-9741-y

Download citation

Keywords

  • Adaptive grid
  • Continuous design
  • Model-based optimal design
  • Nonlinear programming
  • Semidefinite programming

Mathematics Subject Classification

  • 62K05
  • 90C47