Minimax optimal designs via particle swarm optimization methods

Abstract

Particle swarm optimization (PSO) techniques are widely used in applied fields to solve challenging optimization problems but they do not seem to have made an impact in mainstream statistical applications hitherto. PSO methods are popular because they are easy to implement and use, and seem increasingly capable of solving complicated problems without requiring any assumption on the objective function to be optimized. We modify PSO techniques to find minimax optimal designs, which have been notoriously challenging to find to date even for linear models, and show that the PSO methods can readily generate a variety of minimax optimal designs in a novel and interesting way, including adapting the algorithm to generate standardized maximin optimal designs.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

References

  1. Atkinson, A.C., Donev, A.N., Tobias, R.D.: Optimum Experimental Designs, with SAS. Oxford University Press, Oxford (2007)

    MATH  Google Scholar 

  2. Berger, M.P.F.: Applied Optimal Designs. Wiley, New York (2005)

    Book  MATH  Google Scholar 

  3. Berger, M.P.F., King, J., Wong, W.K.: Minimax \(D\)-optimal designs for item response theory models. Psychometrika 65(3), 377–390 (2000)

    MathSciNet  Article  MATH  Google Scholar 

  4. Bogacka, B., Patan, M., Johnson, P.J., Youdim, K., Atkinson, A.C.: Optimum design of experiments for enzyme inhibition linetic models. J. Biopharm. Stat. 21(3), 555–572 (2011)

    MathSciNet  Article  Google Scholar 

  5. Broniatowski, M., Celant, G.: Optimality and bias of some interpolation and extrapolation designs. J. Stat. Plan. Inference 137(3), 858–868 (2007)

    MathSciNet  Article  MATH  Google Scholar 

  6. Brown, L.D., Wong, W.K.: An algorithmic construction of optimal minimax designs for heteroscedastic linear models. J. Stat. Plan. Inference 85(1–2), 103–114 (2000)

    MathSciNet  Article  MATH  Google Scholar 

  7. Chatterjee, A., Siarry, P.: Nonlinear inertia weight variation for dynamic adaptation in particle swarm optimization. Comput. Oper. Res. 33(3), 859–871 (2006)

    Article  MATH  Google Scholar 

  8. Chen, R.B., Wong, W.K., Li, K.Y.: Optimal minimax designs over a prespecified interval in a heteroscedastic polynomial model. Stat. Probab. Lett. 78(13), 1914–1921 (2008)

    MathSciNet  Article  MATH  Google Scholar 

  9. Clerc, M.: Particle Swarm Optimization. Wiley, New York (2006)

    Book  MATH  Google Scholar 

  10. Cook, R.D., Wong, W.K.: On the equivalence of constrained and compound optimal designs. J. Am. Stat. Assoc. 89, 687–692 (1994)

    MathSciNet  Article  MATH  Google Scholar 

  11. Dette, H., Biedermann, S.: Robust and efficient designs for the Michaelis–Menten model. J. Am. Stat. Assoc. 98(463), 679–686 (2003)

    MathSciNet  Article  MATH  Google Scholar 

  12. Dette, H., Wong, W.K.: \(E\)-optimal designs for the Michaelis–Menten model. Stat. Probab. Lett. 44(4), 405–408 (1999)

    MathSciNet  Article  MATH  Google Scholar 

  13. Eberhart, R., Kennedy, J.: A New Optimizer Using Particle Swarm Theory. In: Proceedings of the Sixth International Symposium on, IEEE, Micro Machine and Human Science, 1995. MHS’95, pp. 39–43 (1995)

  14. Eberhart, RC., Shi, Y.: Comparing Inertia Weights and Constriction Factors in Particle Swarm Optimization. In: Proceedings of the 2000 Congress on, IEEE, Evolutionary Computation, 2000, vol. 1, pp. 84–88 (2000)

  15. Fan, S.K.S., Chang, J.M.: A Modified Particle Swarm Optimizer Using an Adaptive Dynamic Weight Scheme. In: Proceedings of the 1st International Conference on Digital Human Modeling, Springer, New York, pp. 56–65 (2007)

  16. Farrell, E.H., Kiefer, J., Walbran, A.: Optimum Multivariate Designs. In: Le Cam, L.M., Neyman, J. (eds.) Proceedings of the Fifth Berkeley Symposium on Probability and Mathematical Statistics, vol. 1, pp. 113–138. University of California Press (1968)

  17. Fedorov, V.V.: Theory of Optimal Experiments. Academic press, New York (1972)

    Google Scholar 

  18. Huang, Y.C., Wong, W.K.: Multiple-objective optimal designs. J. Biopharm. Stat. 8, 635–643 (1998)

    Article  MATH  Google Scholar 

  19. Johnson, R.T., Montgomery, D.C., Jones, B.A.: An expository paper on optimal design. Qual. Eng. 23(3), 287–301 (2011)

    Article  Google Scholar 

  20. Kiefer, J.: Collected Papers: Design of Experiments. Springer, New York (1985)

    Book  MATH  Google Scholar 

  21. Kiefer, J., Wolfowitz, J.: The equivalence of two extremum problems. Can. J. Math. 12(3), 363–366 (1960)

    MathSciNet  Article  MATH  Google Scholar 

  22. Kiefer, J., Wolfowitz, J.: Optimum extrapolation and interpolation designs I. Ann. Inst. Stat. Math. 16(1), 79–108 (1964)

    MathSciNet  Article  MATH  Google Scholar 

  23. Kiefer, J., Wolfowitz, J.: Optimum extrapolation and interpolation designs II. Ann. Inst. Stat. Math. 16(1), 295–303 (1964)

    MathSciNet  Article  Google Scholar 

  24. Kiefer, J., Wolfowitz, J.: On a theorem of Hoel and Levine on extrapolation designs. Ann. Math. Stat. 36(6), 1627–1655 (1965)

    MathSciNet  Article  MATH  Google Scholar 

  25. King, J., Wong, W.K.: Optimal minimax designs for prediction in heteroscedastic models. J. Stat. Plan. Inference 69(2), 371–383 (1998)

    MathSciNet  Article  MATH  Google Scholar 

  26. King, J., Wong, W.K.: Minimax \(D\)-optimal designs for the logistic model. Biometrics 56(4), 1263–1267 (2000)

    MathSciNet  Article  MATH  Google Scholar 

  27. Levine, A.: A problem in minimax cariance polynomial extrapolation. Ann. Math. Stat. 37(4), 898–903 (1966)

    Google Scholar 

  28. Noubiap, R.F., Seidel, W.: A minimax algorithm for constructing optimal symmetrical balanced designs for a logistic regression model. J. Stat. Plan. Inference 91, 151–168 (2000)

    MathSciNet  Article  MATH  Google Scholar 

  29. Pázman, A.: Foundations of Optimum Experimental Design. Springer, New York (1986)

    MATH  Google Scholar 

  30. Poli, R., Kennedy, J., Blackwell, T.: Particle swarm optimization: an overview. Swarm Intell. 1, 33–57 (2007)

    Article  Google Scholar 

  31. Rodriguez, M., Jones, B., Borror, C.M., Montgomery, D.C.: Generating and assessing exact \(G\)-optimal designs. J. Qual. Technol. 42, 3–20 (2010)

    Google Scholar 

  32. Shi, Y., Eberhart, R.: A Modified Particle Swarm Optimizer. In: Evolutionary Computation Proceedings, 1998. The 1998 IEEE International Conference on, IEEE, IEEE World Congress on Computational Intelligence, pp. 69–73 (1998)

  33. Shi, Y., Eberhart, R.: Parameter selection in particle swarm optimization. In: Porto, V.W., Waagen, D. (eds.) Evolutionary Programming VII, pp. 591–600. Springer, New York (1998)

    Google Scholar 

  34. Shor, N.Z.: The Subgradient Method. Springer, New York (1985)

    Book  Google Scholar 

  35. Silvey, S.D.: Optimal Design. Chapman & Hall, London (1980)

    Book  MATH  Google Scholar 

  36. Sitter, R.R.: Robust designs. Biometrics 48, 1145–1155 (1992)

    MathSciNet  Article  Google Scholar 

  37. Spruill, M.C.: Optimal designs for minimax extrapolation. J. Multivar. Anal. 15(1), 52–62 (1984)

    MathSciNet  Article  MATH  Google Scholar 

  38. Spruill, M.C.: Optimal designs for multivariate interpolation. J. Multivar. Anal. 34(1), 141–155 (1990)

    MathSciNet  Article  MATH  Google Scholar 

  39. Whitacre, J.M.: Recent trends indicate rapid growth of nature-inspired optimization in academia and industry. Computing 93, 121–133 (2011a)

    MathSciNet  Article  MATH  Google Scholar 

  40. Whitacre, J.M.: Survival of the flexible: explaining the recent dominance of nature-inspired optimization within a rapidly evolving world. Computing 93, 135–146 (2011b)

    MathSciNet  Article  MATH  Google Scholar 

  41. Wong, W.K.: A unified approach to the construction of minimax designs. Biometrika 79(3), 611 (1992)

    MathSciNet  Article  MATH  Google Scholar 

  42. Wong, W.K., Cook, R.D.: Heteroscedastic \(G\)-optimal designs. J. R. Stat. Soc. Ser. B 55(4), 871–880 (1993)

    MathSciNet  MATH  Google Scholar 

  43. Yang, X.S.: Engineering Optimization: An Introduction with Metaheuristic Applications. Wiley, New York (2010)

    Book  Google Scholar 

  44. Zhu, W., Wong, W.K.: Dual-objective Bayesian optimal designs for a dose-ranging study. Drug Inf. J. 34, 421–428 (2000)

    Google Scholar 

  45. Zhu, W., Wong, W.K.: Bayesian optimal designs for estimating a set of symmetric quantiles. Stat. Med. 20, 123–137 (2001)

    Article  Google Scholar 

Download references

Acknowledgments

Weng Kee Wong worked on the manuscript when he was a visiting fellow at The Sir Isaac Newton Institute at Cambridge, England and a scientific advisor for a 6-month workshop in the design and analysis of experiment. He would like to thank Professor Rosemary Bailey for hosting the workshop and the Institute for the support during his repeated visits there in the second half of 2011. This research of Weng Kee Wong was also partially supported by NSC, Mathematics Research Promotion Center with grant number 102-51 and the Ministry of Education, Taiwan, R.O.C. The Aim for the Top University Project to the National Cheng Kung University (NCKU). The research of Ray-Bing Chen was partially supported by the National Science Council of Taiwan with grant number 101-2118-M-006-002-MY2 and the Mathematics Division of the National Center for Theoretical Sciences (South) in Taiwan. The research of Weichung Wang was partially supported by the National Science Council of Taiwan and the Taida Institute of Mathematical Sciences. We thank the editorial team for all the helpful comments and suggestions.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Weichung Wang.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Chen, RB., Chang, SP., Wang, W. et al. Minimax optimal designs via particle swarm optimization methods. Stat Comput 25, 975–988 (2015). https://doi.org/10.1007/s11222-014-9466-0

Download citation

Keywords

  • Continuous optimal design
  • Equivalence theorem
  • Fisher information matrix
  • Standardized maximin optimality criterion
  • Regression model