Application of Bayesian Optimization for Pharmaceutical Product Development

  • Syusuke SanoEmail author
  • Tadashi Kadowaki
  • Koji Tsuda
  • Susumu Kimura
Original Article



Bayesian optimization has been studied in many fields as a technique for global optimization of black-box functions. We applied these techniques for optimizing the formulation and manufacturing methods of pharmaceutical products to eliminate unnecessary experiments and accelerate method development tasks.


A simulation dataset was generated by the data augmentation from a design of experiment (DoE) which was executed to optimize the formulation and process parameters of orally disintegrating tablets. We defined a composite score for integrating multiple objective functions, physical properties of tablets, to meet the pharmaceutical criteria simultaneously. Performance measurements were used to compare the influence of the selection of initial training sets, by controlling data size and variation, acquisition functions, and schedules of hyperparameter tuning. Additionally, we investigated performance improvements obtained using Bayesian optimization techniques as opposed to random search strategy.


Bayesian optimization efficiently reduces the number of experiments to obtain the optimal formulation and process parameters from about 25 experiments with DoE to 10 experiments. Repeated hyperparameter tuning during the Bayesian optimization process stabilizes variations in performance among different optimization conditions, thus improving average performance.


We demonstrated the elimination of unnecessary experiments using Bayesian optimization. Simulations of different conditions depicted their dependencies, which will be useful in many real-world applications. Bayesian optimization is expected to reduce the reliance on individual skills and experiences, increasing the efficiency and efficacy of optimization tasks, expediting formulation and manufacturing research in pharmaceutical development.


Bayesian optimization Pharmaceutical product Design of experiment Artificial neural network Optimization 



Artificial neural network


Design of experiment


Tensile strength

TM replaces TS

Thompson sampling



  1. 1.
    Box GEP, Wilson KB. On the experimental attainment of optimum conditions Breakthroughs in statistics: methodology and distribution (2012): 270.Google Scholar
  2. 2.
    Sano S, Iwao Y, Kimura S, Itai S. Preparation and evaluation of swelling induced-orally disintegrating tablets by microwave irradiation. Int J Pharm. 2011;416:252–9.PubMedGoogle Scholar
  3. 3.
    Awotwe-Otoo D, Agarabi C, Wu GK, Casey E, Read E, Lute S, et al. Quality by design: iImpact of formulation variables and their interactions on quality attributes of a lyophilized monoclonal antibody. Int J Pharm. 2012;438:167–75.CrossRefPubMedGoogle Scholar
  4. 4.
    Takayama K, Fujikawa M, Obata Y, Morishita M. Neural network based optimization of drug formulations. Adv Drug Deliv Rev. 2003;55:1217–31.CrossRefPubMedGoogle Scholar
  5. 5.
    Wu T, Pan W, Chen J, Zhang R. Formulation optimization technique based on artificial neural network in salbutamol sulfate osmotic pump tablets. Drug Dev Ind Pharm. 2000;26:211–5.CrossRefPubMedGoogle Scholar
  6. 6.
    Li YF, Venkatasubramanian V. Neural network to understand process capability and process intermediates acceptance criteria in monoclonal antibody production process. J Pharm Innov. 2018;13:36–47.CrossRefGoogle Scholar
  7. 7.
    Arai H, Suzuki T, Kaseda C, Ohyama K, Takayama K. Bootstrap re-sampling technique to evaluate the optimal formulation of theophylline tablets predicted by non-linear response surface method incorporating multivariate spline interpolation. Chem Pharm Bull. 2007;55:586–93.CrossRefPubMedGoogle Scholar
  8. 8.
    Bergstra JJ, Yoshua Bengio Yoshuabengio U. Random search for hyper-parameter optimization. J Mach Learn Res. 2012;13:281–305.Google Scholar
  9. 9.
    Jones DR, Schonlau M, Welch WJ. Efficient global optimization of expensive black-box functions. J Glob Optim. 1998;13:455–92.CrossRefGoogle Scholar
  10. 10.
    Snoek J, Larochelle H, Adams RP. Practical Bayesian optimization of machine learning algorithms. Adv Neural Inf Proces Syst. 2012;25:2951–9.Google Scholar
  11. 11.
    Harold JK. A new method for locating the maximum point of an arbitrary multipeak curve in the presence of noise. J Basic Eng. 1964;86:07–106.Google Scholar
  12. 12.
    Mockus J, Tiesis V, Zilinskas A. The application of Bayesian methods for seeking the extremum. L. Dixon, G. Szego Eds. Towards Global Optimization; 1978.Google Scholar
  13. 13.
    Auer P. Using confidence bounds for exploitation-exploration trade-offs. J Mach Learn Res. 2003;3:397–422.Google Scholar
  14. 14.
    Zukerman I, Zukerman I, Albrecht DW, Albrecht DW, Zhou L, White JM, et al. Monte Carlo methods. Mach Learn. 2007;1:1–123.CrossRefGoogle Scholar
  15. 15.
    Srinivas N, Krause A, Kakade SM, Seeger M. Information-theoretic regret bounds for Gaussian process optimization in the bandit setting. IEEE Tr Inf Theo. 2012;58:3250–65.CrossRefGoogle Scholar
  16. 16.
    Chapelle O, Li L. An empirical evaluation of Thompson sampling. Adv Neural Inf Proces Syst. 2011;24:2249–57.Google Scholar
  17. 17.
    Boukouvala F, Ierapetritou MG. Feasibility analysis of black-box processes using an adaptive sampling Kriging-based method. Comput Chem Eng. 2012;36:358–68.CrossRefGoogle Scholar
  18. 18.
    Rogers A, Ierapetritou M. Feasibilityand flexibility analysis of black-box processes Part 1: Surrogate-based feasibility analysis. Chem Eng Sci. 2015;137:986–1004.CrossRefGoogle Scholar
  19. 19.
    Rogers A, Ierapetritou M. Feasibility and flexibility analysis of black-box processes Part 2: Surrogate-based feasibility analysis. Chem Eng Sci. 2015;137:1005–13.CrossRefGoogle Scholar
  20. 20.
    Wang Z, Ierapetritou M. A novel feasibility analysis method for black-box processes using a radial basis function adaptive sampling approach. J AIChE. 2017;63(2):532–50.CrossRefGoogle Scholar
  21. 21.
    Yoshinari T, Forbes RT, York P, Kawashima Y. Moisture induced polymorphic transition of mannitol and its morphological transformation. Int J Pharm. 2002;247:69–77.CrossRefPubMedGoogle Scholar
  22. 22.
    Yoshinari T, Forbes RT, York P, Kawashima Y. The improved compaction properties of mannitol after a moisture-induced polymorphic transition. Int J Pharm. 2003;258:121–31.CrossRefPubMedGoogle Scholar
  23. 23.
    Narazaki R, Harada T, Takami N, Kato Y, Ohwaki T. A new method for disintegration studies of rapid disintegrating tablet. Chem Pharm Bull. 2004;52:704–7.CrossRefPubMedGoogle Scholar
  24. 24.
    Harada T, Narazaki R, Nagira S, Ohwaki T, Aoki S, Iwamoto K. Evaluation of the disintegration properties of commercial famotidine 20 mg orally disintegrating tablets using a simple new test and human sensory test. Chem Pharm Bull. 2006;54:1072–5.CrossRefPubMedGoogle Scholar
  25. 25.
    Sano S, Iwao Y, Noguchi S, Kimura S, Itai S. Design and evaluation of microwave-treated orally disintegrating tablets containing polymeric disintegrant and mannitol. Int J Pharm. 2013;448:132–41.CrossRefPubMedGoogle Scholar
  26. 26.
    R Core Team. R: aA language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing; 2017. Google Scholar
  27. 27.
    Frauke G, Stefan F. Neuralnet: Training of neural networks. The R Journal 2010;2:30–38.Google Scholar
  28. 28.
    Ueno T, Rhone TD, Hou Z, Mizoguchi T, Tsuda K. COMBO: aAn efficient Bayesian optimization library for materials science. Mater Discov. 2016;4:18–21.CrossRefGoogle Scholar
  29. 29.
    Rasmussen CE, Williams CKI. Gaussian processes for machine learning. Cambridge, Mass: MIT Press; 2006.Google Scholar
  30. 30.
    Yang Z, Smola AJ, Song L, Wilson AG. A la Carte-Learning Fast Kernels, in: Proc 18th Int Conf Artif Intell Stat. 2015;1098–1106.Google Scholar
  31. 31.
    Kingma D, Ba J. Adam: A method for stochastic optimization. arXive:1412.6980. 20.Google Scholar
  32. 32.
    Kendal GP, Matthew GH. Determination of the tensile strength of elongated tablets. Powder Technol. 2013;238:169–75.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Pharmaceutical Science and Technologies Functional UnitMedicine Creation Center, Eisai Co, Ltd.KakamigaharaJapan
  2. 2.Data Science Laboratoryhhc Data Creation Center, Eisai Co, Ltd.TsukubaJapan
  3. 3.Department of Computational Biology and Medical Sciences, Graduate School of Frontier SciencesThe University of TokyoKashiwaJapan
  4. 4.Center for Advanced Intelligence ProjectRIKENTokyoJapan
  5. 5.Research and Services Division of Materials Data and Integrated SystemNational Institute for Materials ScienceTsukubaJapan

Personalised recommendations