The Stochastic Quasi-chemical Model for Bacterial Growth: Variational Bayesian Parameter Update

Abstract

We develop Bayesian methodologies for constructing and estimating a stochastic quasi-chemical model (QCM) for bacterial growth. The deterministic QCM, described as a nonlinear system of ODEs, is treated as a dynamical system with random parameters, and a variational approach is used to approximate their probability distributions and explore the propagation of uncertainty through the model. The approach consists of approximating the parameters’ posterior distribution by a probability measure chosen from a parametric family, through minimization of their Kullback–Leibler divergence.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Notes

  1. 1.

    https://github.com/pymc-devs/pymc.

References

  1. Banks, H., Bihari, K.: Modelling and estimating uncertainty in parameter estimation. Inverse Prob. 17, 95 (2001)

    MathSciNet  Article  MATH  Google Scholar 

  2. Banks, H., Browning, W., Catenacci, J., Wood, T.: Analysis of a Quasi-chemical Kinetic Food Chemistry Model. Center for Research in Scientific Computation Technical Report CRSC-TR16-05. NC State University, Raleigh, NC (2016)

  3. Baranyi, J., Roberts, T.: A dynamic approach to predicting bacterial growth in food. Int. J. Food Microbiol. 23, 277–294 (1994)

    Article  Google Scholar 

  4. Baranyi, J., Roberts, T., McClure, P.: A non-autonomous differential equation to model bacterial growth. Food Microbiol. 10, 43–59 (1993)

    Article  MATH  Google Scholar 

  5. Bickel, P., Doksum, K.: Mathematical Statistics: Basic Ideas and Selected Topics, vol. 2. CRC Press, Boca Raton (2015)

    Google Scholar 

  6. Bishop, C.: Pattern Recognition and Machine Learning, Information Science and Statistics. Springer, New York (2006)

    Google Scholar 

  7. Browning, W.J.: Near real-time quantification of stochastic model parameters. Tech. rep., prepared by Applied Mathematics Inc., Small Business Technology Transfer, Phase II Final Report, Army STTR Topic A13A-009 (28 September 2016)

  8. Buchanan, R.: Predictive microbiology: Mathematical modeling of microbial growth in foods. In: ACS Symposium Series-American Chemical Society, (1992)

  9. Buchanan, R., Whiting, R., Damert, W.: When is simple good enough: a comparison of the Gompertz. Baranyi, and three-phase linear models for fitting bacterial growth curves. Food Microbiol. 14, 313–326 (1997)

    Article  Google Scholar 

  10. Byrd, R., Lu, P., Nocedal, J., Zhu, C.: A limited memory algorithm for bound constrained optimization. SIAM J. Sci. Comput. 16, 1190–1208 (1995)

    MathSciNet  Article  MATH  Google Scholar 

  11. Chaloner, K., Verdinelli, I.: Bayesian experimental design: a review. Stat. Sci. 10(3), 273–304 (1995)

    MathSciNet  Article  MATH  Google Scholar 

  12. Chaspari, T., Tsiartas, A., Tsilifis, P., Narayanan, S.: Markov chain monte carlo inference of parametric dictionaries for sparse bayesian approximations. IEEE Trans. Signal Process. 64, 3077–3092 (2016)

    MathSciNet  Article  Google Scholar 

  13. Chen, P., Zabaras, N., Bilionis, I.: Uncertainty propagation using infinite mixture of gaussian processes and variational bayesian inference. J. Comput. Phys. 284, 291–333 (2015)

    MathSciNet  Article  MATH  Google Scholar 

  14. Doona, C., Feeherry, F., Ross, E.: A quasi-chemical model for the growth and death of microorganisms in foods by non-thermal and high-pressure processing. Int. J. Food Microbiol. 100, 21–32 (2005)

    Article  Google Scholar 

  15. Doona, C., Feeherry, F., Ross, E., Kustin, K.: Inactivation kinetics of listeria monocytogenes by highpressure processing: pressure and temperature variation. J. Food Sci. 77, M458–M465 (2012)

    Article  Google Scholar 

  16. Gershman, S., Hoffman, M., Blei, D.: Nonparametric variational inference. In: International Conference on Machine Learning (2012)

  17. Goldbeter, A.: Biochemical Oscillations and Cellular Rhythms: The Molecular Bases of Periodic and Chaotic Behaviour. Cambridge University Press, Cambridge (1997)

    Google Scholar 

  18. Gompertz, B.: On the nature of the function expressive of the law of human mortality and on a new mode of determining the value of life contingencies. Philos. Trans. R. Soc. Lond. 115, 513–583 (1825)

    Article  Google Scholar 

  19. Haario, H., Saksman, E., Tamminen, J.: An adaptive Metropolis algorithm. Bernoulli 7, 223–242 (2001)

    MathSciNet  Article  MATH  Google Scholar 

  20. Hastings, W.: Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57, 97–109 (1970)

    MathSciNet  Article  MATH  Google Scholar 

  21. Huber, M., Bailey, T., Durrant-Whyte, H., Hanebeck, U.: On entropy approximation for Gaussian mixture random vectors, In: IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems, MFI 2008, (pp. 181–188). IEEE, (2008)

  22. Kullback, S., Leibler, R.: On information and sufficiency. Ann. Math. Stat. 22, 79–86 (1951)

    MathSciNet  Article  MATH  Google Scholar 

  23. McMeekin, T., Brown, J., Krist, K., Miles, D., Neumeyer, K., Nichols, D., Olley, J., Presser, K., Ratkowsky, D., Ross, T., Salter, M.: Quantitative microbiology: a basis for food safety. Emerg. Infect. Dis. 3, 541 (1997)

    Article  Google Scholar 

  24. Metropolis, N., Rosenbluth, A., Rosenbluth, M., Teller, A., Teller, E.: Equations of state calculations by fast computing machines. J. Chem. Phys. 21, 1087–1091 (1953)

    Article  Google Scholar 

  25. Pinski, F., Simpson, G., Stuart, A., Weber, H.: Algorithms for Kullback–Leibler approximation of probability measures in infinite dimensions. SIAM J. Sci. Comput. 37, A2733–A2757 (2015)

    MathSciNet  Article  MATH  Google Scholar 

  26. Ricker, W.: Growth rates and models. Fish Physiol. 8, 677–743 (1979)

    Article  Google Scholar 

  27. Robert, C., Casella, G.: Monte Carlo Statistical Methods. Springer, Berlin (2013)

    Google Scholar 

  28. Roberts, G., Rosenthal, J.: Optimal scaling of discrete approximations to Langevin diffusions. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 60, 255–268 (1998)

    MathSciNet  Article  MATH  Google Scholar 

  29. Ross, E., Taub, I., Doona, C., Feeherry, F., Kustin, K.: The mathematical properties of the quasi-chemical model for microorganism growth-death kinetics in foods. Int. J. Food Microbiol. 99, 157–171 (2005)

    Article  Google Scholar 

  30. Schnute, J.: A versatile growth model with statistically stable parameters. Can. J. Fish. Aquat. Sci. 38, 1128–1140 (1981)

    Article  Google Scholar 

  31. Silverman, B.: Density estimation for statistics and data analysis, vol. 26. CRC Press, Boca Raton (1986)

    Google Scholar 

  32. Stuart, A.: Inverse problems: a bayesian perspective. Acta Numer. 19, 451–559 (2010)

    MathSciNet  Article  MATH  Google Scholar 

  33. Tarantola, A.: Inverse Problem Theory and Methods for Model Parameter Estimation. SIAM, Philadelphia (2005)

    Google Scholar 

  34. Taub, I., Ross, E., Feeherry, F.: Model for predicting the growth and death of pathogenic organisms. In: Van Impe, J.F.M., Gernaerts, K. (eds.) Proceedings of the Third International Conference on Predictive Modeling in Foods (2000)

  35. Taub, I., Feeherry, F., Ross, E., Kustin, K., Doona, C.: A quasi-chemical kinetics model for the growth and death of Staphylococcus aureus in intermediate moisture bread. J. Food Sci. 68, 2530–2537 (2003)

    Article  Google Scholar 

  36. Tsilifis, P., Bilionis, I., Katsounaros, I., Zabaras, N.: Computationally efficient variational approximations for bayesian inverse problems. J. Verif. Valid. Uncertain. Quantif. 1, 031004 (2016)

    Article  Google Scholar 

  37. Tsilifis, P., Ghanem, R., Hajali, P.: Efficient bayesian experimentation using an expected information gain lower bound. SIAM/ASA J. Uncertain. Quantif. 5, 30–62 (2017)

    MathSciNet  Article  MATH  Google Scholar 

  38. Vrettas, M., Cornford, D., Opper, M.: Estimating parameters in stochastic systems: a variational bayesian approach. Phys. D 240, 1877–1900 (2011)

    Article  MATH  Google Scholar 

  39. Whiting, R.: Modeling bacterial survival in unfavorable environments. J. Ind. Microbiol. 12, 240–246 (1993)

    Article  Google Scholar 

  40. Whiting, R., Sackitey, S., Calderone, S., Morely, K., Phillips, J.: Model for the survival of Staphylococcus aureus in nongrowth environments. Int. J. Food Microbiol. 31, 231–243 (1996)

    Article  Google Scholar 

  41. Ye, J., Rey, D., Kadakia, N., Eldridge, M., Morone, U., Rozdeba, P., Abarbanel, H., Quinn, J.: Systematic variational method for statistical nonlinear state and parameter estimation. Phys. Rev. E 92, 052901 (2015)

    MathSciNet  Article  Google Scholar 

  42. Zwietering, M., Jongenburger, I., Rombouts, F., Van’t Riet, K.: Modeling of the bacterial growth curve. Appl. Environ. Microbiol. 56, 1875–1881 (1990)

    Google Scholar 

Download references

Acknowledgements

The authors gratefully acknowledge support from US Army Research Office Contract W911NF-14-C-0151.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Paul K. Newton.

Additional information

Communicated by Charles R. Doering.

Appendix A: Gradient Computation of the Log-joint Distribution \(\log p(\mathbf {y}, \varvec{\theta })\)

Appendix A: Gradient Computation of the Log-joint Distribution \(\log p(\mathbf {y}, \varvec{\theta })\)

In order to perform gradient-based optimization of the ELBO approximation \(\mathcal {F}_2[q]\) with respect to \(\{\varvec{\mu }\}_i, \{\varvec{\Sigma }_i\}_i, i= 1,\dots , L\), we need to compute its gradient vector with entries

$$\begin{aligned} \frac{\partial }{\partial \zeta }\mathcal {F}_r[q] = \frac{\partial }{\partial \zeta }\mathcal {H}_0[q] + \frac{\partial }{\partial \zeta }\mathcal {L}_r[q] \end{aligned}$$
(A.1)

where \(\zeta = (\varvec{\mu }_i)_j, (\varvec{\Sigma }_i)_{jk}\), for \(i= 1,\dots ,L, j,k = 1,\dots , d\) and \(r = 0,2\). Below we provide the details in computing the gradient of \(\mathcal {L}_r[q], r=0,2\).

A.1: Gradient of \(\mathcal {L}_r[q]\)

For convenience, we set \(J(\varvec{\theta }) := \log p(\mathbf {y}, \varvec{\theta })\). Then for \(r=0\), the derivatives of \(\mathcal {L}_0[q]\) with respect to \(\zeta = (\varvec{\mu }_i)_j\) are

$$\begin{aligned} \frac{\partial }{\partial \zeta }\mathcal {L}_0[q] = \frac{1}{L}\frac{\partial }{\partial \theta _j}J(\varvec{\mu }_i) \end{aligned}$$
(A.2)

and with respect to \(\zeta = (\varvec{\Sigma })_{jk}\) are

$$\begin{aligned} \frac{\partial }{\partial \zeta }\mathcal {L}_0[q] = 0. \end{aligned}$$
(A.3)

For \(r = 2\) and \(\zeta = (\varvec{\Sigma }_i)_{jk}\), we get

$$\begin{aligned} \frac{\partial }{\partial \zeta }\mathcal {L}_2[q] = \frac{1}{2L} \frac{\partial ^2}{\partial \theta _j \partial \theta _k} J(\varvec{\mu }_i). \end{aligned}$$
(A.4)

As mentioned above, the derivatives of \(\mathcal {L}_2[q]\) with respect to \((\varvec{\mu }_i)_j\) are not used in our optimization scheme and therefore are not computed here.

A.2: Derivatives of \(J(\varvec{\theta })\)

First we rewrite \(\varvec{\theta }= (\varvec{\xi }, \omega )\) and expand

$$\begin{aligned} J(\varvec{\theta }) = J(\varvec{\xi }, \omega ) = \log p(\mathbf {y}| \mathcal {G}(\varvec{\xi }), \omega ) + \log p(\varvec{\xi }) + \log p(\omega ). \end{aligned}$$
(A.5)

Throughout our numerical examples, we work with an isotropic Gaussian likelihood (\(\varvec{\epsilon }\sim \mathcal {N}(\mathbf {0}, \sigma \mathbf {I}_N)\)); therefore, we set

$$\begin{aligned} L(\mathcal {G}(\varvec{\xi }),\omega ; \mathbf {y}) := \log p(\mathbf {y}|\mathcal {G}(\varvec{\xi }), \omega ) = \log \mathcal {N}(\mathbf {y}| \mathcal {G}(\varvec{\xi }), e^{2\omega }\mathbf {I}_N), \end{aligned}$$
(A.6)

and using the chain rule, we have

$$\begin{aligned} \frac{\partial J}{\partial \xi _j}= & {} \sum _{s=1}^N\frac{\partial L}{\partial \mathcal {G}_s} \frac{\partial \mathcal {G}_s}{\partial \xi _j} + \frac{1}{p(\varvec{\xi })}\frac{\partial p(\varvec{\xi })}{\partial \xi _j} \end{aligned}$$
(A.7)
$$\begin{aligned} \frac{\partial J}{\partial \omega }= & {} \frac{\partial L}{\partial \omega } + \frac{1}{p(\omega )}\frac{d p(\omega )}{d\omega }\end{aligned}$$
(A.8)
$$\begin{aligned} \frac{\partial ^2 J}{\partial \xi _j\partial \xi _k}= & {} \sum _{s,t = 1}^N\frac{\partial ^2 L}{\partial \mathcal {G}_s \partial \mathcal {G}_t}\frac{\partial \mathcal {G}_s}{\partial \xi _j}\frac{\partial \mathcal {G}_t}{\partial \xi _k} + \sum _{s=1}^N\frac{\partial L}{\partial \mathcal {G}_s}\frac{\partial ^2 \mathcal {G}_s}{\partial \xi _j\partial \xi _k} \nonumber \\&+ \frac{1}{p(\varvec{\xi })}\frac{\partial ^2 p(\varvec{\xi })}{\partial \xi _j\partial \xi _k} -\frac{1}{p(\varvec{\xi })^2}\frac{\partial p(\varvec{\xi })}{\partial \xi _j}\frac{\partial p(\varvec{\xi })}{\partial \xi _k}\end{aligned}$$
(A.9)
$$\begin{aligned} \frac{\partial ^2 J}{\partial \omega ^2}= & {} \frac{\partial ^2 L}{\partial \omega ^2} + \frac{1}{p(\omega )}\frac{d^2 p(\omega )}{d\omega ^2} -\frac{1}{p(\omega )^2}\frac{d p(\omega )}{d\omega }\end{aligned}$$
(A.10)
$$\begin{aligned} \frac{\partial ^2 J}{\partial \xi _j\partial \omega }= & {} \sum _{s=1}^N\frac{\partial ^2 L}{\partial \mathcal {G}_s \partial \omega }\frac{\partial \mathcal {G}_s}{\partial \xi _j} \end{aligned}$$
(A.11)

In the above expressions, it becomes clear that the Jacobian and Hessian of the forward model \(\mathcal {G}(\varvec{\xi })\) need to be computed. As mentioned in our application, the covariance matrices of the Gaussian mixtures components are taken to be diagonal which implies that only the diagonal elements of the Hessian of \(\mathcal {G}(\varvec{\xi })\) are necessary.

A.3: Log-Likelihood Derivatives

The derivatives of the log-likelihood function required for the expression in the previous subsection are given as follows:

$$\begin{aligned} \frac{\partial L}{\partial \mathcal {G}_s}= & {} e^{-2\omega }(y_s - \mathcal {G}_s(\varvec{\xi })) \end{aligned}$$
(A.12)
$$\begin{aligned} \frac{\partial L}{\partial \omega }= & {} e^{-\omega }\left( ||\mathbf {y}- \mathcal {G}(\varvec{\xi })||_2^2e^{-2\theta } - k+1\right) \end{aligned}$$
(A.13)
$$\begin{aligned} \frac{\partial ^2 L}{\partial \omega ^2}= & {} e^{-\omega }\left( k-1 - 3||\mathbf {y}- \mathcal {G}(\varvec{\xi })||_2^2e^{-2\omega }\right) \end{aligned}$$
(A.14)
$$\begin{aligned} \frac{\partial ^2 L}{\partial \mathcal {G}_s \partial \mathcal {G}_t}= & {} -e^{-2\omega } \end{aligned}$$
(A.15)
$$\begin{aligned} \frac{\partial ^2 L}{\partial \mathcal {G}_s \partial \omega }= & {} -2 e^{-3\omega }\left( y_s - \mathcal {G}_s(\varvec{\xi })\right) . \end{aligned}$$
(A.16)

A.4: Derivatives of the Quasi-chemical Model

For the sake of generality and due to the presence of a nonlinear term in the quasi-chemical model, we present the general derivation of the system of ODEs satisfied by the derivatives of a solution \(\mathbf {u}(t; \varvec{\xi })\) of the QCM with respect to its parameters. Assume \(\mathbf {u}(t;\varvec{\xi })\) satisfies

$$\begin{aligned} \dot{\mathbf {u}}= & {} \mathbf {g}(\mathbf {u}, t ;\varvec{\xi }) \end{aligned}$$
(A.17)
$$\begin{aligned} \mathbf {u}(0)= & {} \mathbf {u}_0, \end{aligned}$$
(A.18)

where \(\varvec{\xi }\in \mathbb {R}^4\) are parameters and the initial condition is fixed and independent of \(\varvec{\xi }\). By simply differentiating the above system of equations, one can derive the following initial value problem satisfied by \(v_{ij} = \partial u_i/\partial \xi _j\):

$$\begin{aligned} \dot{v}_{ij}= & {} \sum _{s= 1}^4 \frac{\partial g_i}{\partial u_s} v_{sj} + \frac{\partial g_i}{\partial \xi _j} \end{aligned}$$
(A.19)
$$\begin{aligned} v_{ij}(0)= & {} 0. \end{aligned}$$
(A.20)

Similarly, for the second derivatives \(w_{ijk} = \partial ^2 u_i / (\partial \xi _j\partial \xi _k)\) we get

$$\begin{aligned} \dot{w}_{ijk}= & {} \sum _{s = 1}^4\frac{\partial g_i}{\partial u_s} w_{ijk} + \sum _{s, t= 1}^4\frac{\partial ^2 g}{\partial u_s\partial u_t} v_{sj}v_{tk} + \frac{\partial ^2 g_i}{\partial \xi _j\partial \xi _k} \end{aligned}$$
(A.21)
$$\begin{aligned} w_{ijk}= & {} 0. \end{aligned}$$
(A.22)

In practice, during numerical implementation one need to first solve (A.17) and then solve (A.19) using the solution of the former as forcing. At last, (A.21) can be solved by using both the QCM solution and its gradient as forcing.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Tsilifis, P., Browning, W.J., Wood, T.E. et al. The Stochastic Quasi-chemical Model for Bacterial Growth: Variational Bayesian Parameter Update. J Nonlinear Sci 28, 371–393 (2018). https://doi.org/10.1007/s00332-017-9411-4

Download citation

Keywords

  • Bayes rule
  • Kullback–Leibler divergence
  • Evidence lower bound
  • Quasi-chemical model
  • Gradient-based optimization