Skip to main content
SpringerLink
Log in

Deep Learning of Biological Models from Data: Applications to ODE Models

  • Original Article
  • Published:
Bulletin of Mathematical Biology Aims and scope Submit manuscript

Abstract

Mathematical equations are often used to model biological processes. However, for many systems, determining analytically the underlying equations is highly challenging due to the complexity and unknown factors involved in the biological processes. In this work, we present a numerical procedure to discover dynamical physical laws behind biological data. The method utilizes deep learning methods based on neural networks, particularly residual networks. It is also based on recently developed mathematical tools of flow-map learning for dynamical systems. We demonstrate that with the proposed method, one can accurately construct numerical biological models for unknown governing equations behind measurement data. Moreover, the deep learning model can also incorporate unknown parameters in the biological process. A successfully trained deep neural network model can then be used as a predictive tool to produce system predictions of different settings and allows one to conduct detailed analysis of the underlying biological process. In this paper, we use three biological models—SEIR model, Morris–Lecar model and the Hodgkin–Huxley model—to show the capability of our proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  • Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, Corrado GS, Davis A, Dean J, Devin M, Ghemawat S, Goodfellow I, Harp A, Irving G, Isard M, Jia Y, Jozefowicz R, Kaiser L, Kudlur M, Levenberg J, Mané D, Monga R, Moore S, Murray D, Olah C, Schuster M, Shlens J, Steiner B, Sutskever I, Talwar K, Tucker P, Vanhoucke V, Vasudevan V, Viégas F, Vinyals O, Warden P, Wattenberg M, Wicke M, Yu Y, Zheng X (2015) TensorFlow: large-scale machine learning on heterogeneous systems. https://www.tensorflow.org/. Software available from tensorflow.org

  • Bianchini M, Scarselli F (2014) On the complexity of neural network classifiers: a comparison between shallow and deep architectures. IEEE Trans Neural Netw Learn Syst 25:1553–1565

    Article  Google Scholar 

  • Bongard J, Lipson H (2007) Automated reverse engineering of nonlinear dynamical systems. Proc Natl Acad Sci 104:9943–9948

    Article  Google Scholar 

  • Chan S, Elsheikh A (2018) A machine learning approach for efficient uncertainty quantification using multiscale methods. J Comput Phys 354:494–511

    Article  MathSciNet  Google Scholar 

  • Chen RTQ, Rubanova Y, Bettencourt J, Duvenaud D (2018) Neural ordinary differential equations. Preprint arXiv:1806.07366

  • Chollet F et al (2015) Keras. https://github.com/fchollet/keras

  • Daniels BC, Nemenman I (2015a) Automated adaptive inference of phenomenological dynamical models. Nat Commun 6:8133

    Article  Google Scholar 

  • Daniels BC, Nemenman I (2015b) Efficient inference of parsimonious phenomenological models of cellular dynamics using S-systems and alternating regression. PLoS ONE 10:e0119821

    Article  Google Scholar 

  • DeAngelis DL, Yurek S (2015) Equation-free modeling unravels the behavior of complex ecological systems. https://doi.org/10.1073/pnas.1503154112

  • Du KL, Swamy M (2014) Neural networks and statistical learning. Springer, London

    Book  Google Scholar 

  • Eldan R, Shamir O (2016) The power of depth for feedforward neural networks. Conf Learn Theory 2016:907–940

    Google Scholar 

  • Fu X, Chang LB, Xiu D (2020) Learning reduced systems via deep neural networks with memory. J Mach Learn Model Comput 2020:1

    Google Scholar 

  • Giannakis D, Majda AJ (2012) Nonlinear Laplacian spectral analysis for time series with intermittency and low-frequency variability. Proc Natl Acad Sci 109:2222–2227

    Article  MathSciNet  Google Scholar 

  • Gonzalez-Garcia R, Rico-Martinez R, Kevrekidis IG (1998) Identification of distributed parameter systems: a neural net based approach. Comput Chem Eng 22:S965–S968

    Article  Google Scholar 

  • Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press, London

    MATH  Google Scholar 

  • Grimm V, Railsback SF (2005) Individual-based modeling and ecology. Princeton University Press, Princeton

    Book  Google Scholar 

  • Hau DT, Coiera EW (1995) Learning qualitative models from physiological signals internal accession date only. Technical report

  • Hesthaven J, Ubbiali S (2018) Non-intrusive reduced order modeling of nonlinear problems using neural networks. J Comput Phys 363:55–78

    Article  MathSciNet  Google Scholar 

  • Hethcote HW (2000) The mathematics of infectious diseases. SIAM Rev 42:599–653

    Article  MathSciNet  Google Scholar 

  • He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  • Hodgkin AL, Huxley AF (Aug 1952) A quantitative description of membrane current and its application to conduction and excitation in nerve. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1392413/

  • Kevrekidis IG, Gear CW, Hyman JM, Kevrekidid PG, Runborg O, Theodoropoulos C et al (2003) Equation-free, coarse-grained multiscale computation: enabling mocroscopic simulators to perform system-level analysis. Commun Math Sci 1:715–762

    Article  MathSciNet  Google Scholar 

  • Khoo Y, Lu J, Ying L (2018) Solving parametric PDE problems with artificial neural networks. Preprint arXiv:1707.03351

  • Kingma DP, Ba J (2017) Adam: a method for stochastic optimization. arXiv:1412.6980

  • Long Z, Lu Y, Ma X, Dong B (2018) PDE-net: learning PDEs from data. In: Proceedings of the 35th international conference on machine learning, 10–15 edn. vol 80 of proceedings of machine learning research, Stockholmsmässan, Stockholm, Sweden, pp 3208–3216

  • Mangan NM, Brunton SL, Proctor JL, Kutz JN (2016) Inferring biological networks by sparse identification of nonlinear dynamics. IEEE Trans Mol Biol Multiscale Commun 2:52–63

    Article  Google Scholar 

  • Mardt A, Pasquali L, Wu H, Noe F (2018) VAMPnets for deep learning of molecular kinetics. Nat Commun 9:5

    Article  Google Scholar 

  • Montufar GF, Pascanu R, Pascanu K, Bengio Y (2014) On the number of linear regions of deep neural networks. Adv Neural Inf Process Syst 2014:2924–2932

    Google Scholar 

  • Morris C, Lecar H (1981) Voltage oscillations in the barnacle giant muscle fiber. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1327511/

  • Perretti CT, Munch SB, Sugihara G (2013) Model-free forecasting outperforms the correct mechanistic model for simulated and experimental data. Proc Natl Acad Sci USA 110:5253–5257. https://doi.org/10.1073/pnas.1216076110

    Article  Google Scholar 

  • Poggio T, Mhaskar H, Rosasco L, Miranda B, Liao Q (2017) Why and when can deep-but not shallow-networks avoid the curse of dimensionality: a review. Int J Autom Comput 14:503–519

    Article  Google Scholar 

  • Qin T, Wu K, Xiu D (2019) Data driven governing equations approximation using deep neural networks. J Comput Phys 395:620–635

    Article  MathSciNet  Google Scholar 

  • Qin T, Chen Z, Jakeman J, Xiu D (2020) Data-driven learning of non-autonomous systems. SIAM J Sci Comput 2020:1

    Google Scholar 

  • Radhakrishnan K, Hindmarsh AC (1993) Description and use of LSODE, the Livemore solver for ordinary differential equations. Technical report, Lawrence Livermore National Laboratory (LLNL), Livermore. https://doi.org/10.2172/15013302. http://www.osti.gov/servlets/purl/15013302-7Xcq5X/native/

  • Ray D, Hesthaven J (2018) An artificial neural network as a troubled-cell indicator. J Comput Phys 367:166–191

    Article  MathSciNet  Google Scholar 

  • Rinzel J, Ermentrout G (1998) Analysis of neural excitability and oscillations. In: Koch C, Segev I (eds) Methods in neuronal modeling, 2nd edn. MIT Press, London, pp 251–291

    Google Scholar 

  • Schmidhuber J (2015) Deep learning in neural networks: an overview. Neural Netw 61:85–117

    Article  Google Scholar 

  • Schmidt M, Lipson H (2009) Distilling free-form natural laws from experimental data. Science 324:81–85

    Article  Google Scholar 

  • Schmidt MD, Vallabhajosyula RR, Jenkins JW, Hood JE, Soni AS, Wikswo JP, Lipson H (2011) Automated refinement and inference of analytical models for metabolic networks. Phys Biol 8:055011

    Article  Google Scholar 

  • Sugihara G, May R, Ye H, Hsieh C, Deyle E, Fogarty M, Munch S (2012) Detecting causality in complex ecosystems. Science 338:496–500

    Article  Google Scholar 

  • Tripathy R, Bilionis I (2018) Deep UQ: learning deep neural network surrogate model for high dimensional uncertainty quantification. J Comput Phys 375:565–588

    Article  MathSciNet  Google Scholar 

  • Tsumoto K, Kitajima H, Yoshinaga T, Aihara K, Kawakami H (2006) Bifurcations in Morris–Lecar neuron model. Neurocomputing 69:293–316. https://doi.org/10.1016/j.neucom.2005.03.006

    Article  Google Scholar 

  • Voss HU, Kolodner P, Abel M, Kurths J (1999) Amplitude equations from spatiotemporal binary-fluid convection data. Phys Rev Lett 83:3422

    Article  Google Scholar 

  • Wang Y, Cheung SW, Chung ET, Efendiev Y, Wang M (2018) Deep multiscale model learning. Preprint arXiv:1806.04830

  • Weinan E, Engquist B, Huang Z (2003) Heterogeneous multiscale method: a general methodology for multiscale modeling. Phys. Rev. B 67:092101

    Article  Google Scholar 

  • Wood SN, Thomas MB (1999) Super-sensitivity to structure in biological models. Proc R Soc B Biol Sci 266:565–570. https://doi.org/10.1098/rspb.1999.0673

    Article  Google Scholar 

  • Wu K, Xiu D (2019) Numerical aspects for approximating governing equations using data. J Comput Phys 384:200–221

    Article  MathSciNet  Google Scholar 

  • Ye H, Beamish RJ, Glaser SM, Grant SCH, Hsieh C, Richards LJ, Schnute JT, Sugihara G (2015) Equation-free mechanistic ecosystem forecasting using empirical dynamic modeling. Proc Natl Acad Sci 112:1569–E1576

    Article  Google Scholar 

  • Yodzis P (1988) The indeterminacy of ecological interactions as perceived through perturbation. Technical report, p 2

  • Zhu Y, Zabaras N (2018) Bayesian deep convolutional encoder–decoder networks for surrogate modeling and uncertainty quantification. J Comput Phys 366:415–447

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, The Ohio State University, Columbus, OH, 43221, USA

    Wei-Hung Su, Ching-Shan Chou & Dongbin Xiu

Authors
  1. Wei-Hung Su
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ching-Shan Chou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dongbin Xiu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dongbin Xiu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Su, WH., Chou, CS. & Xiu, D. Deep Learning of Biological Models from Data: Applications to ODE Models. Bull Math Biol 83, 19 (2021). https://doi.org/10.1007/s11538-020-00851-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11538-020-00851-7

Keywords

Search

Navigation