Neural Gray-Box Identification of Nonlinear Partial Differential Equations

  • Riku SasakiEmail author
  • Naoya Takeishi
  • Takehisa Yairi
  • Koichi Hori
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11671)


Many branches of the modern computational science and engineering are based on numerical simulations, for which we must prepare appropriate equations that well reflect the behavior of real-world phenomena and numerically solve them. For these purposes, we may utilize the data-driven identification and simulation technique of nonlinear partial differential equations (NPDEs) using deep neural networks (DNNs). A potential issue of the DNN-based identification and simulation in practice is the high variance due to the complexity of DNNs. To alleviate it, we propose a simple yet efficient way to incorporate prior knowledge of phenomena. Specifically, we can often anticipate what kinds of terms are present in a part of an appropriate NPDE, which should be utilized as prior knowledge for identifying the remaining part of the NPDE. To this end, we design DNN’s inputs and the loss function for identification according to the prior knowledge. We present the results of the experiments conducted using three different types of NPDEs: the Korteweg–de Vries equation, the Navier–Stokes equation, and the Kuramoto–Sivashinsky equation. The experimental results show the effectiveness of the proposed method, i.e., utilizing known terms of an NPDE.


Gray-box system identification Nonlinear PDEs Data-driven discovery Machine learning 



This work was supported by JSPS KAKENHI Grant Numbers JP18H06487, JP19K21550 and JP19K12094.


  1. Anderson, J.S., Kevrekidisi, I.G., Rico-Martínez, R.: A comparison of recurrent training algorithms for time series analysis and system identification. Comput. Chem. Eng. 20, S751–S756 (1996)CrossRefGoogle Scholar
  2. Bongard, J., Lipson, H.: Automated reverse engineering of nonlinear dynamical systems. Proc. Nat. Acad. Sci. U.S.A. 104(24), 9943–9948 (2007)CrossRefGoogle Scholar
  3. Brunton, S.L., Proctor, J.L., Kutz, J.N.: Discovering governing equations from data: sparse identification of nonlinear dynamical systems. Proc. Nat. Acad. Sci. U.S.A. 113(15), 3932–3937 (2015)MathSciNetCrossRefGoogle Scholar
  4. Crutchfield, J.P., McNamara, B.: Equations of motion from a data series. Complex Syst. 1, 417–452 (1987)MathSciNetzbMATHGoogle Scholar
  5. Daniels, B.C., Nemenman, I.: Automated adaptive inference of phenomenological dynamical models. Nat. Commun. 6, 8133 (2015a)CrossRefGoogle Scholar
  6. Daniels, B.C., Nemenman, I.: Efficient inference of parsimonious phenomenological models of cellular dynamics using S-systems and alternating regression. PLoS ONE 10(3), e0119821 (2015b)CrossRefGoogle Scholar
  7. Driscoll, T.A., Hale, N., Trefethen, L.N. (eds.): Chebfun Guide. Pafnuty Publications, Oxford (2014)Google Scholar
  8. González-García, R., Rico-Martínez, R., Kevrekidis, I.G.: Identification of distributed parameter systems: a neural net based approach. Comput. Chem. Eng. 22, S965–S968 (1998)CrossRefGoogle Scholar
  9. Hastie, T., Tibshirani, R., Wainwright, M.: Statistical Learning with Sparsity: The Lasso and Generalizations. CRC Press, Boca Raton (2018)zbMATHGoogle Scholar
  10. Kevrekidis, I.G., Gear, C.W., Hyman, J.M., Kevrekidis, P.G., Runborg, O., Theodoropoulos, C.: Equation-free multiscale computation: enabling microscopic simulators to perform system-level tasks. Commun. Math. Sci. 1(4), 715–762 (2003)MathSciNetCrossRefGoogle Scholar
  11. Mangan, N.M., Brunton, S.L., Proctor, J.L., Kutz, J.N.: Inferring biological networks by sparse identification of nonlinear dynamics. IEEE Trans. Mol. Biol. Multi-Scale Commun. 2(1), 52–63 (2016)CrossRefGoogle Scholar
  12. Raissi, M., Perdikaris, P., Karniadakis, G.: Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys. 378, 686–707 (2019)MathSciNetCrossRefGoogle Scholar
  13. Raissi, M.: Deep hidden physics models: deep learning of nonlinear partial differential equations. J. Mach. Learn. Res. 19(25), 1–24 (2018)MathSciNetzbMATHGoogle Scholar
  14. Rico-Martínes, R., Kevrekidis, I.G., Kube, M.C., Hudson, J.L.: Discrete- vs. continuous-time nonlinear signal processing: attractors, transitions and parallel implementation issues. Chem. Eng. Commun. 118(1), 25–48 (1992)CrossRefGoogle Scholar
  15. Roberts, A.J.: Model Emergent Dynamics in Complex Systems. SIAM (2015)Google Scholar
  16. Rudy, S.H., Brunton, S.L., Proctor, J.L., Kutz, J.N.: Data-driven discovery of partial differential equations. Sci. Adv. 3(4), e1602614 (2017)CrossRefGoogle Scholar
  17. Schaeffer, H., Osher, S., Caflisch, R., Hauck, C.: Sparse dynamics for partial differential equations. Proc. Nat. Acad. Sci. U.S.A. 110(17), 6634–6639 (2013)MathSciNetCrossRefGoogle Scholar
  18. Schmidt, M., Lipson, H.: Distilling free-form natural laws from experimental data. Science 324(5923), 81–85 (2012)CrossRefGoogle Scholar
  19. Schmidt, M.D., et al.: Automated refinement and inference of analytical models for metabolic networks. Phys. Biol. 8(5), 055011 (2011)CrossRefGoogle Scholar
  20. Strang, G.: Computational Science and Engineering. Wellesley-Cambridge Press, Wellesley (2007)zbMATHGoogle Scholar
  21. Sugihara, G., et al.: Detecting causality in complex ecosystems. Science 338(6106), 496–500 (2012)CrossRefGoogle Scholar
  22. Tran, G., Ward, R.: Exact recovery of chaotic systems from highly corrupted data. Multiscale Model. Simul. 15(3), 1108–1129 (2017)MathSciNetCrossRefGoogle Scholar
  23. Ye, H., et al.: Equation-free mechanistic ecosystem forecasting using empirical dynamic modeling. Proc. Nat. Acad. Sci. U.S.A. 112(13), E1569–E1576 (2015)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Riku Sasaki
    • 1
    Email author
  • Naoya Takeishi
    • 2
  • Takehisa Yairi
    • 1
  • Koichi Hori
    • 1
  1. 1.The University of TokyoTokyoJapan
  2. 2.RIKEN Center for Advanced Intelligence ProjectTokyoJapan

Personalised recommendations