Skip to main content
SpringerLink
Log in

Parameter estimation and modeling of nonlinear dynamical systems based on Runge–Kutta physics-informed neural network

  • Original Paper
  • Published:
Nonlinear Dynamics Aims and scope Submit manuscript

Abstract

To identify the nonlinear behavior of dynamical systems subjected to external excitations such as earthquakes, explosions, and impacts, a novel method is proposed for dynamical system parameter estimation and modeling based on the combination of the physics-informed neural network (PINN) and Runge–Kutta algorithm. Drawing inspiration from classical numerical integration solution rules for differential equations, a new recurrent neural network architecture is designed for modeling. PINN cells are embedded as the basic integration units of the architecture to introduce prior physics-based biases. And the motion equations of different dynamical systems can be flexibly added to this architecture as soft constraints for neural network training. The method, referred to as the Runge–Kutta physics-informed neural network (RK-PINN), differs from black-box learning, as it models the evolutionary process of nonlinear dynamical systems. The satisfactory parameter estimation capability of the RK-PINN method was demonstrated through two illustrative examples. The results indicate that embedding physics can reduce the data required for training, and the trained network can serve as a surrogate model for the dynamical system to predict future states or responses.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Data availability

The datasets generated or analyzed during the current study are available on GitHub at https://github.com/VVeida/RK4_PINN or from the corresponding author on reasonable request.

References

  1. Wanner, G., Hairer, E.: Solving Ordinary Differential Equations II, vol. 375. Springer, Berlin Heidelberg New York (1996)

  2. Zhang, L., Sun, Y., Wang, A., Zhang, J.: Neural network modeling and dynamic behavior prediction of nonlinear dynamic systems. Nonlinear Dyn. 1–22 (2023)

  3. Runge, C.: Über die numerische auflösung von differentialgleichungen. Math. Ann. 46(2), 167–178 (1895)

    Article  MathSciNet  MATH  Google Scholar 

  4. Kutta, W.: Beitrag zur näherungsweisen Integration totaler Differentialgleichungen. Teubner (1901)

  5. Dormand, J.R., Prince, P.J.: A family of embedded runge-kutta formulae. J. Comput. Appl. Math. 6(1), 19–26 (1980)

    Article  MathSciNet  MATH  Google Scholar 

  6. Aniszewska, D.: Multiplicative runge-kutta methods. Nonlinear Dyn. 50(1–2), 265–272 (2007)

    Article  MATH  Google Scholar 

  7. Schober, M., Duvenaud, D.K., Hennig, P.: Probabilistic ode solvers with runge-kutta means. Adv. Neural Inf. Process. Syst. 27 (2014)

  8. Hornik, K., Stinchcombe, M., White, H.: Multilayer feedforward networks are universal approximators. Neural Netw. 2(5), 359–366 (1989)

    Article  MATH  Google Scholar 

  9. Weinan, E.: A proposal on machine learning via dynamical systems. Commun. Math. Stat. 1(5), 1–11 (2017)

    MathSciNet  MATH  Google Scholar 

  10. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  11. Lu, Y., Zhong, A., Li, Q., Dong, B.: Beyond finite layer neural networks: bridging deep architectures and numerical differential equations. In: International Conference on Machine Learning, pp. 3276–3285. PMLR (2018)

  12. Wang, Y.J., Lin, C.T.: Runge-kutta neural network for identification of dynamical systems in high accuracy. IEEE Trans. Neural Netw. 9(2), 294–307 (1998)

    Article  MathSciNet  Google Scholar 

  13. Zhu, M., Chang, B., Fu, C.: Convolutional neural networks combined with runge-kutta methods. Neural Comput. Appl. 35(2), 1629–1643 (2023)

    Article  Google Scholar 

  14. Chen, R.T., Rubanova, Y., Bettencourt, J., Duvenaud, D.K.: Neural ordinary differential equations. Adv. Neural Inf. Process. Syst. 31 (2018)

  15. Goyal, P., Benner, P.: Discovery of nonlinear dynamical systems using a runge-kutta inspired dictionary-based sparse regression approach. Proc. R. Soc. A 478(2262), 20210883 (2022)

    Article  MathSciNet  Google Scholar 

  16. Foroutannia, A., Ghasemi, M.: Predicting cortical oscillations with bidirectional lstm network: a simulation study. Nonlinear Dyn. 111(9), 8713–8736 (2023)

    Article  Google Scholar 

  17. Liu, H., Zhao, C., Huang, X., Yao, G.: Data-driven modeling for the dynamic behavior of nonlinear vibratory systems. Nonlinear Dyn. 1–26 (2023)

  18. Tan, Y., Hu, C., Zhang, K., Zheng, K., Davis, E.A., Park, J.S.: Lstm-based anomaly detection for non-linear dynamical system. IEEE Access 8, 103301–103308 (2020)

    Article  Google Scholar 

  19. Salmela, L., Tsipinakis, N., Foi, A., Billet, C., Dudley, J.M., Genty, G.: Predicting ultrafast nonlinear dynamics in fibre optics with a recurrent neural network. Nat. Mach. Intell. 3(4), 344–354 (2021)

    Article  Google Scholar 

  20. Li, S., Yang, Y.: A recurrent neural network framework with an adaptive training strategy for long-time predictive modeling of nonlinear dynamical systems. J. Sound Vib. 506, 116167 (2021)

    Article  Google Scholar 

  21. Alemany, S., Beltran, J., Perez, A., Ganzfried, S.: Predicting hurricane trajectories using a recurrent neural network. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 468–475 (2019)

  22. Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back-propagating errors. Nature 323(6088), 533–536 (1986)

    Article  MATH  Google Scholar 

  23. Kim, Y.H., Lewis, F.L., Abdallah, C.T.: A dynamic recurrent neural-network-based adaptive observer for a class of nonlinear systems. Automatica 33(8), 1539–1543 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  24. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  25. Chen, R., Jin, X., Laima, S., Huang, Y., Li, H.: Intelligent modeling of nonlinear dynamical systems by machine learning. Int. J. Non-Linear Mech. 142, 103984 (2022)

    Article  Google Scholar 

  26. Wang, R., Kalnay, E., Balachandran, B.: Neural machine-based forecasting of chaotic dynamics. Nonlinear Dyn. 98(4), 2903–2917 (2019)

    Article  MATH  Google Scholar 

  27. Kalman, R.E.: A new approach to linear filtering and prediction problems (1960)

  28. Jazwinski, A.H.: Stochastic processes and filtering theory. Courier Corporation (2007)

  29. Wan, E.A., Van Der Merwe, R.: The unscented kalman filter for nonlinear estimation. In: Proceedings of the IEEE 2000 Adaptive Systems for Signal Processing, Communications, and Control Symposium (Cat. No. 00EX373), pp. 153–158. IEEE (2000)

  30. Bisaillon, P., Sandhu, R., Khalil, M., Pettit, C., Poirel, D., Sarkar, A.: Bayesian parameter estimation and model selection for strongly nonlinear dynamical systems. Nonlinear Dyn. 82, 1061–1080 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  31. Lindley, D.: The use of prior probability distributions in statistical inference and decisions. In: Proc. 4th Berkeley Symp. on Math. Stat. and Prob, pp. 453–468 (1960)

  32. Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L., Lerer, A.: Automatic differentiation in pytorch (2017)

  33. Karniadakis, G.E., Kevrekidis, I.G., Lu, L., Perdikaris, P., Wang, S., Yang, L.: Physics-informed machine learning. Nat. Rev. Phys. 3(6), 422–440 (2021)

    Article  Google Scholar 

  34. Raissi, M.: Deep hidden physics models: deep learning of nonlinear partial differential equations. J. Mach. Learn. Res. 19(1), 932–955 (2018)

    MathSciNet  MATH  Google Scholar 

  35. Raissi, M., Perdikaris, P., Karniadakis, G.E.: Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys. 378, 686–707 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  36. Zhang, R., Liu, Y., Sun, H.: Physics-informed multi-lstm networks for metamodeling of nonlinear structures. Comput. Methods Appl. Mech. Eng. 369, 113226 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  37. Li, S., Yang, Y.: Data-driven identification of nonlinear normal modes via physics-integrated deep learning. Nonlinear Dyn. 106, 3231–3246 (2021)

    Article  Google Scholar 

  38. Agarwal, V., Wang, R., Balachandran, B.: Data driven forecasting of aperiodic motions of non-autonomous systems. Chaos Interdiscip. J. Nonlinear Sci. 31(2) (2021)

  39. Demir-Kavuk, O., Kamada, M., Akutsu, T., Knapp, E.W.: Prediction using step-wise l1, l2 regularization and feature selection for small data sets with large number of features. BMC Bioinform. 12, 1–10 (2011)

    Article  Google Scholar 

  40. Lorenz, E.N.: Deterministic nonperiodic flow. J. Atmos. Sci. 20(2), 130–141 (1963)

    Article  MathSciNet  MATH  Google Scholar 

  41. Breunung, Thomas, Balachandran, Balakumar: Computationally efficient simulations of stochastically perturbed nonlinear dynamical systems. J. Comput. Nonlinear Dyn. 17(9), 091008 (2022)

    Google Scholar 

  42. Zhao, Xiangxue, Azarm, Shapour, Balachandran, Balakumar: Online data-driven prediction of spatio-temporal system behavior using high-fidelity simulations and sparse sensor measurements. J. Mech. Des. 143(2), 021701 (2021)

    Article  Google Scholar 

Download references

Funding

This research was supported by grants from the National Natural Science Foundation of China (Grant No. 51978216, 52192664) and Natural Science Foundation of Heilongjiang Province (Grant No. LH2022E119).

Author information

Authors and Affiliations

  1. School of Civil Engineering, Harbin Institute of Technology, Harbin, 150090, China

    Weida Zhai & Yuequan Bao

  2. Institute of Engineering Mechanics, China Earthquake Administration, Harbin, 150080, China

    Dongwang Tao

Authors
  1. Weida Zhai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dongwang Tao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yuequan Bao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yuequan Bao.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhai, W., Tao, D. & Bao, Y. Parameter estimation and modeling of nonlinear dynamical systems based on Runge–Kutta physics-informed neural network. Nonlinear Dyn 111, 21117–21130 (2023). https://doi.org/10.1007/s11071-023-08933-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11071-023-08933-6

Keywords

Search

Navigation