Skip to main content
SpringerLink
Log in

General solutions for nonlinear differential equations: a rule-based self-learning approach using deep reinforcement learning

  • Original Paper
  • Published:
Computational Mechanics Aims and scope Submit manuscript

Abstract

A universal rule-based self-learning approach using deep reinforcement learning (DRL) is proposed for the first time to solve nonlinear ordinary differential equations and partial differential equations. The solver consists of a deep neural network-structured actor that outputs candidate solutions, and a critic derived only from physical rules (governing equations and boundary and initial conditions). Solutions in discretized time are treated as multiple tasks sharing the same governing equation, and the current step parameters provide an ideal initialization for the next owing to the temporal continuity of the solutions, which shows a transfer learning characteristic and indicates that the DRL solver has captured the intrinsic nature of the equation. The approach is verified through solving the Schrödinger, Navier–Stokes, Burgers’, Van der Pol, and Lorenz equations and an equation of motion. The results indicate that the approach gives solutions with high accuracy, and the solution process promises to get faster.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Abbreviations

DRL:

Deep reinforcement learning

ODE:

Ordinary differential equation

PDE:

Partial differential equation

AGI:

Artificial general intelligence

DNN:

Deep neural network

MDP:

Markov decision process

SDOF:

Single degree of freedom

MDOF:

Multi-degree of freedom

\( {\mathbf{x}} \) :

Spatial coordinate

\( t \) :

Time coordinate

\( u \) :

Solution of the ODE or PDE

\( u_{t} \) :

Derivative of \( u \) with respect to time

\( \vartheta \) :

Parameters of ODE or PDE

\( s \) :

State

\( {{\boldsymbol{\mathcal{S}}}} \) :

Domain of the state

\( a \) :

Action

\( {{\boldsymbol{\mathcal{A}}}} \) :

Domain of the action

\( \theta \) :

Trainable parameters of the policy network

\( \pi_{\theta } \left( {a|s} \right) \) :

Probabilistic policy of the action given the state

\( \mu_{\theta } \) :

Mean value of the action determined by the probabilistic policy of the action given the state

\( \sigma_{\theta } \) :

Standard deviation of the action determined by the probabilistic policy of the action given the state

\( r\left( {s,a} \right) \) :

Imbalance of an ODE or PDE

\( r_{Eq} \) :

Imbalance of governing equation

\( r_{B} \) :

Imbalance of boundary conditions

\( r_{I} \) :

Imbalance of initial conditions

\( J \) :

Loss function of policy network

\( {\rm E}_{{\pi_{\theta } }} \) :

Expectation calculated on probabilistic policy

\( \hat{u} \) :

Candidate solution sampled from policy

\( N \) :

Batch size of sampling states

References

  1. Cole JD (1951) On a quasi-linear parabolic equation occurring in aerodynamics. Q Appl Math 9(3):225–236

    Article  MathSciNet  Google Scholar 

  2. Soliman AA (2006) The modified extended tanh-function method for solving Burgers-type equations. Physica A 361(2):394–404

    Article  MathSciNet  Google Scholar 

  3. Feit M, Fleck J Jr, Steiger A (1982) Solution of the Schrödinger equation by a spectral method. J Comput Phys 47(3):412–433

    Article  MathSciNet  Google Scholar 

  4. Wang J X, Kurth-Nelson Z, Tirumala D, Soyer H, Leibo J Z, Munos R, Blundell C, Kumaran D, Botvinick M (2016) Learning to reinforcement learn. arXiv preprint arXiv:1611.05763

  5. Finn C, Abbeel P, Levine S (2017) Model-agnostic meta-learning for fast adaptation of deep networks. arXiv preprint arXiv:1703.03400

  6. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444

    Article  Google Scholar 

  7. Sutton RS, Barto AG (1998) Introduction to reinforcement learning, vol 135. MIT press, Cambridge

    MATH  Google Scholar 

  8. Raissi M, Yazdani A, Karniadakis G E (2018) Hidden fluid mechanics: a Navier–Stokes informed deep learning framework for assimilating flow visualization data. arXiv preprint arXiv:1808.04327

  9. Raissi M, Wang Z, Triantafyllou MS, Karniadakis GE (2019) Deep learning of vortex-induced vibrations. J Fluid Mech 861:119–137

    Article  MathSciNet  Google Scholar 

  10. Lagaris IE, Likas A, Fotiadis DI (1998) Artificial neural networks for solving ordinary and partial differential equations. IEEE Trans Neural Netw 9(5):987–1000

    Article  Google Scholar 

  11. Raissi M, Perdikaris P, Karniadakis GE (2019) Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J Comput Phys 378:686–707

    Article  MathSciNet  Google Scholar 

  12. Han J, Jentzen A, EW (2017) Overcoming the curse of dimensionality: solving high-dimensional partial differential equations using deep learning. arXiv preprint arXiv:1707.02568

  13. Mills K, Spanner M, Tamblyn I (2017) Deep learning and the Schrödinger equation. Phys Rev A 96(4):042113

    Article  Google Scholar 

  14. Han EW, Jentzen A (2017) Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations. Commun Math Stat 5(4):349–380

    Article  MathSciNet  Google Scholar 

  15. Khoo Y, Lu J, Ying L (2017) Solving parametric PDE problems with artificial neural networks. arXiv preprint arXiv:1707.03351

  16. Bostanabad R, Zhang Y, Li X, Kearney T, Brinson LC, Apley DW, Liu WK, Chen W (2018) Computational microstructure characterization and reconstruction: review of the state-of-the-art techniques. Prog Mater Sci 95:1–41

    Article  Google Scholar 

  17. Li X, Yang Z, Brinson L C, Choudhary A, Agrawal A, Chen W (2018). A deep adversarial learning methodology for designing microstructural material systems. Paper presented at the ASME 2018 international design engineering technical conferences and computers and information in engineering conference

  18. Mnih V, Kavukcuoglu K, Silver D, Graves A, Antonoglou I, Wierstra D, Riedmiller M (2013) Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602

  19. Silver D, Huang A, Maddison CJ, Guez A, Sifre L, Van Den Driessche G, Schrittwieser J, Antonoglou I, Panneershelvam V, Lanctot M (2016) Mastering the game of Go with deep neural networks and tree search. Nature 529(7587):484

    Article  Google Scholar 

  20. Wang K, Sun W (2019) Meta-modeling game for deriving theory-consistent, microstructure-based traction–separation laws via deep reinforcement learning. Comput Methods Appl Mech Eng 346:216–241

    Article  MathSciNet  Google Scholar 

  21. Wang K, Sun W, Du Q (2019) A cooperative game for automated learning of elasto-plasticity knowledge graphs and models with AI-guided experimentation. arXiv preprint arXiv:1903.04307

  22. Jin X, Cheng P, Chen W-L, Li H (2018) Prediction model of velocity field around circular cylinder over various Reynolds numbers by fusion convolutional neural networks based on pressure on the cylinder. Phys Fluids 30(4):047105

    Article  Google Scholar 

  23. Li S, Laima S, Li H (2018) Data-driven modeling of vortex-induced vibration of a long-span suspension bridge using decision tree learning and support vector regression. J Wind Eng Ind Aerodyn 172:196–211

    Article  Google Scholar 

  24. Silver D, Lever G, Heess N, Degris T, Wierstra D, Riedmiller M (2014) Deterministic policy gradient algorithms. In: ICML

  25. Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980

  26. Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT press, Cambridge

    MATH  Google Scholar 

  27. Bergstra J, Bengio Y (2012) Random search for hyper-parameter optimization. J Mach Learn Res 13:281–305

    MathSciNet  MATH  Google Scholar 

  28. Snoek J, Larochelle H, Adams R P (2012). Practical bayesian optimization of machine learning algorithms. Paper presented at the advances in neural information processing systems

  29. Guckenheimer J, Holmes P (2013) Nonlinear oscillations, dynamical systems, and bifurcations of vector fields, vol 42. Springer, Belrin

    MATH  Google Scholar 

  30. Tsatsos M (2006) Theoretical and Numerical study of the Van der Pol equation. Doctoral desertation, Aristotle University of Thessaloniki, vol 4, p 6

  31. Chopra AK (2007) Dynamics of structures: theory and applications to earthquake engineering. Prentice-Hall, Upper Saddle River

    Google Scholar 

  32. Hopf E (1950) The partial differential equation ut + uux = μxx. Commun Pure Appl Math 3(3):201–230

    Article  Google Scholar 

  33. Skeel RD, Berzins M (1990) A method for the spatial discretization of parabolic equations in one space variable. SIAM J Sci Stat Comput 11(1):1–32

    Article  MathSciNet  Google Scholar 

  34. Schrödinger E (1926) An undulatory theory of the mechanics of atoms and molecules. Phys Rev 28(6):1049

    Article  Google Scholar 

  35. Cox SM, Matthews PC (2002) Exponential time differencing for stiff systems. J Comput Phys 176(2):430–455

    Article  MathSciNet  Google Scholar 

  36. Driscoll TA, Hale N, Trefethen LN (2014) Chebfun guide. Pafnuty Publications, Oxford

    Google Scholar 

  37. Bessa M, Bostanabad R, Liu Z, Hu A, Apley DW, Brinson C, Chen W, Liu WK (2017) A framework for data-driven analysis of materials under uncertainty: countering the curse of dimensionality. Comput Methods Appl Mech Eng 320:633–667

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

This research was funded by the National Natural Sciences Foundation of China (NSFC) (Grant Nos. U1711265 and 51638007).

Author information

Author notes

  1. Shiyin Wei and Xiaowei Jin contributed to the work equally and should be regarded as co-first authors.

Authors and Affiliations

  1. Key Lab of Smart Prevention and Mitigation of Civil Engineering Disasters of the Ministry of Industry and Information Technology, Harbin Institute of Technology, Harbin, 150090, China

    Shiyin Wei, Xiaowei Jin & Hui Li

  2. Key Lab of Structures Dynamic Behavior and Control of the Ministry of Education, Harbin Institute of Technology, Harbin, 150090, China

    Shiyin Wei, Xiaowei Jin & Hui Li

  3. School of Civil Engineering, Harbin Institute of Technology, Harbin, 150090, China

    Shiyin Wei, Xiaowei Jin & Hui Li

Authors
  1. Shiyin Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaowei Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hui Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hui Li.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wei, S., Jin, X. & Li, H. General solutions for nonlinear differential equations: a rule-based self-learning approach using deep reinforcement learning. Comput Mech 64, 1361–1374 (2019). https://doi.org/10.1007/s00466-019-01715-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00466-019-01715-1

Keywords

Search

Navigation