Skip to main content
SpringerLink
Log in

The neural network collocation method for solving partial differential equations

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

This paper presents a meshfree collocation method that uses deep learning to determine the basis functions as well as their corresponding weights. This method is shown to be able to approximate elliptic, parabolic, and hyperbolic partial differential equations for both forced and unforced systems, as well as linear and nonlinear partial differential equations. By training a homogeneous network and particular network separately, new forcing functions are able to be approximated quickly without the burden of retraining the full network. The network is demonstrated on several numerical examples including a nonlinear elasticity problem. In addition to providing meshfree approximations to strong form partial differential equations directly, this technique could also provide a foundation for deep learning methods to be used as preconditioners to traditional methods, where the deep learning method will get close to a solution and traditional solvers can finish the solution.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Similar content being viewed by others

Availability of data and material

All data are available upon request and clearance for dispersal.

Code availability

All code is available upon request and clearance for dispersal.

References

  1. Cajori F (1928) The early history of partial differential equations and of partial differentiation and integration. Am Math Month 35:459–467

    Article  MathSciNet  Google Scholar 

  2. Grossmann C, Roos H-G, Stynes M (2007) Numerical treatment of partial differential equations. Springer, Berlin

    Book  Google Scholar 

  3. Bathe K-J (2014) Finite element procedures, 2nd edn. K.J. Bathe, Watertown

    MATH  Google Scholar 

  4. Cook RD, Malkus DS, Plesha ME, Witt RJ (2001) Concepts and applications of finite element analysis, 4th edn. Wiley, New York

    Google Scholar 

  5. Eymard R, Gallouet T, Herbin R (2000) Finite Volume Methods. In: Techniques of Scientific Computing, Part III, Handbook of Numerical Analysis, VII, North-Holland, pp 713–1020

  6. Gingold R, Monaghan J (1977) Smoother particle hydrodynamics: theory and application to non-spherical stars. Mon Not R Astr Soc 181(375):375–389

    Article  Google Scholar 

  7. Lucy L (1977) A numerical approach to the testing of the fission hypothesis. Aston J 82:1013–1024

    Article  Google Scholar 

  8. Chen JS, Hillman M, Chi SW (2017) Meshfree methods: progress made after 20 Years. J Eng Mech, 143(4):04017001

    Article  Google Scholar 

  9. Cybenko G (1989) Approximations by supersistion of sigmoidal functions. Math Contr Signal Syst 2(4):313–314

    Article  Google Scholar 

  10. Hornik K (1991) Approximation capabilities of multilayer feedforward networks. Neural Netw 4(2):251–257

    Article  MathSciNet  Google Scholar 

  11. Park J, Sandberg I (1991) Universal approximation using radial-basis function networks. Neural Comput 3(2):246–257

    Article  Google Scholar 

  12. Lee H, Kang IS (1990) Neural algorithm for solving differential equations. J Comput Phys 91:110–131

    Article  MathSciNet  Google Scholar 

  13. Meade A Jr, Fernandez A (1994) The numerical solution of linear ordinary differential equations by feedforward neural networks. Math Comput Modell 19(12):1–25

    Article  MathSciNet  Google Scholar 

  14. Lagaris IE, Likas A, Fotiadis DI (1998) Artificial neural networks for solving ordinary and partial differential equations. IEEE Trans Neural Netw 9(5):987–1000

    Article  Google Scholar 

  15. Lagaris I, Likas A, Papageorgiou D (2000) Neural-network methods for boundary value problems with irregular boundaries. IEEE Trans Neural Netw 11(5):1041–1049

    Article  Google Scholar 

  16. Effati S, Pakdaman M (2010) Artificial neural network approach for solving fuzzy differential equations. Inf Sci 180:1434–1457

    Article  MathSciNet  Google Scholar 

  17. Kumar M, Yadav N (2011) Multilayer perceptrons and radial basis function neural network methods for the solution of differential equations: a survey. Comput Math Appl 62:3796–3811

    Article  MathSciNet  Google Scholar 

  18. Rudd K, Ferrari S (2015) a constrained integration (CINT) approach to solving partial differential equations using artificial neural networks. Neurocomputing 155:277–285

    Article  Google Scholar 

  19. Sirignano J, Spiliopoulos K (2018) DGM: a deep learning algorithm for solving partial differential equations. J Computl Phys 375:1339–1364

    Article  MathSciNet  Google Scholar 

  20. Raissi M, Perdikaris P, Karniadakis GE (2018) Numerical gaussian processes for time-dependent and nonlinear partial differential equations. SIAM J Sci Comput 40(1):A172–A198

    Article  MathSciNet  Google Scholar 

  21. Raissi M, Karniadakis GE (2018) Hidden physics models: machine learning of nonlinear partial differential equations. J Comput Phys 357:125–141

    Article  MathSciNet  Google Scholar 

  22. Raissi M, Perdikaris P, Karniadakis G (2019) Physics-Informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J Comput Phys 378:686–707

    Article  MathSciNet  Google Scholar 

  23. Russell R, Shampine L (1972) A collocation method for boundary value problems. Numer Math 19(1):1–28

    Article  MathSciNet  Google Scholar 

  24. Kansa E (1990) Multiquadrics—a scattered data approximation scheme with applications to computation fluid dynamics—I surface approximations and partial derivative estimates. Comput Math Appl 19:127–145

    Article  MathSciNet  Google Scholar 

  25. Kansa E (1990) Multiquadrics—a scattered data approximation scheme with applications to computation fluid dynamics—II solutions to parabolic, hyperbolic and elliptic partial differential equations. Comput Math Appl 19:147–161

    Article  MathSciNet  Google Scholar 

  26. Tang B (1993) OPrthogonal array-based latin hypercubes. J Am Statist Assoc 88:1392–1397

    Article  Google Scholar 

  27. Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press, Cambridge

    MATH  Google Scholar 

  28. Farlow SJ (1982) Partial differential equations for scientists and engineers. Wiley, New York

    MATH  Google Scholar 

  29. Abadi M et al (2015) TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems, Software Available from tensorflow.org

  30. Chollet F (2015) Keras documentation. https://keras.io. Accessed 17 March 2020

  31. Wolfram Research, Inc. (2019) Mathematica, Version 12.0, Champaign: Wolfram Research, Inc.

  32. Belytschko T, Kam Liu W, Moran B (2000) Nonlinear finite elements for continua and structures. Wiley, New York

    MATH  Google Scholar 

Download references

Acknowledgements

Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the US Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525. This paper describes objective technical results and analysis. Any subjective views or opinions that might be expressed in the paper do not necessarily represent the views of the US Department of Energy or the US Government.

Funding

This work was sponsored by Sandia National Laboratories’ Lab Directed Research and Development (LDRD) 2019 campaign.

Author information

Authors and Affiliations

  1. Department of Structural Mechanics, Sandia National Laboratories, P.O. Box 5800, MS 0346, Albuquerque, NM, 57185-0346, USA

    Adam R. Brink

  2. Department of Component Sciences and Mechanics, ATA Engineering, Inc, 13290 Evening Creek Drive S, San Diego, CA, 92128, USA

    David A. Najera-Flores

  3. Department of Applied Machine Learning, Sandia National Laboratories, P.O. Box 5800, MS 0346, Albuquerque, NM, 57185-0346, USA

    Cari Martinez

Authors
  1. Adam R. Brink
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David A. Najera-Flores
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Cari Martinez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Adam R. Brink.

Ethics declarations

Conflict of interest

The authors have no conflicts of interest that they are aware of.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Brink, A.R., Najera-Flores, D.A. & Martinez, C. The neural network collocation method for solving partial differential equations. Neural Comput & Applic 33, 5591–5608 (2021). https://doi.org/10.1007/s00521-020-05340-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-020-05340-5

Keywords

Search

Navigation