Skip to main content
SpringerLink
Log in

Neural network as a function approximator and its application in solving differential equations

  • Published:
Applied Mathematics and Mechanics Aims and scope Submit manuscript

Abstract

A neural network (NN) is a powerful tool for approximating bounded continuous functions in machine learning. The NN provides a framework for numerically solving ordinary differential equations (ODEs) and partial differential equations (PDEs) combined with the automatic differentiation (AD) technique. In this work, we explore the use of NN for the function approximation and propose a universal solver for ODEs and PDEs. The solver is tested for initial value problems and boundary value problems of ODEs, and the results exhibit high accuracy for not only the unknown functions but also their derivatives. The same strategy can be used to construct a PDE solver based on collocation points instead of a mesh, which is tested with the Burgers equation and the heat equation (i.e., the Laplace equation).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. VAN GERVEN, M. and BOHTE, S. Artificial neural networks as models of neural information processing. Frontiers in Computational Neuroscience, 11, 00114 (2017)

    Article  Google Scholar 

  2. HAYKIN, S. S. Neural Networks and Learning Machines, 3rd ed., Prentice Hall, Englewood, New Jersey (2008)

    Google Scholar 

  3. KRIZHEVSKY, A., SUTSKEVER, I., and HINTON, G. E. Imagenet classification with deep convolutional neural networks. Proceedings of the 25th International Conference on Neural Information Processing Systems, 1, 1097–1105 (2012)

    Google Scholar 

  4. LECUN, Y., BENGIO, Y., and HINTON, G. Deep learning. nature, 521, 436–444 (2015)

    Article  Google Scholar 

  5. LAKE, B. M., SALAKHUTDINOV, R., and TENENBAUM, J. B. Human-level concept learning through probabilistic program induction. Science, 350, 1332–1338 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  6. ALIPANAHI, B., DELONG, A., WEIRAUCH, M. T., and FREY, B. J. Predicting the sequence specificities of DNA- and RNA-binding proteins by deep learning. Nature Biotechnology, 33, 831–838 (2015)

    Article  Google Scholar 

  7. MCCULLOCH, W. S. and PITTS, W. A logical calculus of the ideas immanent in nervous activity. The Bulletin of Mathematical Biophysics, 5, 115–133 (1943)

    Article  MathSciNet  MATH  Google Scholar 

  8. ROSENBLATT, F. The perceptron: a probabilistic model for information storage and organization in the brain. Psychological Review, 65, 386–408 (1958)

    Article  Google Scholar 

  9. ROSENBLATT, F. Principles of Neurodynamics: Perceptions and the Theory of Brain Mechanism, Spartan Books, Washington, D.C. (1961)

    Book  Google Scholar 

  10. MINSKY, M. and PAPERT, S. A. Perceptrons: An Introduction to Computational Geometry, MIT Press, Cambridge (1969)

    MATH  Google Scholar 

  11. WERBOS, P. Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences, Ph.D. dissertation, Harvard University (1974)

    Google Scholar 

  12. RALL, L. B. Automatic Differentiation: Techniques and Applications, Springer-Verlag, Berlin (1981)

    Book  MATH  Google Scholar 

  13. BAYDIN, A. G., PEARLMUTTER, B. A., RADUL, A. A., and SISKIND, J. M. Automatic differentiation in machine learning: a survey. The Journal of Machine Learning Research, 18, 1–43 (2018)

    MATH  Google Scholar 

  14. RAISSI, M., PERDIKARIS, P., and KARNIADAKIS, G. E. Physics informed deep learning (part i): data-driven solutions of nonlinear partial differential equations. arXiv, arXiv: 1711.10561 (2017) https://arxiv.org/abs/1711.10561

  15. MALL, S. and CHAKRAVERTY, S. Application of legendre neural network for solving ordinary differential equations. Applied Soft Computing, 43, 347–356 (2016)

    Article  Google Scholar 

  16. MALL, S. and CHAKRAVERTY, S. Chebyshev neural network based model for solving laneemden type equations. Applied Mathematics and Computation, 247, 100–114 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  17. BERG, J. and NYSTRÖM, K. A unified deep artificial neural network approach to partial differential equations in complex geometries. arXiv, arXiv: 1711.06464 (2017) https://arxiv.org/abs/ 1711.06464

  18. MAI-DUY, N. and TRAN-CONG, T. Numerical solution of differential equations using multiquadric radial basis function networks. Neural Networks, 14, 185–199 (2001)

    Article  MATH  Google Scholar 

  19. JIANYU, L., SIWEI, L., YINGJIAN, Q., and YAPING, H. Numerical solution of elliptic partial differential equation using radial basis function neural networks. Neural Networks, 16, 729–734 (2003)

    Article  Google Scholar 

  20. ABADI, M., BARHAM, P., CHEN, J., CHEN, Z., DAVIS, A., DEAN, J., DEVIN, M., GHEMAWAT, S., IRVING, G., ISARD, M., and KUDLUR, M. Tensorflow: a system for large-scale machine learning. The 12th USENIX Symposium on Operating Systems Design and Implementation, 16, 265–283 (2016)

    Google Scholar 

  21. HORNIK, K., STINCHCOMBE, M., and WHITE, H. Multilayer feedforward networks are universal approximators. Neural Networks, 2, 359–366 (1989)

    Article  MATH  Google Scholar 

  22. CYBENKO, G. Approximation by superpositions of a sigmoidal function. Mathematics of Control, Signals and Systems, 2, 303–314 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  23. JONES, L. K. Constructive approximations for neural networks by sigmoidal functions. Proceedings of the IEEE, 78, 1586–1589 (1990)

    Article  Google Scholar 

  24. CARROLL, S. and DICKINSON, B. Construction of neural networks using the Radon transform. IEEE International Conference on Neural Networks, 1, 607–611 (1989)

    Article  Google Scholar 

  25. LIU, D. C. and NOCEDAL, J. On the limited memory BFGS method for large scale optimization. Mathematical Programming, 45, 503–528 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  26. LEE, C. B. Possible universal transitional scenario in a flat plate boundary layer: measurement and visualization. Physical Review E, 62, 3659–3670 (2000)

    Article  Google Scholar 

  27. LEE, C. B. and WU, J. Z. Transition in wall-bounded flows. Applied Mechanics Reviews, 61, 030802 (2008)

    Article  MATH  Google Scholar 

  28. LEE, C. B. New features of CS solitons and the formation of vortices. Physics Letters A, 247, 397–402 (1998)

    Article  Google Scholar 

  29. LEE, C. B. and FU, S. On the formation of the chain of ring-like vortices in a transitional boundary layer. Experiments in Fluids, 30, 354–357 (2001)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. State Key Laboratory for Turbulence and Complex System, Peking University, Beijing, 100871, China

    Zeyu Liu, Yantao Yang & Qingdong Cai

  2. Department of Mechanics and Engineering Science, College of Engineering, Peking University, Beijing, 100871, China

    Zeyu Liu, Yantao Yang & Qingdong Cai

  3. Center for Applied Physics and Technology, College of Engineering, Peking University, Beijing, 100871, China

    Qingdong Cai

Authors
  1. Zeyu Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yantao Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qingdong Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qingdong Cai.

Additional information

Project supported by the National Natural Science Foundation of China (No. 11521091)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, Z., Yang, Y. & Cai, Q. Neural network as a function approximator and its application in solving differential equations. Appl. Math. Mech.-Engl. Ed. 40, 237–248 (2019). https://doi.org/10.1007/s10483-019-2429-8

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10483-019-2429-8

Key words

Chinese Library Classification

2010 Mathematics Subject Classification

Search

Navigation