Advertisement

Applied Mathematics and Mechanics

, Volume 40, Issue 2, pp 237–248 | Cite as

Neural network as a function approximator and its application in solving differential equations

  • Zeyu Liu
  • Yantao Yang
  • Qingdong CaiEmail author
Article
  • 32 Downloads

Abstract

A neural network (NN) is a powerful tool for approximating bounded continuous functions in machine learning. The NN provides a framework for numerically solving ordinary differential equations (ODEs) and partial differential equations (PDEs) combined with the automatic differentiation (AD) technique. In this work, we explore the use of NN for the function approximation and propose a universal solver for ODEs and PDEs. The solver is tested for initial value problems and boundary value problems of ODEs, and the results exhibit high accuracy for not only the unknown functions but also their derivatives. The same strategy can be used to construct a PDE solver based on collocation points instead of a mesh, which is tested with the Burgers equation and the heat equation (i.e., the Laplace equation).

Key words

neural network (NN) function approximation ordinary differential equation (ODE) solver partial differential equation (PDE) solver 

Chinese Library Classification

O241 

2010 Mathematics Subject Classification

65D15 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    VAN GERVEN, M. and BOHTE, S. Artificial neural networks as models of neural information processing. Frontiers in Computational Neuroscience, 11, 00114 (2017)CrossRefGoogle Scholar
  2. [2]
    HAYKIN, S. S. Neural Networks and Learning Machines, 3rd ed., Prentice Hall, Englewood, New Jersey (2008)Google Scholar
  3. [3]
    KRIZHEVSKY, A., SUTSKEVER, I., and HINTON, G. E. Imagenet classification with deep convolutional neural networks. Proceedings of the 25th International Conference on Neural Information Processing Systems, 1, 1097–1105 (2012)Google Scholar
  4. [4]
    LECUN, Y., BENGIO, Y., and HINTON, G. Deep learning. nature, 521, 436–444 (2015)CrossRefGoogle Scholar
  5. [5]
    LAKE, B. M., SALAKHUTDINOV, R., and TENENBAUM, J. B. Human-level concept learning through probabilistic program induction. Science, 350, 1332–1338 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  6. [6]
    ALIPANAHI, B., DELONG, A., WEIRAUCH, M. T., and FREY, B. J. Predicting the sequence specificities of DNA- and RNA-binding proteins by deep learning. Nature Biotechnology, 33, 831–838 (2015)CrossRefGoogle Scholar
  7. [7]
    MCCULLOCH, W. S. and PITTS, W. A logical calculus of the ideas immanent in nervous activity. The Bulletin of Mathematical Biophysics, 5, 115–133 (1943)MathSciNetCrossRefzbMATHGoogle Scholar
  8. [8]
    ROSENBLATT, F. The perceptron: a probabilistic model for information storage and organization in the brain. Psychological Review, 65, 386–408 (1958)CrossRefGoogle Scholar
  9. [9]
    ROSENBLATT, F. Principles of Neurodynamics: Perceptions and the Theory of Brain Mechanism, Spartan Books, Washington, D.C. (1961)CrossRefGoogle Scholar
  10. [10]
    MINSKY, M. and PAPERT, S. A. Perceptrons: An Introduction to Computational Geometry, MIT Press, Cambridge (1969)zbMATHGoogle Scholar
  11. [11]
    WERBOS, P. Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences, Ph.D. dissertation, Harvard University (1974)Google Scholar
  12. [12]
    RALL, L. B. Automatic Differentiation: Techniques and Applications, Springer-Verlag, Berlin (1981)CrossRefzbMATHGoogle Scholar
  13. [13]
    BAYDIN, A. G., PEARLMUTTER, B. A., RADUL, A. A., and SISKIND, J. M. Automatic differentiation in machine learning: a survey. The Journal of Machine Learning Research, 18, 1–43 (2018)zbMATHGoogle Scholar
  14. [14]
    RAISSI, M., PERDIKARIS, P., and KARNIADAKIS, G. E. Physics informed deep learning (part i): data-driven solutions of nonlinear partial differential equations. arXiv, arXiv: 1711.10561 (2017) https://arxiv.org/abs/1711.10561Google Scholar
  15. [15]
    MALL, S. and CHAKRAVERTY, S. Application of legendre neural network for solving ordinary differential equations. Applied Soft Computing, 43, 347–356 (2016)CrossRefGoogle Scholar
  16. [16]
    MALL, S. and CHAKRAVERTY, S. Chebyshev neural network based model for solving laneemden type equations. Applied Mathematics and Computation, 247, 100–114 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  17. [17]
    BERG, J. and NYSTRÖM, K. A unified deep artificial neural network approach to partial differential equations in complex geometries. arXiv, arXiv: 1711.06464 (2017) https://arxiv.org/abs/ 1711.06464Google Scholar
  18. [18]
    MAI-DUY, N. and TRAN-CONG, T. Numerical solution of differential equations using multiquadric radial basis function networks. Neural Networks, 14, 185–199 (2001)CrossRefzbMATHGoogle Scholar
  19. [19]
    JIANYU, L., SIWEI, L., YINGJIAN, Q., and YAPING, H. Numerical solution of elliptic partial differential equation using radial basis function neural networks. Neural Networks, 16, 729–734 (2003)CrossRefGoogle Scholar
  20. [20]
    ABADI, M., BARHAM, P., CHEN, J., CHEN, Z., DAVIS, A., DEAN, J., DEVIN, M., GHEMAWAT, S., IRVING, G., ISARD, M., and KUDLUR, M. Tensorflow: a system for large-scale machine learning. The 12th USENIX Symposium on Operating Systems Design and Implementation, 16, 265–283 (2016)Google Scholar
  21. [21]
    HORNIK, K., STINCHCOMBE, M., and WHITE, H. Multilayer feedforward networks are universal approximators. Neural Networks, 2, 359–366 (1989)CrossRefzbMATHGoogle Scholar
  22. [22]
    CYBENKO, G. Approximation by superpositions of a sigmoidal function. Mathematics of Control, Signals and Systems, 2, 303–314 (1989)MathSciNetCrossRefzbMATHGoogle Scholar
  23. [23]
    JONES, L. K. Constructive approximations for neural networks by sigmoidal functions. Proceedings of the IEEE, 78, 1586–1589 (1990)CrossRefGoogle Scholar
  24. [24]
    CARROLL, S. and DICKINSON, B. Construction of neural networks using the Radon transform. IEEE International Conference on Neural Networks, 1, 607–611 (1989)CrossRefGoogle Scholar
  25. [25]
    LIU, D. C. and NOCEDAL, J. On the limited memory BFGS method for large scale optimization. Mathematical Programming, 45, 503–528 (1989)MathSciNetCrossRefzbMATHGoogle Scholar
  26. [26]
    LEE, C. B. Possible universal transitional scenario in a flat plate boundary layer: measurement and visualization. Physical Review E, 62, 3659–3670 (2000)CrossRefGoogle Scholar
  27. [27]
    LEE, C. B. and WU, J. Z. Transition in wall-bounded flows. Applied Mechanics Reviews, 61, 030802 (2008)CrossRefzbMATHGoogle Scholar
  28. [28]
    LEE, C. B. New features of CS solitons and the formation of vortices. Physics Letters A, 247, 397–402 (1998)CrossRefGoogle Scholar
  29. [29]
    LEE, C. B. and FU, S. On the formation of the chain of ring-like vortices in a transitional boundary layer. Experiments in Fluids, 30, 354–357 (2001)CrossRefGoogle Scholar

Copyright information

© Shanghai University and Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.State Key Laboratory for Turbulence and Complex SystemPeking UniversityBeijingChina
  2. 2.Department of Mechanics and Engineering Science, College of EngineeringPeking UniversityBeijingChina
  3. 3.Center for Applied Physics and Technology, College of EngineeringPeking UniversityBeijingChina

Personalised recommendations