Skip to main content
SpringerLink
Log in

Accelerating Explicit Time-Stepping with Spatially Variable Time Steps Through Machine Learning

  • Published:
Journal of Scientific Computing Aims and scope Submit manuscript

Abstract

Use of machine learning (ML) to solve partial differential equations (PDEs) is a growing area of research. In this work we apply ML to accelerate a numerical discretization scheme: the radial basis function time domain (RBF-TD) method. The RBF-TD method uses scattered nodes in both space and time to time-step PDEs. Here we investigate replacing a costly L1 minimization step in the original RBF-TD method with an ensemble ML model known as an extremely randomized trees regressor (ERT). We show that an ERT model trained on a simple PDE can maintain the high order accuracy of the original method with often significant speed up, while generalizing to a variety of convection-dominated PDEs. Through this work, we also extend the RBF-TD method to problems in 2-D space plus time. This study illustrates a novel opportunity to use ML to augment finite difference-related approximations while maintaining high order convergence under node refinement.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data Availability

The code and models are written in a combination of Matlab and Python and are available upon reasonable request addressed to the corresponding author.

References

  1. Abrahamsen, D., Fornberg, B.: Explicit time stepping of PDEs with local refinement in space-time. J. Sci. Comput. 81(3), 1945–1962 (2019). https://doi.org/10.1007/s10915-019-01065-3

    Article  MathSciNet  MATH  Google Scholar 

  2. Bayona, V., Flyer, N., Fornberg, B.: On the role of polynomials in RBF-FD approximations: III. Behavior near domain boundaries. J. Comput. Phys. 380, 378–399 (2019). https://doi.org/10.1016/j.jcp.2018.12.013

    Article  MathSciNet  MATH  Google Scholar 

  3. Bayona, V., Flyer, N., Fornberg, B., Barnett, G.A.: On the role of polynomials in RBF-FD approximations: II. Numerical solution of elliptic PDEs. J. Comput. Phys. 332, 257–273 (2017). https://doi.org/10.1016/j.jcp.2016.12.008

    Article  MathSciNet  MATH  Google Scholar 

  4. Beck, A.D., Zeifang, J., Schwarz, A., Flad, D.G.: A neural network based shock detection and localization approach for discontinuous Galerkin methods. J. Comput. Phys. 423, 109824 (2020). https://doi.org/10.1016/j.jcp.2020.109824

    Article  MathSciNet  MATH  Google Scholar 

  5. Flyer, N., Barnett, G., Wicker, L.: Enhancing finite differences with radial basis functions: experiments on the Navier–Stokes equations. J. Comput. Phys. 316, 39–62 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  6. Flyer, N., Fornberg, B., Bayona, V., Barnett, G.A.: On the role of polynomials in RBF-FD approximations: I. Interpolation and accuracy. J. Comput. Phys. 321, 21–38 (2016). https://doi.org/10.1016/j.jcp.2016.05.026

    Article  MathSciNet  MATH  Google Scholar 

  7. Fornberg, B., Flyer, N.: Fast generation of 2-D node distributions for mesh-free PDE discretizations. Comput. Math. Appl. 7, 531–544 (2015). https://doi.org/10.1016/j.camwa.2015.01.009

    Article  MathSciNet  MATH  Google Scholar 

  8. Fornberg, B., Flyer, N.: A primer on radial basis functions with applications to the geosciences. Soc. Ind. Appl. Math. (2015)

  9. Geurts, P., Ernst, D., Wehenkel, L.: Extremely randomized trees. Mach. Learn. 63(1), 3–42 (2006)

    Article  MATH  Google Scholar 

  10. Hsieh, J., Zhao, S., Eismann, S., Mirabella, L., Ermon, S.: Learning neural PDE solvers with convergence guarantees. In: International Conference on Learning Representations (2019). https://openreview.net/forum?id=rklaWn0qK7

  11. Huang, B., Boutros, P.: The parameter sensitivity of random forests. BMC Bioinf. (2016). https://doi.org/10.1186/s12859-016-1228-x

    Article  Google Scholar 

  12. Kossaczká, T., Ehrhardt, M., Günther, M.: Enhanced fifth order WENO shock-capturing schemes with deep learning. arXiv:2103.04988 [physics] (2021)

  13. Lagaris, I.E., Likas, A., Fotiadis, D.I.: Artificial neural networks for solving ordinary and partial differential equations. IEEE Trans. Neural Netw. 9(5), 987–1000 (1998)

    Article  Google Scholar 

  14. Lagaris, I.E., Likas, A.C., Papageorgiou, D.G.: Neural-network methods for boundary value problems with irregular boundaries. IEEE Trans. Neural Netw. 11(5), 1041–1049 (2000)

    Article  Google Scholar 

  15. Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A., Anandkumar, A.: Fourier neural operator for parametric partial differential equations. arXiv:2010.08895 [cs, math] (2021)

  16. Long, Z., Lu, Y., Ma, X., Dong, B.: PDE-Net: Learning PDEs from data. In: Proceedings of the 35th International Conference on Machine Learning, pp. 3208–3216. PMLR (2018)

  17. Lu, L., Jin, P., Karniadakis, G.: DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators. arXiv:1910.03193 [cs, stat] (2020)

  18. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, E.: Scikit-learn: machine learning in python. J. Mach. Learn. Res. 12, 2825–2830 (2011)

    MathSciNet  MATH  Google Scholar 

  19. Psichogios, D., Ungar, L.: A hybrid neural network-first principles approach to process modeling. AIChE J. 38(10), 1499–1511 (1992)

    Article  Google Scholar 

  20. Raissi, M., Karniadakis, G.: Hidden physics models: machine learning of nonlinear partial differential equations. J. Comput. Phys. 357, 125–141 (2018). https://doi.org/10.1016/j.jcp.2017.11.039

    Article  MathSciNet  MATH  Google Scholar 

  21. Raissi, M., Perdikaris, P., Karniadakis, G.: Machine learning of linear differential equations using gaussian processes. J. Comput. Phys. 348, 683–693 (2017). https://doi.org/10.1016/j.jcp.2017.07.050

    Article  MathSciNet  MATH  Google Scholar 

  22. Raissi, M., Perdikaris, P., Karniadakis, G.: Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys. 378, 686–707 (2019). https://doi.org/10.1016/j.jcp.2018.10.045

    Article  MathSciNet  MATH  Google Scholar 

  23. Slak, J., Kosec, G.: On generation of node distributions for meshless pde discretizations. SIAM J. Sci. Comput. 41(5), A3202–A3229 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  24. Stevens, B., Colonius, T.: Enhancement of shock-capturing methods via machine learning. Theoret. Comput. Fluid Dyn. 34(4), 483–496 (2020). https://doi.org/10.1007/s00162-020-00531-1

    Article  MathSciNet  Google Scholar 

  25. van der Sande, K., Fornberg, B.: Fast variable density 3-D node generation. SIAM J. Sci. Comput. 43(1), A242–A257 (2021)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Funding

The authors declare that no funds, grants, or other support were received during the preparation of this manuscript.

Author information

Authors and Affiliations

  1. Department of Applied Mathematics, University of Colorado, Boulder, CO, 80309, USA

    Kiera van der Sande & Bengt Fornberg

  2. Southwest Research Institute, Boulder, CO, 80302, USA

    Kiera van der Sande

  3. Flyer Research LLC, Boulder, CO, 80305, USA

    Natasha Flyer

Authors
  1. Kiera van der Sande
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Natasha Flyer
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bengt Fornberg
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kiera van der Sande.

Ethics declarations

Conflict of interest

The authors have no relevant financial or non-financial interests to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

van der Sande, K., Flyer, N. & Fornberg, B. Accelerating Explicit Time-Stepping with Spatially Variable Time Steps Through Machine Learning. J Sci Comput 96, 31 (2023). https://doi.org/10.1007/s10915-023-02260-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10915-023-02260-z

Keywords

Mathematics Subject Classification

Search

Navigation