Skip to main content
SpringerLink
Log in

Inherently interpretable machine learning solutions to differential equations

  • Original Article
  • Published:
Engineering with Computers Aims and scope Submit manuscript

Abstract

A machine learning method for the discovery of analytic solutions to differential equations is assessed. The method utilizes an inherently interpretable machine learning algorithm, genetic programming-based symbolic regression. An advantage of its interpretability is the output of symbolic expressions that can be used to assess error in algebraic terms, as opposed to purely numerical quantities. Therefore, models output by the developed method are verified by assessing its ability to recover known analytic solutions for two differential equations, as opposed to assessing numerical error. To demonstrate its improvement, the developed method is compared to a conventional, purely data-driven genetic programming-based symbolic regression algorithm. The reliability of successful evolution of the true solution, or an algebraic equivalent, is demonstrated.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data availability

All GPSR codes and data used to generate the results discussed in this paper are available for download here: https://github.com/HongsupOH/physics-regularized-bingo.

Notes

  1. Specific vendor and manufacturer names are explicitly mentioned only to accurately describe the test hardware. The use of vendor and manufacturer names does not imply an endorsement by the U.S. Government nor does it imply that the specified equipment is the best available.

References

  1. Karniadakis GE, Kevrekidis IG, Lu L, Perdikaris P, Wang S, Yang L (2021) Physics-informed machine learning. Nat Rev Phys 3(6):422–440. https://doi.org/10.1038/s42254-021-00314-5

    Article  Google Scholar 

  2. Raissi M, Perdikaris P, Karniadakis G (2019) Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J Comput Phys 378:686–707. https://doi.org/10.1016/j.jcp.2018.10.045

    Article  MathSciNet  MATH  Google Scholar 

  3. Karpatne A, Atluri G, Faghmous JH, Steinbach M, Banerjee A, Ganguly A, Shekhar S, Samatova N, Kumar V (2017) Theory-guided data science: a new paradigm for scientific discovery from data. IEEE Trans Knowl Data Eng 29(10):2318–2331. https://doi.org/10.1109/TKDE.2017.2720168

    Article  Google Scholar 

  4. Raissi M, Perdikaris P, Karniadakis GE (2017) Physics informed deep learning (part I): data-driven solutions of nonlinear partial differential equations. arXiv preprint arXiv:1711.10561. https://doi.org/10.48550/arXiv.1711.10561

  5. Heckman NM, Ivanoff TA, Roach AM, Jared BH, Tung DJ, Brown-Shaklee HJ, Huber T, Saiz DJ, Koepke JR, Rodelas JM, Madison JD, Salzbrenner BC, Swiler LP, Jones RE, Boyce BL (2020) Automated high-throughput tensile testing reveals stochastic process parameter sensitivity. Mater Sci Eng A 772:138632. https://doi.org/10.1016/j.msea.2019.138632

    Article  Google Scholar 

  6. Warner JE, Cuevas J, Bomarito GF, Leser PE, Leser WP (2020) Inverse estimation of elastic modulus using physics-informed generative adversarial networks. arXiv preprint arXiv:2006.05791. https://doi.org/10.48550/arXiv.2006.05791

  7. Adadi A, Berrada M (2018) Peeking inside the black-box: a survey on explainable artificial intelligence (xai). IEEE Access 6:52138–52160. https://doi.org/10.1109/ACCESS.2018.2870052

    Article  Google Scholar 

  8. Došilović FK, Brčić M, Hlupić N (2018) Explainable artificial intelligence: a survey. In: 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), pp 0210–0215. https://doi.org/10.23919/MIPRO.2018.8400040 . IEEE

  9. Rudin C, Radin J (2019) Why are we using black box models in AI when we don’t need to? a lesson from an explainable AI competition. Harvard Data Sci Rev. https://doi.org/10.1162/99608f92.5a8a3a3d

    Article  Google Scholar 

  10. Versino D, Tonda A, Bronkhorst CA (2017) Data driven modeling of plastic deformation. Comput Methods Appl Mech Eng 318:981–1004. https://doi.org/10.1016/j.cma.2017.02.016

    Article  MATH  Google Scholar 

  11. Bomarito G, Townsend T, Stewart K, Esham K, Emery J, Hochhalter J (2021) Development of interpretable, data-driven plasticity models with symbolic regression. Comput Struct 252:106557. https://doi.org/10.1016/j.compstruc.2021.106557

    Article  Google Scholar 

  12. Hernandez A, Balasubramanian A, Yuan F, Mason SA, Mueller T (2019) Fast, accurate, and transferable many-body interatomic potentials by symbolic regression. NPJ Comput Mater 5(1):1–11. https://doi.org/10.1038/s41524-019-0249-1

    Article  Google Scholar 

  13. Schmidt M, Lipson H (2009) Distilling free-form natural laws from experimental data. Science 324(5923):81–85. https://doi.org/10.1126/science.1165893

    Article  Google Scholar 

  14. Birky D, Garbrecht K, Emery J, Alleman C, Bomarito G, Hochhalter J (2023) Generalizing the Gurson model using symbolic regression and transfer learning to relax inherent assumptions. Modell Simul Mater Sci Eng. https://doi.org/10.1088/1361-651X/acfe28

    Article  Google Scholar 

  15. Garbrecht K, Birky D, Lester B, Emery J, Hochhalter J Complementing a continuum thermodynamic approach to constitutive modeling with symbolic regression. Available at SSRN 4552645

  16. Koza JR (1994) Genetic programming as a means for programming computers by natural selection. Stat Comput 4:87–112

    Article  Google Scholar 

  17. d’Ascoli S, Kamienny P-A, Lample G, Charton F (2022) Deep symbolic regression for recurrent sequences. arXiv preprint arXiv:2201.04600

  18. Xiong S, Wang W (2003) A new hybrid structure genetic programming in symbolic regression. In: The 2003 Congress on Evolutionary Computation, 2003. CEC’03., vol. 3, pp 1500–1506. IEEE

  19. Schmidt M, Lipson H (2010) Symbolic regression of implicit equations. In: Genetic Programming Theory and Practice VII, pp 73–85. Springer. https://doi.org/10.1007/978-1-4419-1626-6_5

  20. Wang Y, Wagner N, Rondinelli JM (2019) Symbolic regression in materials science. MRS Commun 9(3):793–805. https://doi.org/10.1557/mrc.2019.85

    Article  Google Scholar 

  21. NASA (2013) Bingo. GitHub. https://github.com/nasa/bingo

  22. Randall DL, Townsend TS, Hochhalter JD, Bomarito GF (2022) Bingo: a customizable framework for symbolic regression with genetic programming. In: Proceedings of the Genetic and Evolutionary Computation Conference Companion. GECCO ’22 Companion. ACM, Boston, MA, USA. https://doi.org/10.1145/3520304.3534031

  23. Mahfoud SW (1995) Niching methods for genetic algorithms. PhD thesis, University of Illinois at Urbana-Champaign

  24. Fernandezi F, SPEZZAN02 G, Tomassini M, Vanneschi L (2005) 6 parallel genetic programming. Parallel metaheuristics: a new class of algorithms, 127

  25. Whitley D, Rana S, Heckendorn RB (1999) The island model genetic algorithm: on separability, population size and convergence. J Comput Inf Technol 7(1):33–47

    Google Scholar 

  26. Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, Killeen T, Lin Z, Gimelshein N, Antiga L, Desmaison A, Kopf A, Yang E, DeVito Z, Raison M, Tejani A, Chilamkurthy S, Steiner B, Fang L, Bai J, Chintala S (2019) Pytorch: an imperative style, high-performance deep learning library. arXiv preprint arXiv:1912.01703. https://doi.org/10.48550/arXiv.1912.01703

  27. Kokhlikyan N, Miglani V, Martin M, Wang E, Reynolds J, Melnikov A, Lunova N, Reblitz-Richardson O (2019) PyTorch Captum. GitHub

    Google Scholar 

  28. Virtanen P, Gommers R, Oliphant TE, Haberland M, Reddy T, Cournapeau D, Burovski E, Peterson P, Weckesser W, Bright J, van der Walt SJ, Brett M, Wilson J, Millman KJ, Mayorov N, Nelson ARJ, Jones E, Kern R, Larson E, Carey CJ, Polat İ, Feng Y, Moore EW, VanderPlas J, Laxalde D, Perktold J, Cimrman R, Henriksen I, Quintero EA, Harris CR, Archibald AM, Ribeiro AH, Pedregosa F, van Mulbregt P, SciPy 1.0 Contributors (2001) SciPy: Open source scientific tools for Python. http://www.scipy.org/

  29. De Melo VV, Fowler B, Banzhaf W (2015) Evaluating methods for constant optimization of symbolic regression benchmark problems. In: 2015 Brazilian Conference on Intelligent Systems (BRACIS), pp 25–30. https://doi.org/10.1109/BRACIS.2015.55. IEEE

  30. Evans LC (2010) Partial differential equations. Am Math Soc. https://doi.org/10.1090/gsm/019

    Article  MATH  Google Scholar 

  31. Han Q, Lin F (2011) Elliptic partial differential equations. Am Math Soc. https://doi.org/10.1090/cln/001

    Article  MATH  Google Scholar 

  32. Meurer A, Smith CP, Paprocki M, Čertík O, Kirpichev SB, Rocklin M, Kumar A, Ivanov S, Moore JK, Singh S, Rathnayake T, Vig S, Granger BE, Muller RP, Bonazzi F, Gupta H, Vats S, Johansson F, Pedregosa F, Curry MJ, Terrel AR, Roučka V, Saboo A, Fernando I, Kulal S, Cimrman R, Scopatz A (2017) SymPy: symbolic computing in Python. https://docs.sympy.org/

Download references

Acknowledgements

The support and resources from the Center for High Performance Computing at the University of Utah are gratefully acknowledged. This research was sponsored in part by the Army Research Laboratory (ARL) under Cooperative Agreement Number W911NF-12-2-0023 and by Sandia National Laboratories under Agreement 2262518. The second and fourth authors were partially supported by the Defense Advanced Research Projects Agency (DARPA) through the Transformative Design (TRADES) program under the award HR0011-17-2-0016. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of ARL or the US Government. The US Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein.

Author information

Authors and Affiliations

  1. Department of Mechanical Engineering, University of Utah, Salt Lake City, USA

    Hongsup Oh & Jacob Hochhalter

  2. Scientific Computing and Imaging Institute, University of Utah, Salt Lake City, USA

    Roman Amici & Robert M. Kirby

  3. School of Computing, University of Utah, Salt Lake City, USA

    Roman Amici, Shandian Zhe, Robert M. Kirby & Jacob Hochhalter

  4. NASA Langley Research Center, Hampton, USA

    Geoffrey Bomarito

Authors
  1. Hongsup Oh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Roman Amici
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Geoffrey Bomarito
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shandian Zhe
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Robert M. Kirby
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jacob Hochhalter
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jacob Hochhalter.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Oh, H., Amici, R., Bomarito, G. et al. Inherently interpretable machine learning solutions to differential equations. Engineering with Computers (2023). https://doi.org/10.1007/s00366-023-01915-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00366-023-01915-7

Keywords

Search

Navigation