Abstract
Standard neural networks can approximate general nonlinear operators, represented either explicitly by a combination of mathematical operators, e.g. in an advection–diffusion reaction partial differential equation, or simply as a black box, e.g. a system-of-systems. The first neural operator was the Deep Operator Network (DeepONet) proposed in 2019 based on rigorous approximation theory. Since then, a few other less general operators have been published, e.g. based on graph neural networks or Fourier transforms. For black-box systems, training of neural operators is data driven only but if the governing equations are known they can be incorporated into the loss function during training to develop physics-informed neural operators. Neural operators can be used as surrogates in design problems, uncertainty quantification, autonomous systems, and almost any application requiring real-time inference. Moreover, independently pre-trained DeepONets can be used as components of a complex multi-physics system by coupling them together with relatively light training. Here, we present a review of DeepONet, the Fourier neural operator, and the graph neural operator, as well as appropriate extensions with feature expansions, and highlight their usefulness in diverse applications in computational mechanics, including porous media, fluid mechanics, and solid mechanics.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Cai S, Wang Z, Lu L, Zaki TA, Karniadakis GE (2021) Deepm&mnet: Inferring the electroconvection multiphysics fields based on operator approximation by neural networks. J Comput Phys 436:110296
Cai S, Wang Z, Wang S, Perdikaris P, Karniadakis GE (2021) Physics-informed neural networks for heat transfer problems. J Heat Transf 143(6)
Chamberlain B, Rowbottom J, Gorinova MI, Bronstein M, Webb S, Rossi E (2021) Grand: Graph neural diffusion. In: International conference on machine learning. PMLR, pp 1407–1418
Chen T, Chen H (1995) Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems. IEEE Trans Neural Netw 6(4):911–917
Chen T, Chen H (1995) Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems. IEEE Trans Neural Netw 6(4):911–917
De Hoop M, Huang DZ, Qian E, Stuart AM (2022) The cost-accuracy trade-off in operator learning with neural networks. arXiv:2203.13181
De Ryck T, Mishra S (2022) Generic bounds on the approximation error for physics-informed (and) operator learning. arXiv:2205.11393
De S, Hassanaly M, Reynolds M, King RN, Doostan A (2022) Bi-fidelity modeling of uncertain and partially unknown systems using deeponets. arXiv:2204.00997
Deng B, Shin Y, Lu L, Zhang Z, Karniadakis GE (2022) Approximation rates of deeponets for learning operators arising from advection-diffusion equations. Neural Netw
GarcĂa Trillos N, Gerlach M, Hein M, SlepÄŤev D (2020) Error estimates for spectral convergence of the graph laplacian on random geometric graphs toward the laplace–beltrami operator. Found Comput Math 20(4):827–887
Geelen R, Wright S, Willcox K (2022) Operator inference for non-intrusive model reduction with nonlinear manifolds. arXiv:2205.02304
Gilmer J, Schoenholz SS, Riley PF, Vinyals O, Dahl GE (2017) Neural message passing for quantum chemistry. In: International conference on machine learning. PMLR, pp 1263–1272
Goswami S, Anitescu C, Rabczuk T (2019) Adaptive phase field analysis with dual hierarchical meshes for brittle fracture. Eng Fract Mech 218:106608
Goswami S, Anitescu C, Rabczuk T (2020a) Adaptive fourth-order phase field analysis for brittle fracture. Comput Methods Appl Mech Eng 361:112808
Goswami S, Anitescu C, Rabczuk T (2020b) Adaptive fourth-order phase field analysis using deep energy minimization. Theor Appl Fract Mech 107:102527
Goswami S, Anitescu C, Chakraborty S, Rabczuk T (2020c) Transfer learning enhanced physics informed neural network for phase-field modeling of fracture. Theor Appl Fract Mech 106:102447
Goswami S, Yin M, Yu Y, Karniadakis GE (2022d) A physics-informed variational deeponet for predicting crack path in quasi-brittle materials. Comput Methods Appl Mech Eng 391:114587
Goswami S, Kontolati K, MD, Karniadakis GE (2022e) Deep transfer learning for partial differential equations under conditional shift with deeponet. arXiv:2204.09810
Goswami S, Li DS, Rego BV, Latorre M, Humphrey JD, Karniadakis GE (2022f) Neural operator learning of heterogeneous mechanobiological insults contributing to aortic aneurysms. arXiv:2205.03780
Gupta G, Xiao X, Bogdan P (2021) Multiwavelet-based operator learning for differential equations. Adv Neural Inf Process Syst 34:24048–24062
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778
Howard AA, Perego M, Karniadakis GE, Stinis P (2022) Multifidelity deep operator networks. arXiv:2204.09157
Jin P, Meng S, Lu L (2022) Mionet: Learning multiple-input operators via tensor product. arXiv:2202.06137
Karniadakis GE, Kevrekidis IG, Lu L, Perdikaris P, Wang S, Yang L (2021) Physics-informed machine learning. Nature Reviews. Physics 3(6):422–440
King R, Glaws A, Geraci G, Eldred MS (2020) A probabilistic approach to estimating wind farm annual energy production with bayesian quadrature. In: AIAA scitech 2020 forum, p 1951
Kipf TN, Welling M (2016) Semi-supervised classification with graph convolutional networks. arXiv:1609.02907
Kissas G, Seidman J, Guilhoto LF, Preciado VM, Pappas GJ, Perdikaris P (2022) Learning operators with coupled attention. arXiv:2201.01032
Kontolati K, Goswami S, Shields MD, Karniadakis GE (2022) On the influence of over-parameterization in manifold based surrogates and deep neural operators. arXiv:2203.05071
Konuk T, Shragge J (2021) Physics-guided deep learning using fourier neural operators for solving the acoustic vti wave equation. In: 82nd EAGE annual conference and exhibition, European association of geoscientists and engineers, pp 1–5
Kovachki N, Lanthaler S, Mishra S (2021a) On universal approximation and error bounds for fourier neural operators. J Mach Learn Res 22:Art–No
Kovachki N, Li Z, Liu B, Azizzadenesheli K, Bhattacharya K, Stuart A, Anandkumar A (2021b) Neural operator: Learning maps between function spaces. arXiv:2108.08481
Lanthaler S, Mishra S, Karniadakis GE (2022) Error estimates for deeponets: A deep learning framework in infinite dimensions. Trans Math Its Appl 6(1):tnac001
Li Z (2020a) Graph based neural operators. https://github.com/zongyi-li/graph-pde
Li Z, Kovachki N, Azizzadenesheli K, Liu B, Bhattacharya K, Stuart A, Anandkumar A (2020b) Fourier neural operator for parametric partial differential equations. arXiv:2010.08895
Li Z, Kovachki N, Azizzadenesheli K, Liu B, Bhattacharya K, Stuart A, Anandkumar A (2020c) Neural operator: Graph kernel network for partial differential equations. arXiv:2003.03485
Li Z, Kovachki N, Azizzadenesheli K, Liu B, Stuart A, Bhattacharya K, Anandkumar A (2020d) Multipole graph neural operator for parametric partial differential equations. Adv Neural Inf Process Syst 33:6755–6766
Liu N, Yu Y, You H, Tatikola N (2022) Ino: Invariant neural operators for learning complex physical systems with momentum conservation. Under Review
Li Z, Zheng H, Kovachki N, Jin D, Chen H, Liu B, Azizzadenesheli K, Anandkumar A (2021) Physics-informed neural operator for learning partial differential equations. arXiv:2111.03794
Lu L, Jin P, Pang G, Zhang Z, Karniadakis GE (2021) Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nat Mach Intell 3(3):218–229
Lu L, Meng X, Cai S, Mao Z, Goswami S, Zhang Z, Karniadakis GE (2022a) A comprehensive and fair comparison of two neural operators (with practical extensions) based on fair data. Comput Methods Appl Mech Eng 393:114778
Lu L, Pestourie R, Johnson SG, Romano G (2022b) Multifidelity deep neural operators for efficient learning of partial differential equations with application to fast inverse design of nanoscale heat transport. arXiv:2204.06684
Marcati C, Schwab C (2021) Exponential convergence of deep operator networks for elliptic partial differential equations. arXiv:2112.08125
McClenny L, Braga-Neto U (2020) Self-adaptive physics-informed neural networks using a soft attention mechanism. arXiv:2009.04544
Meng X, Guo Z (2015) Multiple-relaxation-time lattice Boltzmann model for incompressible miscible flow with large viscosity ratio and high PĂ©clet number. Phys Rev E 92(4):043305
Oommen V, Shukla K, Goswami S, Dingreville R, Karniadakis GE (2022) Learning two-phase microstructure evolution using neural operators and autoencoder architectures. arXiv:2204.07230
Raissi M, Perdikaris P, Karniadakis GE (2019) Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J Comput Phys 378:686–707
Riffaud S, Bergmann M, Farhat C, Grimberg S, Iollo A (2021) The dgdd method for reduced-order modeling of conservation laws. J Comput Phys 437:110336
Ruthotto L, Haber E (2019) Deep neural networks motivated by partial differential equations. J Math Imaging Vis 1–13
Samaniego E, Anitescu C, Goswami S, Nguyen-Thanh VM, Guo H, Hamdia K, Zhuang X, Rabczuk T (2020) An energy approach to the solution of partial differential equations in computational mechanics via machine learning: Concepts, implementation and applications. Comput Methods Appl Mech Eng 362:112790
Shukla K, Xu M, Trask N, Karniadakis GE (2022) Scalable algorithms for physics-informed neural and graph networks. Data-Centric Eng 3
Tripura T, Chakraborty S (2022) Wavelet neural operator: a neural operator for parametric partial differential equations. arXiv:2205.02191
Veličković P, Cucurull G, Casanova A, Romero A, Lio P, Bengio Y (2017) Graph attention networks. arXiv:1710.10903
Wang S, Wang H, Perdikaris P (2021) Learning the solution operator of parametric partial differential equations with physics-informed deeponets. Sci Adv 7(40):eabi8605
Yin M, Ban E, Rego BV, Zhang E, Cavinato C, Humphrey JD, Em Karniadakis G (2020) Simulating progressive intramural damage leading to aortic dissection using deeponet: an operator–regression neural network. J R Soc Interface 19(187):20210670
Yin M, Zhang E, Yu Y, Karniadakis GE (2022) Interfacing finite elements with deep neural operators for fast multiscale modeling of mechanics problems. Comput Methods Appl Mech Eng 115027
You H, Yu Y, D’Elia M, Gao T, Silling S (2022a) Nonlocal kernel network (NKN): a stable and resolution-independent deep neural network. arXiv:2201.02217
You H, Yu Y, Silling S, D’Elia M (2021b) Data-driven learning of nonlocal models: from high-fidelity simulations to constitutive laws. In: Accepted in AAAI spring symposium. MLPS
You H, Yu Y, Silling S, D’Elia M (2022c) A data-driven peridynamic continuum model for upscaling molecular dynamics. Comput Methods Appl Mech Eng 389:114400
You H, Yu Y, Trask N, Gulian M, D’Elia M (2021d) Data-driven learning of robust nonlocal physics from high-fidelity synthetic data. Comput Methods Appl Mech Eng 374:113553
You H, Zhang Q, Ross CJ, Lee C-H, Hsu M-C, Yu Y (2022e) A physics-guided neural operator learning approach to model biological tissues from digital image correlation measurements. arXiv:2204.00205
You H, Zhang Q, Ross CJ, Lee C-H, Yu Y (2022f) Learning deep implicit fourier neural operators (IFNOs) with applications to heterogeneous material modeling, To appear on Comput Methods Appl Mech Eng
Yu A, Becquey C, Halikias D, Mallory ME, Townsend A (2021) Arbitrary-depth universal approximation theorems for operator neural networks. arXiv:2109.11354
Zhang E, Dao M, Karniadakis GE, Suresh S (2022) Analyses of internal structures and defects in materials using physics-informed neural networks. Sci Adv 8(7):eabk0644
Zhu Y, Zabaras N, Koutsourelakis P-S, Perdikaris P (2019) Physics-constrained deep learning for high-dimensional surrogate modeling and uncertainty quantification without labeled data. J Comput Phys 394:56–81
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Goswami, S., Bora, A., Yu, Y., Karniadakis, G.E. (2023). Physics-Informed Deep Neural Operator Networks. In: Rabczuk, T., Bathe, KJ. (eds) Machine Learning in Modeling and Simulation. Computational Methods in Engineering & the Sciences. Springer, Cham. https://doi.org/10.1007/978-3-031-36644-4_6
Download citation
DOI: https://doi.org/10.1007/978-3-031-36644-4_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-36643-7
Online ISBN: 978-3-031-36644-4
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)