Abstract
Physics-informed neural networks (PINN) has emerged as a promising approach for solving partial differential equations (PDEs). However, the training process for PINN can be computationally expensive, limiting its practical applications. To address this issue, we investigate several acceleration techniques for PINN that combine Fourier neural operators, separable PINN, and first-order PINN. We also propose novel acceleration techniques based on second-order PINN and Koopman neural operators. We evaluate the efficiency of these techniques on various PDEs, and our results show that the hybrid models can provide much more accurate results than classical PINN under time constraints for the training, making PINN a more viable option for practical applications. The proposed methodology in the manuscript is generic and can be extended on a larger set of problems including inverse problems.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
Not applicable
Code availability
Not applicable
References
Balay, S. (2018). PETSc users manual: Revision 3.10. Technical Report. https://doi.org/10.2172/1483828
Basir, S., & Senocak, I. (2022). Critical investigation of failure modes in physics-informed neural networks. https://doi.org/10.2514/6.2022-2353. arXiv:2206.09961.
Berg, J., & Nyström, K. (2017). A unified deep artificial neural network approach to partial differential equations in complex geometries. https://doi.org/10.1016/j.neucom.2018.06.056. arXiv:1711.06464.
Cai, S., Wang, Z., Wang, S., Perdikaris, P., & Karniadakis, G. E. (2021). Physics-informed neural networks for heat transfer problems. Journal of Heat Transfer, 143(6), 060801. https://doi.org/10.1115/1.4050542
Chen, Y., Lu, L., Karniadakis, G. E., & Negro, L. D. (2019). Physics-informed neural networks for inverse problems in nano-optics and metamaterials. arXiv:1912.01085 [physics.comp-ph].
Chen, X., Gong, C., Wan, Q., Deng, L., Wan, Y., Liu, Y., Chen, B., & Liu, J. (2021). Transfer learning for deep neural network-based partial differential equations solving. Advances in Aerodynamics, 3(1), 1–14. https://doi.org/10.1186/s42774-021-00094-7
Cho, J., Nam, S., Yang, H., Yun, S.-B., Hong, Y., & Park, E. (2022). Separable PINN: Mitigating the curse of dimensionality in physics-informed neural networks. arXiv:2211.08761 [cs.LG].
DOLFINx. (2017). https://github.com/FEniCS/dolfinx. accessed: 12.11.2022.
Gladstone, R. J., Nabian, M. A., & Meidani, H. (2022). FO-PINNs: A first-order formulation for physics informed neural networks.
Glorot, X., & Bengio, Y. (2010) Understanding the difficulty of training deep feedforward neural networks. In Teh, Y. W., Titterington, M. (eds.) Proceedings of the thirteenth international conference on artificial intelligence and statistics. Proceedings of machine learning research, (vol. 9, pp. 249–256). PMLR, Chia Laguna Resort, Sardinia, Italy (2010). https://proceedings.mlr.press/v9/glorot10a.html.
Goswami, S., Bora, A., Yu, Y., & Karniadakis, G. E. (2022). Physics-informed deep neural operator networks.
Goswami, S., Anitescu, C., Chakraborty, S., & Rabczuk, T. (2020). Transfer learning enhanced physics informed neural network for phase-field modeling of fracture. Theoretical and Applied Fracture Mechanics, 106, 102447. https://doi.org/10.1016/j.tafmec.2019.102447
Grossmann, T. G., Komorowska, U. J., Latz, J., & Schönlieb, C.-B. (2023). Can physics-informed neural networks beat the finite element method? arXiv. https://doi.org/10.48550/ARXIV.2302.04107.
Gupta, J.K., & Brandstetter, J. (2022). Towards multi-spatiotemporal-scale generalized pde modeling. arXiv preprint arXiv:2209.15616.
Hennigh, O., Narasimhan, S., Nabian, M.A., Subramaniam, A., Tangsali, K., Fang, Z., Rietmann, M., Byeon, W., & Choudhry, S. (2021). NVIDIA SimNet™: An AI-accelerated multi-physics simulation framework. In Computational science – ICCS 2021, (pp. 447–461). Springer. https://doi.org/10.1007/978-3-030-77977-1_36
Huang, X., Alkhalifah, T., & Song, C. (2021). A modified physics-informed neural network with positional encoding. In First international meeting for applied geoscience: energy expanded abstracts. Society of exploration geophysicists. https://doi.org/10.1190/segam2021-3584127.1.
Jiang, X., Wang, D., Chen, X., & Zhang, M. (2022). Physics-informed neural network for optical fiber parameter estimation from the nonlinear Schrödinger equation. Journal of Lightwave Technology. https://doi.org/10.1109/jlt.2022.3199782
Kollmannsberger, S., D’Angella, D., Jokeit, M., & Herrmann, L. (2021). Physics-informed neural networks. Deep learning in computational mechanics (Vol. 977, pp. 55–84). Springer. https://doi.org/10.1007/978-3-030-76587-3_5
Konuk, T., & Shragge, J. (2021). Physics-guided deep learning using fourier neural operators for solving the acoustic VTI wave equation. In 82nd EAGE annual conference exhibition. European Association of Geoscientists & Engineers. https://doi.org/10.3997/2214-4609.202113304.
Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A., & Anandkumar, A. (2020). Fourier neural operator for parametric partial differential equations. arXiv. https://doi.org/10.48550/ARXIV.2010.08895 .
Li, Z., Zheng, H., Kovachki, N., Jin, D., Chen, H., Liu, B., Azizzadenesheli, K., & Anandkumar, A. (2021). Physics-informed neural operator for learning partial differential equations.
Lu, L., Jin, P., & Karniadakis, G .E. (2019) Deeponet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators https://doi.org/10.1038/s42256-021-00302-5. arXiv:1910.03193.
Markidis, S. (2021). The old and the new: Can physics-informed deep-learning replace traditional linear solvers? Frontiers in Big Data. https://doi.org/10.3389/fdata.2021.669097
Mishra, S., & Molinaro, R. (2021). Physics informed neural networks for simulating radiative transfer. Journal of Quantitative Spectroscopy and Radiative Transfer, 270, 107705. https://doi.org/10.1016/j.jqsrt.2021.107705
Pathak, J., Subramanian, S., Harrington, P., Raja, S., Chattopadhyay, A., Mardani, M., Kurth, T., Hall, D., Li, Z., Azizzadenesheli, K., Hassanzadeh, P., Kashinath, K., & Anandkumar, A. (2022). FourCastNet: A global data-driven high-resolution weather model using adaptive fourier neural operators.
Raissi, M., Perdikaris, P., & Karniadakis, G. E. (2019). Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378, 686–707.
Ronneberger, O., Fischer, P., & Brox, T. (2015). U-Net: Convolutional networks for biomedical image segmentation.
Ryck, T.D., Jagtap, A.D., & Mishra, S. (2022). Error estimates for physics informed neural networks approximating the Navier-Stokes equations.
Sharma, R., & Shankar, V. (2022). Accelerated training of physics-informed neural networks (PINNs) using meshless discretizations. arXiv. https://doi.org/10.48550/ARXIV.2205.09332 .
Tang, H., Yang, H., Liao, Y., Xie, L. (2021) A transfer learning enhanced the physics-informed neural network model for vortex-induced vibration. arXiv. https://doi.org/10.48550/ARXIV.2112.14448 .
Vorst, H. A. (2003). Iterative Krylov methods for large linear systems. New York: Cambridge University Press.
Wang, S., Wang, H., & Perdikaris, P. (2021). On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks. Computer Methods in Applied Mechanics and Engineering, 384, 113938. https://doi.org/10.1016/j.cma.2021.113938
Weinan, & Yu, B., (2018). The deep Ritz method: A deep learning-based numerical algorithm for solving variational problems. Communication in Mathematics Statitics, 6(1), 1–12. https://doi.org/10.1007/s40304-018-0127-z
Wen, G., Li, Z., Azizzadenesheli, K., Anandkumar, A., & Benson, S. M. (2021). U-FNO–an enhanced Fourier neural operator-based deep-learning model for multiphase flow.
Wight, C. L., & Zhao, J. (2020). Solving Allen–Cahn and Cahn–Hilliard equations using the adaptive physics informed neural networks.
Wong, J. C., Ooi, C., Gupta, A., & Ong, Y.-S. (2022). Learning in sinusoidal spaces with physics-informed neural networks. IEEE Transactions on Artificial Intelligence. https://doi.org/10.1109/tai.2022.3192362
Wu, P., Pan, K., Ji, L., Gong, S., Feng, W., Yuan, W., & Pain, C. (2022). Navier–stokes generative adversarial network: a physics-informed deep learning model for fluid flow generation. Neural Computing and Applications, 34(14), 11539–11552. https://doi.org/10.1007/s00521-022-07042-6
Xiong, W., Huang, X., Zhang, Z., Deng, R., Sun, P., & Tian, Y. (2023). Koopman neural operator as a mesh-free solver of non-linear partial differential equations.
Xu, Z.-Q. J. (2018) Frequency principle in deep learning with general loss functions and its potential application. arXiv. https://doi.org/10.48550/ARXIV.1811.10146 .
Xu, Z., Guo, Y., & Saleh, J. H. (2022). A physics-informed dynamic deep autoencoder for accurate state-of-health prediction of lithium-ion battery. Neural Computing and Applications, 34(18), 15997–16017. https://doi.org/10.1007/s00521-022-07291-5
Funding
Not applicable
Author information
Authors and Affiliations
Contributions
JG supervised the project. FB and JG conceived the original idea. FB and IC designed the model and the computational framework and analysed the data. EK proposed the FEM experiment in discussions.
Corresponding author
Ethics declarations
Conflicts of interest
All authors declare that they have no conflicts of interest.
Ethical approval
Not applicable
Consent to participate
Not applicable
Consent for publication
Not applicable
Additional information
Editors: Vu Nguyen, Dani Yogatama.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Buzaev, F., Gao, J., Chuprov, I. et al. Hybrid acceleration techniques for the physics-informed neural networks: a comparative analysis. Mach Learn (2023). https://doi.org/10.1007/s10994-023-06442-6
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10994-023-06442-6