Skip to main content
SpringerLink
Log in

Failure-Informed Adaptive Sampling for PINNs, Part II: Combining with Re-sampling and Subset Simulation

  • Original Paper
  • Published:
Communications on Applied Mathematics and Computation Aims and scope Submit manuscript

Abstract

This is the second part of our series works on failure-informed adaptive sampling for physic-informed neural networks (PINNs). In our previous work (SIAM J. Sci. Comput. 45: A1971–A1994), we have presented an adaptive sampling framework by using the failure probability as the posterior error indicator, where the truncated Gaussian model has been adopted for estimating the indicator. Here, we present two extensions of that work. The first extension consists in combining with a re-sampling technique, so that the new algorithm can maintain a constant training size. This is achieved through a cosine-annealing, which gradually transforms the sampling of collocation points from uniform to adaptive via the training progress. The second extension is to present the subset simulation (SS) algorithm as the posterior model (instead of the truncated Gaussian model) for estimating the error indicator, which can more effectively estimate the failure probability and generate new effective training points in the failure region. We investigate the performance of the new approach using several challenging problems, and numerical experiments demonstrate a significant improvement over the original algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22

Similar content being viewed by others

References

  1. Au, S.K., Beck, J.L.: Estimation of small failure probabilities in high dimensions by subset simulation. Probab. Eng. Mech. 16, 263–277 (2001)

    Article  Google Scholar 

  2. Bischof, R., Kraus, M.: Multi-objective loss balancing for physics-informed deep learning. arXiv:2110.09813 (2021)

  3. Daw, A., Bu, J., Wang, S., Perdikaris, P., Karpatne, A.: Rethinking the importance of sampling in physics-informed neural networks. arXiv:2207.02338 (2022)

  4. E, W.N.: Machine learning and computational mathematics. Commun. Comput. Phys. 28, 1639–1670 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  5. E, W.N., Yu, B.: The deep Ritz method: a deep learning-based numerical algorithm for solving variational problems. Commun. Math. Stat. 6, 1–12 (2018)

  6. Gao, Z., Yan, L., Zhou, T.: Failure-informed adaptive sampling for PINNs. SIAM J. Sci. Comput. 45, A1971–A1994 (2023)

    Article  MathSciNet  MATH  Google Scholar 

  7. Krishnapriyan, A., Gholami, A., Zhe, S., Kirby, R., Michael, W.M.: Characterizing possible failure modes in physics-informed neural networks. Adv. Neural Inform. Process. Syst. 34, 26548–26560 (2021)

    Google Scholar 

  8. Lu, L., Meng, X., Mao, Z., Karniadakis, G.E.: DeepXDE: a deep learning library for solving differential equations. SIAM Rev. 63, 208–228 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  9. McClenny, L., Braga-Neto, U.: Self-adaptive physics-informed neural networks using a soft attention mechanism. arXiv:2009.04544 (2020)

  10. Peng, W., Zhou, W., Zhang, X., Yao, W., Liu, Z.: RANG: a residual-based adaptive node generation method for physics-informed neural networks. arXiv:2205.01051 (2022)

  11. Raissi, M., Perdikaris, P., George, E.K.: Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys. 378, 686–707 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  12. Sirignano, J., Spiliopoulos, K.: DGM: a deep learning algorithm for solving partial differential equations. J. Comput. Phys. 375, 1339–1364 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  13. Subramanian, S., Kirby, R.M., Mahoney, M.W., Gholami, A.: Adaptive self-supervision algorithms for physics-informed neural networks. arXiv:2207.04084 (2022)

  14. Tang, K., Wan, X., Yang, C.: DAS: a deep adaptive sampling method for solving partial differential equations. arXiv:2112.14038 (2021)

  15. Wang, S., Sankaran, S., Perdikaris, P.: Respecting causality is all you need for training physics-informed neural networks. arXiv:2203.07404 (2022)

  16. Wang, S., Teng, Y., Perdikaris, P.: Understanding and mitigating gradient flow pathologies in physics-informed neural networks. SIAM J. Sci. Comput. 43, A3055–A3081 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  17. Wang, S., Yu, X.L., Perdikaris, P.: When and why PINNs fail to train: a neural tangent kernel perspective. J. Comput. Phys. 449, 110768 (2022)

    Article  MathSciNet  MATH  Google Scholar 

  18. Wu, C.X., Zhu, M., Tan, Q., Kartha, Y., Lu, L.: A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks. Comput. Methods Appl. Mech. Eng. 403, 115671 (2023)

    Article  MathSciNet  MATH  Google Scholar 

  19. Xiang, Z., Wei, P., Liu, X., Yao, W.: Self-adaptive loss balanced physics-informed neural networks. Neurocomputing 496, 11–34 (2022)

    Article  Google Scholar 

  20. Zang, Y., Bao, G., Ye, X., Zhou, H.: Weak adversarial networks for high-dimensional partial differential equations. J. Comput. Phys. 411, 109409 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  21. Zuev, K.: Subset simulation method for rare event estimation: an introduction. arXiv:1505.03506 (2015)

Download references

Author information

Authors and Affiliations

  1. School of Mathematics, Southeast University, Nanjing, 210096, Jiangsu, China

    Zhiwei Gao & Liang Yan

  2. Division of Science and Technology, BNU-HKBU United International College, Zhuhai, 519087, Guangdong, China

    Tao Tang

  3. Nanjing Center for Applied Mathematics, Nanjing, 211135, Jiangsu, China

    Liang Yan

  4. Institute of Computational Mathematics, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing, 100190, China

    Tao Zhou

Authors
  1. Zhiwei Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tao Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Liang Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tao Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tao Tang.

Additional information

LY’s work was supported by the NSF of China (No.12171085). This work was supported by the National Key R&D Program of China (2020YFA0712000), the NSF of China (No. 12288201), the Strategic Priority Research Program of Chinese Academy of Sciences (No. XDA25010404), and the Youth Innovation Promotion Association (CAS).

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gao, Z., Tang, T., Yan, L. et al. Failure-Informed Adaptive Sampling for PINNs, Part II: Combining with Re-sampling and Subset Simulation. Commun. Appl. Math. Comput. (2023). https://doi.org/10.1007/s42967-023-00312-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s42967-023-00312-7

Keywords

Mathematics Subject Classification

Search

Navigation