Skip to main content
SpringerLink
Log in

Hierarchical deep-learning neural networks: finite elements and beyond

  • Original Paper
  • Published:
Computational Mechanics Aims and scope Submit manuscript

Abstract

The hierarchical deep-learning neural network (HiDeNN) is systematically developed through the construction of structured deep neural networks (DNNs) in a hierarchical manner, and a special case of HiDeNN for representing Finite Element Method (or HiDeNN-FEM in short) is established. In HiDeNN-FEM, weights and biases are functions of the nodal positions, hence the training process in HiDeNN-FEM includes the optimization of the nodal coordinates. This is the spirit of r-adaptivity, and it increases both the local and global accuracy of the interpolants. By fixing the number of hidden layers and increasing the number of neurons by training the DNNs, rh-adaptivity can be achieved, which leads to further improvement of the accuracy for the solutions. The generalization of rational functions is achieved by the development of three fundamental building blocks of constructing deep hierarchical neural networks. The three building blocks are linear functions, multiplication, and inversion. With these building blocks, the class of deep learning interpolation functions are demonstrated for interpolation theories such as Lagrange polynomials, NURBS, isogeometric, reproducing kernel particle method, and others. In HiDeNN-FEM, enrichment functions through the multiplication of neurons is equivalent to the enrichment in standard finite element methods, that is, generalized, extended, and partition of unity finite element methods. Numerical examples performed by HiDeNN-FEM exhibit reduced approximation error compared with the standard FEM. Finally, an outlook for the generalized HiDeNN to high-order continuity for multiple dimensions and topology optimizations are illustrated through the hierarchy of the proposed DNNs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27
Fig. 28
Fig. 29
Fig. 30
Fig. 31
Fig. 32
Fig. 33
Fig. 34

Similar content being viewed by others

References

  1. Deng L, Dong Yu (2014) Deep learning: methods and applications. Found Trends Signal Process 7(3–4):197–387

    Article  MathSciNet  Google Scholar 

  2. Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press, Cambridge

    MATH  Google Scholar 

  3. Hornik K, Stinchcombe M, White H et al (1989) Multilayer feedforward networks are universal approximators. Neural Netw 2(5):359–366

    Article  Google Scholar 

  4. Cybenko G (1989) Approximation by superpositions of a sigmoidal function. Math Control Signals Syst 2(4):303–314

    Article  MathSciNet  Google Scholar 

  5. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444

    Article  Google Scholar 

  6. Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504–507

    Article  MathSciNet  Google Scholar 

  7. Lee H, Kang IS, and (1990) Neural algorithm for solving differential equations. J Comput Phys 91(1):110–131

    Article  MathSciNet  Google Scholar 

  8. Meade AJ Jr, Fernandez AA (1994) The numerical solution of linear ordinary differential equations by feedforward neural networks. Math Comput Model 19(12):1–25

    Article  MathSciNet  Google Scholar 

  9. Lagaris IE, Likas A, Fotiadis DI (1998) Artificial neural networks for solving ordinary and partial differential equations. IEEE Trans Neural Netw 9(5):987–1000

    Article  Google Scholar 

  10. Wu JL, Wang JX, Xiao H, Ling J (2017) A priori assessment of prediction confidence for data-driven turbulence modeling. Flow Turbul Combust 99:25–46

    Article  Google Scholar 

  11. Xiao H, Wu JL, Wang JX, Sun R, Roy CJ (2016) Quantifying and reducing model-form uncertainties in reynolds-averaged navier-stokes simulations: a data-driven, physics-informed bayesian approach. J Comput Phys 324:115–136

    Article  MathSciNet  Google Scholar 

  12. Weinan E, Han J, Jentzen A (2017) Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations. Commun Math Stat 5(4):349–380

    Article  MathSciNet  Google Scholar 

  13. Berg J, Nyström K (2018) A unified deep artificial neural network approach to partial differential equations in complex geometries. Neurocomputing 317:28–41

    Article  Google Scholar 

  14. Raissi M, Perdikaris P, Karniadakis GE (2017) Physics informed deep learning (part i): data-driven solutions of nonlinear partial differential equations. arXiv preprint arXiv:1711.10561

  15. Raissi M, Perdikaris P, Karniadakis GE (2019) Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J Comput Phys 378:686–707

    Article  MathSciNet  Google Scholar 

  16. Sirignano J, Spiliopoulos K (2018) DGM: a deep learning algorithm for solving partial differential equations. J Comput Phys 375:1339–1364

    Article  MathSciNet  Google Scholar 

  17. Hughes TJR (2012) The finite element method: linear static and dynamic finite element analysis. Courier Corporation, Mineola

    Google Scholar 

  18. Belytschko T, Liu WK, Moran B, Elkhodary K (2013) Nonlinear finite elements for continua and structures. Wiley, New York

    MATH  Google Scholar 

  19. Höllig K (2003) Finite element methods with B-splines. SIAM, Philadelphia

    Book  Google Scholar 

  20. Rogers DF (2000) An introduction to NURBS: with historical perspective. Elsevier, Amsterdam

    Google Scholar 

  21. Hughes TJR, Cottrell JA, Bazilevs Y (2005) Isogeometric analysis: CAD, finite elements, NURBS, exact geometry and mesh refinement. Comput Methods Appl Mech Eng 194(39):4135–4195

    Article  MathSciNet  Google Scholar 

  22. Schillinger D, Dede L, Scott MA, Evans JA, Borden MJ, Rank E, Hughes TJR (2012) An isogeometric design-through-analysis methodology based on adaptive hierarchical refinement of NURBS, immersed boundary methods, and T-spline CAD surfaces. Comput Methods Appl Mech Eng 249:116–150

    Article  MathSciNet  Google Scholar 

  23. Cottrell JA, Hughes TJR, Bazilevs Y (2009) Isogeometric analysis: toward integration of CAD and FEA. Wiley, New York

    Book  Google Scholar 

  24. Liu WK, Jun S, Zhang YF (1995) Reproducing kernel particle methods. Int J Numer Methods Fluids 20(8–9):1081–1106

    Article  MathSciNet  Google Scholar 

  25. Chen JS, Pan C, Cheng-Tang W, Liu WK (1996) Reproducing kernel particle methods for large deformation analysis of non-linear structures. Comput Methods Appl Mech Eng 139(1–4):195–227

    Article  MathSciNet  Google Scholar 

  26. Li S, Liu WK (2002) Meshfree and particle methods and their applications. Appl Mech Rev 55(1):1–34

    Article  Google Scholar 

  27. Duarte CA, Babuška I, Oden JT (2000) Generalized finite element methods for three-dimensional structural mechanics problems. Comput Struct 77(2):215–232

    Article  MathSciNet  Google Scholar 

  28. Belytschko T, Moës N, Usui S, Parimi C (2001) Arbitrary discontinuities in finite elements. Int J Numer Methods Eng 50(4):993–1013

    Article  Google Scholar 

  29. Melenk JM, Babuška I (1996) The partition of unity finite element method: basic theory and applications. In Research Report/Seminar für Angewandte Mathematik, volume 1996. Eidgenössische Technische Hochschule, Seminar für Angewandte Mathematik

  30. Zhang L, Tang S, Liu WK (2020) Analytical expression of RKPM shape functions. Comput Mech 1–10

  31. Chilimbi T, Suzue Y, Apacible J, Kalyanaraman K (2014) Project adam: building an efficient and scalable deep learning training system. In: 11th USENIX symposium on operating systems design and implementation (OSDI 14), 571–582

  32. ABAQUS/Standard User’s Manual (2016). Dassault Systèmes Simulia Corp, United States

  33. Weinan E, Bing Yu (2018) The deep ritz method: a deep learning-based numerical algorithm for solving variational problems. Commun Math Stat 6(1):1–12

    MathSciNet  MATH  Google Scholar 

  34. Bendsoe MP, Sigmund O (2013) Topology optimization: theory, methods, and applications. Springer, New York

    MATH  Google Scholar 

Download references

Acknowledgements

R. Domel is supported by Research Experience for Undergraduate supplement to U.S. National Science Foundation (NSF) grant number CMMI-1762035 (summer 2019). W. K. Liu and H.Y. Li are also supported by the same NSF grant. L. Cheng, J. Gao, and C. Yu conducted this research as an unfunded, exploratory (interdisciplinary) collaboration between Northwestern University and Peking University. L. Zhang, Y. Yang, and S. Tang are supported by National Natural Science Foundation of China grant number 11832001, 11988102 and 11890681. We would like to acknowledge the nice comments from Dr. Ye Lu.

Author information

Author notes

  1. Lei Zhang and Lin Cheng: Co-first authors.

Authors and Affiliations

  1. HEDPS and LTCS, College of Engineering, Peking University, Beijing, 100871, China

    Lei Zhang, Yang Yang & Shaoqiang Tang

  2. Department of Mechanical Engineering, Northwestern University, 2145 Sheridan Rd., Evanston, IL 60208-3111, USA

    Lin Cheng, Hengyang Li, Jiaying Gao, Cheng Yu & Wing Kam Liu

  3. University of Notre Dame, Notre Dame, IN 46556, USA

    Reno Domel

Authors
  1. Lei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lin Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hengyang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jiaying Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Cheng Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Reno Domel
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Yang Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Shaoqiang Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Wing Kam Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Shaoqiang Tang or Wing Kam Liu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Reno Domel: Summer 2019 undergraduate research intern at Department of Mechanical Engineering, Northwestern University.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, L., Cheng, L., Li, H. et al. Hierarchical deep-learning neural networks: finite elements and beyond. Comput Mech 67, 207–230 (2021). https://doi.org/10.1007/s00466-020-01928-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00466-020-01928-9

Keywords

Search

Navigation