Abstract
The hierarchical deep-learning neural network (HiDeNN) is systematically developed through the construction of structured deep neural networks (DNNs) in a hierarchical manner, and a special case of HiDeNN for representing Finite Element Method (or HiDeNN-FEM in short) is established. In HiDeNN-FEM, weights and biases are functions of the nodal positions, hence the training process in HiDeNN-FEM includes the optimization of the nodal coordinates. This is the spirit of r-adaptivity, and it increases both the local and global accuracy of the interpolants. By fixing the number of hidden layers and increasing the number of neurons by training the DNNs, rh-adaptivity can be achieved, which leads to further improvement of the accuracy for the solutions. The generalization of rational functions is achieved by the development of three fundamental building blocks of constructing deep hierarchical neural networks. The three building blocks are linear functions, multiplication, and inversion. With these building blocks, the class of deep learning interpolation functions are demonstrated for interpolation theories such as Lagrange polynomials, NURBS, isogeometric, reproducing kernel particle method, and others. In HiDeNN-FEM, enrichment functions through the multiplication of neurons is equivalent to the enrichment in standard finite element methods, that is, generalized, extended, and partition of unity finite element methods. Numerical examples performed by HiDeNN-FEM exhibit reduced approximation error compared with the standard FEM. Finally, an outlook for the generalized HiDeNN to high-order continuity for multiple dimensions and topology optimizations are illustrated through the hierarchy of the proposed DNNs.
Similar content being viewed by others
References
Deng L, Dong Yu (2014) Deep learning: methods and applications. Found Trends Signal Process 7(3–4):197–387
Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press, Cambridge
Hornik K, Stinchcombe M, White H et al (1989) Multilayer feedforward networks are universal approximators. Neural Netw 2(5):359–366
Cybenko G (1989) Approximation by superpositions of a sigmoidal function. Math Control Signals Syst 2(4):303–314
LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444
Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504–507
Lee H, Kang IS, and (1990) Neural algorithm for solving differential equations. J Comput Phys 91(1):110–131
Meade AJ Jr, Fernandez AA (1994) The numerical solution of linear ordinary differential equations by feedforward neural networks. Math Comput Model 19(12):1–25
Lagaris IE, Likas A, Fotiadis DI (1998) Artificial neural networks for solving ordinary and partial differential equations. IEEE Trans Neural Netw 9(5):987–1000
Wu JL, Wang JX, Xiao H, Ling J (2017) A priori assessment of prediction confidence for data-driven turbulence modeling. Flow Turbul Combust 99:25–46
Xiao H, Wu JL, Wang JX, Sun R, Roy CJ (2016) Quantifying and reducing model-form uncertainties in reynolds-averaged navier-stokes simulations: a data-driven, physics-informed bayesian approach. J Comput Phys 324:115–136
Weinan E, Han J, Jentzen A (2017) Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations. Commun Math Stat 5(4):349–380
Berg J, Nyström K (2018) A unified deep artificial neural network approach to partial differential equations in complex geometries. Neurocomputing 317:28–41
Raissi M, Perdikaris P, Karniadakis GE (2017) Physics informed deep learning (part i): data-driven solutions of nonlinear partial differential equations. arXiv preprint arXiv:1711.10561
Raissi M, Perdikaris P, Karniadakis GE (2019) Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J Comput Phys 378:686–707
Sirignano J, Spiliopoulos K (2018) DGM: a deep learning algorithm for solving partial differential equations. J Comput Phys 375:1339–1364
Hughes TJR (2012) The finite element method: linear static and dynamic finite element analysis. Courier Corporation, Mineola
Belytschko T, Liu WK, Moran B, Elkhodary K (2013) Nonlinear finite elements for continua and structures. Wiley, New York
Höllig K (2003) Finite element methods with B-splines. SIAM, Philadelphia
Rogers DF (2000) An introduction to NURBS: with historical perspective. Elsevier, Amsterdam
Hughes TJR, Cottrell JA, Bazilevs Y (2005) Isogeometric analysis: CAD, finite elements, NURBS, exact geometry and mesh refinement. Comput Methods Appl Mech Eng 194(39):4135–4195
Schillinger D, Dede L, Scott MA, Evans JA, Borden MJ, Rank E, Hughes TJR (2012) An isogeometric design-through-analysis methodology based on adaptive hierarchical refinement of NURBS, immersed boundary methods, and T-spline CAD surfaces. Comput Methods Appl Mech Eng 249:116–150
Cottrell JA, Hughes TJR, Bazilevs Y (2009) Isogeometric analysis: toward integration of CAD and FEA. Wiley, New York
Liu WK, Jun S, Zhang YF (1995) Reproducing kernel particle methods. Int J Numer Methods Fluids 20(8–9):1081–1106
Chen JS, Pan C, Cheng-Tang W, Liu WK (1996) Reproducing kernel particle methods for large deformation analysis of non-linear structures. Comput Methods Appl Mech Eng 139(1–4):195–227
Li S, Liu WK (2002) Meshfree and particle methods and their applications. Appl Mech Rev 55(1):1–34
Duarte CA, Babuška I, Oden JT (2000) Generalized finite element methods for three-dimensional structural mechanics problems. Comput Struct 77(2):215–232
Belytschko T, Moës N, Usui S, Parimi C (2001) Arbitrary discontinuities in finite elements. Int J Numer Methods Eng 50(4):993–1013
Melenk JM, Babuška I (1996) The partition of unity finite element method: basic theory and applications. In Research Report/Seminar für Angewandte Mathematik, volume 1996. Eidgenössische Technische Hochschule, Seminar für Angewandte Mathematik
Zhang L, Tang S, Liu WK (2020) Analytical expression of RKPM shape functions. Comput Mech 1–10
Chilimbi T, Suzue Y, Apacible J, Kalyanaraman K (2014) Project adam: building an efficient and scalable deep learning training system. In: 11th USENIX symposium on operating systems design and implementation (OSDI 14), 571–582
ABAQUS/Standard User’s Manual (2016). Dassault Systèmes Simulia Corp, United States
Weinan E, Bing Yu (2018) The deep ritz method: a deep learning-based numerical algorithm for solving variational problems. Commun Math Stat 6(1):1–12
Bendsoe MP, Sigmund O (2013) Topology optimization: theory, methods, and applications. Springer, New York
Acknowledgements
R. Domel is supported by Research Experience for Undergraduate supplement to U.S. National Science Foundation (NSF) grant number CMMI-1762035 (summer 2019). W. K. Liu and H.Y. Li are also supported by the same NSF grant. L. Cheng, J. Gao, and C. Yu conducted this research as an unfunded, exploratory (interdisciplinary) collaboration between Northwestern University and Peking University. L. Zhang, Y. Yang, and S. Tang are supported by National Natural Science Foundation of China grant number 11832001, 11988102 and 11890681. We would like to acknowledge the nice comments from Dr. Ye Lu.
Author information
Authors and Affiliations
Corresponding authors
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Reno Domel: Summer 2019 undergraduate research intern at Department of Mechanical Engineering, Northwestern University.
Rights and permissions
About this article
Cite this article
Zhang, L., Cheng, L., Li, H. et al. Hierarchical deep-learning neural networks: finite elements and beyond. Comput Mech 67, 207–230 (2021). https://doi.org/10.1007/s00466-020-01928-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00466-020-01928-9