This chapter contains a literature survey of the work published by the authors in the timeframe of their collaboration, where the concept presented in this book have been applied to real-life industrial settings, and new methodologies have been developed.

The listed contributions are grouped in the following themes: linear manifold learning in Sect. 6.1, nonlinear dimensionality reduction via auto-encoder in Sect. 6.2, piecewise linear dimensionality reduction via dictionary-based ROM-nets in Sect. 6.3 and manifold learning of physics problems assisted by black-box regressors in Sect. 6.4.

6.1 Linear Manifold Learning

A priori hyper-reduction method: an adaptive approach Model reduction methods are usually based on offline preliminar simulations to build the shape functions of the reduced order model (ROM) before the computation of the reduced state variables. They are a posteriori approaches. Most of the time these offline computations are as complex as the simulation which we want to simplify by the ROM. The a priori reduction method proposed in [17] avoids such preliminary computations. It is an a priori approach based on the analysis of some state evolutions, such that all the state evolutions needed to perform the model reduction are described by an approximate ROM. The ROM and the state evolution are simultaneously improved by the method, thanks to an adaptive strategy. An initial set of known shape functions can be used to define the ROM to adapt, but it is not necessary. The adaptive procedure includes extensions of the subspace spanned by the shape functions of the ROM and selections of the most relevant shape functions in order to represent the state evolution. The hyper-reduction is achieved by selecting a part of the integration points of the finite element model to forecast the evolution of the reduced state variables. In this method, both the number of degrees of freedom and the number of integration points are reduced.

A nonintrusive distributed reduced-order modeling framework for nonlinear structural mechanics—Application to elastoviscoplastic computations In [8] is proposed a framework that constructs reduced-order models for nonlinear structural mechanics in a nonintrusive fashion and can handle large-scale simulations. Three steps are carried out: (i) the production of high-fidelity solutions by commercial software, (ii) the offline stage of the model reduction, and (iii) the online stage where the reduced-order model is exploited. The nonintrusivity assumes that only the displacement field solution is known, and the proposed framework carries out operations on these simulation data during the offline phase. The compatibility with a new commercial code only needs the implementation of a routine converting the discretized solution into the used data format. The nonintrusive capabilities of the framework are demonstrated on numerical experiments using commercial versions of Z-set and Ansys Mechanical. The nonlinear constitutive equations are evaluated by using an external plugin. The large-scale simulations are handled using domain decomposition and parallel computing with distributed memory. The features and performances of the framework are evaluated on two numerical applications involving elastoviscoplastic materials: the second one involves a model of high-pressure blade, where the framework is used to extrapolate cyclic loadings in 6.5 h, whereas the reference high-fidelity computation would take 9.5 days.

Fast computation of transient thermal profiling of high-pressure compressors under non-parametrized boundary conditions variability In [9], a transient thermal problem is considered, with a nonlinear term coming from the radiation boundary condition and a nonparametrized variability in the form complex scenarios for the initial condition and the convection coefficients and external temperatures. A posteriori reduced order modeling by snapshot Proper Orthogonal Decomposition is used. To treat the nonlinearity, hyperreduction is required since precomputing the polynomial nonlinearities becomes too expensive for the radiation term. The Empirical Cubature Method, originally proposed for nonlinear structural mechanics, is derived for these equations. The method is applied to the design of high-pressure compressors for civilian aircraft engines, where a fast evaluation of the solution temperature is required when testing new configurations. When using in the reduced solver the same model as the one from the high-fidelity code, the approximation is very accurate. However, when using a commercial code to generate the high-fidelity data, where the implementation of the model and solver is unknown, the reduced model is less accurate but still within engineering tolerances. Hence, the regularizing property of reduced order models, together with a nonintrusive approach, enables the use of commercial software to generate the data, even under some degree of uncertainty in the proprietary model or solver of the commercial software.

Time Stable Reduced Order Modeling by an Enhanced Reduced Order Basis of the Turbulent and Incompressible 3D Navier–Stokes Equations In [3], the problem of constructing a time stable reduced order model of the 3D turbulent and incompressible Navier–Stokes equations is considered. The lack of stability associated with the order reduction methods of the Navier–Stokes equations is a well-known problem and, in general, it is very difficult to account for different scales of a turbulent flow in the same reduced space. To remedy this problem, a new stabilization technique is proposed, based on an a priori enrichment of the classical proper orthogonal decomposition (POD) modes with dissipative modes associated with the gradient of the velocity fields. The main idea is to be able to do an a priori analysis of different modes in order to arrange a POD basis in a different way, which is defined by the enforcement of the energetic dissipative modes within the first orders of the reduced order basis. This enables the modeling of the production and dissipation of the turbulent kinetic energy (TKE) in a separate fashion within the high ranked new velocity modes, hence to ensure good stability of the reduced order model. The importance of this a priori enrichment of the reduced basis is illustrated on a typical aeronautical injector with Reynolds number of 45,000. This order reduction technique is able to recover large scale features for very long integration times (25 ms in the present case). Moreover, the reduced order model exhibits periodic fluctuations with a period of 2.2 ms corresponding to the time scale of the precessing vortex core (PVC) associated with this test case.

An updated Gappy-POD to capture non-parameterized geometrical variation in fluid dynamics problems In [4], a method is proposed to fill the gap within an incomplete turbulent and incompressible data field, while satisfying the topological and intensity changes of the fluid flow after a non-parameterized geometrical variation in the fluid domain. A single baseline large eddy simulation (LES) is assumed to be performed prior geometrical variations. The method enhances the Gappy-POD method proposed by Everson and Sirovich in 1995, in the case where the given set of empirical eigenfunctions is not sufficient and is not interpolant for the recovering of the modal coefficients for each Gappy snapshot by a least squares procedure. This is typically the case when we introduce non-parameterized geometrical modifications in the fluid domain. Here, after the baseline simulation, additional solutions of the incompressible Navier–Stokes equations are solely performed over a restricted fluid domain, that contains the geometrical modifications. These local LESs, called hybrid simulations, are performed by using the immersed boundary technique, which uses of a fluid boundary condition and the baseline velocity field. Then, the POD modes are updated using a local modification of the baseline POD modes in the restricted fluid domain. Furthermore, a physical correction of the latter enhanced Gappy-POD modal coefficients is obtained thanks to a Galerkin projection of the Navier–Stokes equations upon the new modes of the available data. This enhancement procedure on the global velocity reconstruction by the physical constraint was tested on a 3D semi-industrial test case of a typical aeronautical injection system and, a 2D laminar and unsteady incompressible test case. The speed-up relative to this new technique is equal to 100, which allows the fast exploration of two new designs of the aeronautical injection system.

6.2 Nonlinear Dimensionality Reduction via Auto-Encoder

Data-Targeted Prior Distribution for Variational AutoEncoder In [1], Variational AutoEncoders (VAE) are used to study unsteady and compressible fluid flows in aircraft engines. Inferential methods enable the computation of sharp approximations of the posterior probability of the parameters of the VAE using the transient dynamics of the training velocity fields, and the generatation of plausible velocity fields. An important application is the initialization of such transient simulations. It is known by the Bayes theorem that the choice of the prior distribution is very important for the computation of the posterior probability, proportional to the product of likelihood with the prior probability. A new inference model is proposed, based on a new prior defined by the density estimate with the realizations of the kernel proper orthogonal decomposition coefficients of the available training data. It is illustrated that this inference model improves the results obtained with the usual standard normal prior distribution. This new generative approach can also be seen as an improvement of the kernel proper orthogonal decomposition method, for which we do not usually have a robust technique for expressing the pre-image in the input physical space of the stochastic reduced field in the feature high-dimensional space with a kernel inner product.

A Bayesian Nonlinear Reduced Order Modeling Using Variational AutoEncoders In [2], a new nonlinear projection based model reduction using convolutional VAEs. This framework is applied on transient incompressible flows. The accuracy is obtained thanks to the expression of the velocity and pressure fields in a nonlinear manifold maximising the likelihood on pre-computed data in the offline stage. A confidence interval is obtained for each time instant thanks to the definition of the reduced dynamic coefficients as independent random variables for which the posterior probability given the offline data is known. The parameters of the nonlinear manifold are optimized as the ones of the decoder layers of an autoencoder. The parameters of the conditional posterior probability of the reduced coefficients are the ones of the encoder layers of the same autoencoder. This reduced-order model is not a regression model over the offline pre-computed data. The numerical resolution of the ROM is based on the Chorin projection method. This new nonlinear projection-based reduced order modeling is applied to a 2D Karman Vortex street flow and a 3D incompressible and unsteady flow in an aeronautical injection system.

Deep multimodal autoencoder for crack criticality assessment In continuum mechanics, the prediction of defect harmfulness requires to solve approximately partial differential equations with given boundary conditions. In [14], boundary conditions are learnt for tight local volumes (TLV) surrounding cracks in three-dimensional volumes. A nonparametric data-driven approach is used to define the space of defects, by considering defects observed via X-Ray computed tomography. The dimension of the ambient space for the observed images of defects is huge. A nonlinear dimensionality reduction scheme is proposed in order to train a reduced latent space for both the morphology of defects and their local mechanical effects in the TLV. A multimodal autoencoder enables to mix morphological and mechanical data. It contains a single latent space, termed mechanical latent space. But this latent space is fed by two encoders. One is related to the images of defects and the other to mechanical fields in the TLV. The latent variables are input variables for a geometrical decoder and for a mechanical decoder. In this work, mechanical variables are displacement fields. The autoencoder on mechanical variables enables projection-based model order reduction as proposed in the study of Lee and Carlberg. The main novelty of this work is a submodeling approach assisted by artificial intelligence. Here, for defect images in the test set, Dirichlet boundary conditions are applied to TLV. These boundary conditions are forecasted by the mechanical decoder with a latent vector predicted by the morphological encoder. For that purpose, a mapping is trained to convert morphological latent variables into mechanical latent variables, denoted “direct mapping”. An “inverse mapping” is also trained for error estimation with respect to morphological predictions. Errors on mechanical predictions are close to 5% with simulation speed-up ranging for 3 to 120. It is illustrated that latent variables forecasted by the images of defects are prone to a better understanding of the predictions.

6.3 Piecewise Linear Dimensionality Reduction via Dictionary-Based ROM-Nets

Model order reduction assisted by deep neural networks (ROM-net) In [12], a general framework is proposed for projection-based model order reduction assisted by deep neural networks. The proposed methodology, called ROM-net, consists in using deep learning techniques to adapt the reduced-order model to a stochastic input tensor whose nonparametrized variabilities strongly influence the quantities of interest for a given physics problem. In particular, the concept of dictionary-based ROM-nets is introduced, where deep neural networks recommend a suitable local reduced-order model from a dictionary. The dictionary of local reduced-order models is constructed from a clustering of simplified simulations enabling the identification of the subspaces in which the solutions evolve for different input tensors. The training examples are represented by points on a Grassmann manifold, on which distances are computed for clustering. This methodology is applied to an anisothermal elastoplastic problem in structural mechanics, where the damage field depends on a random temperature field. When using deep neural networks, the selection of the best reduced-order model for a given thermal loading is 60 times faster than when following the clustering procedure used in the training phase.

Mechanical dissimilarity of defects in welded joints via Grassmann manifold and machine learning Assessing the harmfulness of defects based on images is becoming more and more common in industry. At present, these defects can be inserted in digital twins that aim to replicate in a mechanical model what is observed on a component so that an image-based diagnosis can be further conducted. However, the variety of defects, the complexity of their shape, and the computational complexity of finite element models related to their digital twin make this kind of diagnosis too slow for any practical application. In [18], a classification of observed defects enables the definition of a dictionary of digital twins. These digital twins prove to be representative of model-reduction purposes while preserving an acceptable accuracy for stress prediction. Nonsupervised machine learning is used for both the classification issue and the construction of reduced digital twins. The dictionary items are medoids found by a k-medoids clustering algorithm. Medoids are assumed to be well distributed in the training dataset according to a metric or a dissimilarity measurement. A new dissimilarity measurement between defects is proposed. It is theoretically founded according to approximation errors in hyper-reduced predictions. In doing so, defect classes are defined according to their mechanical effect and not directly according to their morphology. In practice, each defect in the training dataset is encoded as a point on a Grassmann manifold. This methodology is evaluated through a test set of observed defects totally different from the training dataset of defects used to compute the dictionary of digital twins. The most appropriate item in the dictionary for model reduction is selected according to an error indicator related to the hyper-reduced prediction of stresses. No plasticity effect is considered here (merely isotropic elastic materials), which is a strong assumption but which is not critical for the purpose of this work. In spite of the large variety of defects, accurate predictions of stresses for most of defects in the test set are obtained.

Multimodal data augmentation for digital twining assisted by artificial intelligence in mechanics of materials Digital twins in the mechanics of materials usually involve multimodal data in the sense that an instance of a mechanical component has both experimental and simulated data. These simulations aim not only to replicate experimental observations but also to extend the data. Whether spatially, temporally, or functionally, augmentation is needed for various possible uses of the components to improve the predictions of mechanical behavior. Related multimodal data are scarce, high-dimensional and a physics-based causality relation exists between observational and simulated data. In [5], a data augmentation scheme coupled with data pruning is proposed, in order to limit memory requirements for high-dimensional augmented data. This augmentation is desirable for digital twining assisted by artificial intelligence when performing nonlinear model reduction. Here, data augmentation aims at preserving similarities in terms of the validity domain of reduced digital twins. A specimen subjected to a mechanical test at high temperature is considered, where the as-manufactured geometry may impact the lifetime of the component. Hence, an instance is represented by a digital twin that includes 3D X-Ray tomography data of the specimen, the related finite element mesh, and the finite element predictions of thermo-mechanical variables at several time steps. There is, thus, for each specimen, geometrical and mechanical information. Multimodal data, which couple different representation modalities together, are hard to collect, and annotating them requires a significant effort. Thus, the analysis of multimodal data generally suffers from the problem of data scarcity. The proposed data augmentation scheme aims at training a recommending system that recognizes a category of data available in a training set that has already been fully analyzed by using high-fidelity models. Such a recommending system enables the use of a ROM-net for fast lifetime assessment via local reduced-order models.

Real-Time Data Assimilation in Welding Operations Using Thermal Imaging and Accelerated Digital Twinning Welding operations may be subjected to different types of defects when the process is not properly controlled and most defect detection is done a posteriori. The mechanical variables that are at the origin of these imperfections are often not observable in situ. In [16], an offline/online data assimilation approach is proposed, that allows for joint parameter and state estimations based on local probabilistic surrogate models and thermal imaging in real-time. Offline, the surrogate models are built from a high-fidelity thermomechanical finite element parametric study of the weld. The online estimations are obtained by conditioning the local models by the observed temperature and known operational parameters, thus fusing high-fidelity simulation data and experimental measurements.

Mechanical assessment of defects in welded joints: morphological classification and data augmentation In [15], a methodology is developed for classifying defects based on their morphology and induced mechanical response. The proposed approach is fairly general and relies on morphological operators and spherical harmonic decomposition as a way to characterize the geometry of the pores, and on the Grassman distance evaluated on FFT-based computations, for the predicted elastic response. The approach is implemented and detailed on a set of trapped gas pores observed in X-ray tomography of welded joints, that significantly alter the mechanical reliability of these materials. The space of morphological and mechanical responses is first partitioned into clusters using the “k-medoids” criterion and associated distance functions. Second, multiple-layer perceptron neuronal networks are used to associate a defect and corresponding morphological representation to its mechanical response. It is found that the method provides accurate mechanical predictions if the training data contains a sufficient number of defects representing each mechanical class. To do so, the original set of defects is supplemented by data augmentation techniques. Artificially-generated pore shapes are obtained using the spherical harmonic decomposition and a singular value decomposition performed on the pores signed distance transform.

Data Augmentation and Feature Selection for Automatic Model Recommendation in Computational Physics Classification algorithms have recently found applications in computational physics for the selection of numerical methods or models adapted to the environment and the state of the physical system. For such classification tasks, labeled training data come from numerical simulations and generally correspond to physical fields discretized on a mesh. Three challenging difficulties arise: the lack of training data, their high dimensionality, and the non-applicability of common data augmentation techniques to physics data. In [13], two algorithms are introduced to address these issues: one for dimensionality reduction via feature selection, and one for data augmentation. These algorithms are combined with a wide variety of classifiers for their evaluation. When combined with a stacking ensemble made of six multilayer perceptrons and a ridge logistic regression, they enable reaching an accuracy of 90% on the considered classification problem of nonlinear structural mechanics.

Physics-informed cluster analysis and a priori efficiency criterion for the construction of local reduced-order bases Nonlinear model order reduction has opened the door to parameter optimization and uncertainty quantification in complex physics problems governed by nonlinear equations. In particular, the computational cost of solving these equations can be reduced by means of local reduced-order bases. In [11], the benefits of a physics-informed cluster analysis for the construction of cluster-specific reduced-order bases are examined. The choice of the dissimilarity measure for clustering is fundamental and highly affects the performances of the local reduced-order bases. It is shown that clustering with an angle-based dissimilarity on simulation data efficiently decreases the intra-cluster Kolmogorov N-width. Additionally, an a priori efficiency criterion is introduced to assess the relevance of ROM-nets. This criterion also provides engineers with a very practical method for ROM-nets’ hyperparameters calibration under constrained computational costs for the training phase. On five different physics problems, the physics-informed clustering strategy significantly outperforms classic strategies for the construction of local reduced-order bases in terms of projection errors.

6.4 Extension: Manifold Learning of Physics Problems Assisted by Black-Box Regressors

A priori compression of convolutional neural networks for wave simulators Convolutional Neural Networks (CNNs) are seeing widespread use in a variety of fields, including image classification, facial and object recognition, medical imaging analysis, and many more. In addition, there are applications such as physics-informed simulators in which accurate forecasts in real time with a minimal lag are required. The current neural network designs include millions of parameters, which makes it difficult to install such complex models on devices that have limited memory. Compression techniques might be able to resolve these issues by decreasing the size of CNN models that are created by reducing the number of parameters that contribute to the complexity of the models. In [7], a compressed tensor format of convolutional layer is proposed a priori, before the training of the neural network, for finite element (FE) predictions of physical data. 3-way kernels or 2-way kernels in convolutional layers are replaced by one-way fiters. The overfitting phenomena will be reduced also. The time needed to make predictions or time required for training using the original CNN model would be cut significantly if there were fewer parameters to deal with. The priori compressed models is validated on physical data from a FE model solving a 2D wave equation. Tn the considered application, the proposed convolutional compression technique achieves equivalent performance in the prediction error as classical convolutional layers with fewer trainable parameters (around 20%) and lower memory footprint.

Accelerated uncertainty quantification in impact simulations using generative adversarial networks and submodeling The analysis of parametric and non-parametric uncertainties of very large dynamical systems requires the construction of a stochastic model of said system. Linear approaches relying on random matrix theory and principal component analysis can be used when systems undergo low-frequency vibrations. In the case of fast dynamics and wave propagation, a random generator of boundary conditions for fast submodels by using machine learning is investigated in [6]. It is illustrated that the use of non-linear techniques in machine learning and data-driven methods is highly relevant. Physics-Informed Neural Networks (PINNs) are a possible choice for a data-driven method to replace linear modal analysis. An architecture that supports a random component is necessary for the construction of the stochastic model of the physical system for non-parametric uncertainties, since the goal is to learn the underlying probabilistic distribution of uncertainty in the data. Generative Adversarial Networks (GANs) are suited for such applications, where the Wasserstein-GAN with gradient penalty variant offers improved convergence results for the considered problem. The objective of this approach is to train a GAN on data from a finite element method code so as to extract stochastic boundary conditions for faster finite element predictions on a submodel. The submodel and the training data have both the same geometrical support. It is a zone of interest for uncertainty quantification and relevant to engineering purposes. In the exploitation phase, the framework can be viewed as a randomized and parametrized simulation generator on the submodel, which can be used as a Monte Carlo estimator.

MMGP: a Mesh Morphing Gaussian Process-based machine learning method for regression of physical problems under non-parameterized geometrical variability When learning simulations for modeling physical phenomena in industrial designs, geometrical variabilities are of prime interest. While classical regression techniques prove effective for parameterized geometries, practical scenarios often involve the absence of shape parametrization during the inference stage, providing only mesh discretizations as available data. Learning simulations from such mesh-based representations poses significant challenges, with recent advances relying heavily on deep graph neural networks to overcome the limitations of conventional machine learning approaches. Despite their promising results, graph neural networks exhibit certain drawbacks, including their dependency on extensive datasets and limitations in providing built-in predictive uncertainties or handling large meshes. In [10], a machine learning method that do not rely on graph neural networks is proposed. Complex geometrical shapes and variations with fixed topology are dealt with using well-known mesh morphing onto a common support, combined with classical dimensionality reduction techniques and Gaussian processes. The proposed methodology can easily deal with large meshes without the need for explicit shape parameterization and provides crucial predictive uncertainties, which are essential for informed decision-making. In the considered numerical experiments, the proposed method is competitive with respect to existing graph neural networks, regarding training efficiency and accuracy of the predictions.