# Non-intrusive *Sparse Subspace Learning* for Parametrized Problems

- 391 Downloads
- 5 Citations

## Abstract

We discuss the use of hierarchical collocation to approximate the numerical solution of parametric models. With respect to traditional projection-based reduced order modeling, the use of a collocation enables non-intrusive approach based on sparse adaptive sampling of the parametric space. This allows to recover the low-dimensional structure of the parametric solution subspace while also *learning* the functional dependency from the parameters in explicit form. A sparse low-rank approximate tensor representation of the parametric solution can be built through an incremental strategy that only needs to have access to the output of a deterministic solver. Non-intrusiveness makes this approach straightforwardly applicable to challenging problems characterized by nonlinearity or non affine weak forms. As we show in the various examples presented in the paper, the method can be interfaced with no particular effort to existing third party simulation software making the proposed approach particularly appealing and adapted to practical engineering problems of industrial interest.

## Notes

### Acknowledgements

The authors of the paper would like to acknowledge Jean-Louis Duval, Jean-Christophe Allain and Julien Charbonneaux from the ESI group for the support and data for crash and stamping simulations.

### Compliance with Ethical Standards

### Conflict of interest

The authors declare that they have no conflict of interest.

### Informed Consent

All the authors are informed and provided their consent.

### Research Involving Human and Animal Participants

The research does not involve neither human participants nor animals.

## References

- 1.Zorriassatine F, Wykes C, Parkin R, Gindy N (2003) A survey of virtual proto-typing techniques for mechanical product development. Proc Inst Mech Eng Part B J Eng Manuf 217(4):513–530CrossRefGoogle Scholar
- 2.Oden JT, Belytschko T, Fish J, Hughes TJR, Johnson C, Keyes D, Laub A, Petzold L, Srolovitz D, Yip S (2006) Simulation-based engineering science: revolutionizing engineering science through simulation. Report of the NSF Blue Ribbon Panel on Simulation-Based Engineering Science. National Science Foundation, ArlingtonGoogle Scholar
- 3.Glotzer SC, Kim S, Cummings PT, Deshmukh A, Head-Gordon M, Karniadakis G, Petzold L, Sagui C, Shinozuka M (2009) International assessment of research and development in simulation-based engineering and science. Panel Report. World Technology Evaluation Center Inc, BaltimoreGoogle Scholar
- 4.Bellman RE (2003) Dynamic programming. Courier Dover Publications, New York (
**Republished edition**)zbMATHGoogle Scholar - 5.Montgomery DC (2013) Design and analysis of experiments, 8th edn. Wiley, HobokenGoogle Scholar
- 6.Chong EKP, Zak SH (2013) An introduction to optimization, 4th edn. Wiley series on discrete mathematics and optimization. Wiley, HobokenzbMATHGoogle Scholar
- 7.Antoulas A, Sorensen DC, Gugercin S (2001) A survey of model reduction methods for large-scale systems. Contemp Math 280:193–220MathSciNetCrossRefzbMATHGoogle Scholar
- 8.Bialecki RA, Kassab AJ, Fic A (2005) Proper orthogonal decomposition and modal analysis for acceleration of transient FEM thermal analysis. Int J Numer Method Eng 62(6):774–797CrossRefzbMATHGoogle Scholar
- 9.Quarteroni A, Manzoni A, Negri E (2015) Reduced basis methods for partial differential equations: an introduction. Modeling and simulation in science, engineering and technology, 1st edn. Springer, BaselGoogle Scholar
- 10.Huynh DBP, Rozza G, Sen S, Patera AT (2007) A successive constraint linear optimization method for lower bounds of parametric coercivity and infsup stability constants. C R Math 345(8):473–478zbMATHGoogle Scholar
- 11.Daversin C, Prud’homme C (2015) Simultaneous empirical interpolation and reduced basis method for non-linear problems. C R Math 353(12):1105–1109MathSciNetzbMATHGoogle Scholar
- 12.Barrault M, Maday Y, Nguyen NC, Patera AT (2004) An “empirical interpolation method”: application to efficient reduced-basis discretization of partial differential equations. C R Math 339(9):667–672MathSciNetCrossRefzbMATHGoogle Scholar
- 13.Farhat C, Chapman T, Avery P (2015) Structure-preserving, stability, and accuracy properties of the energy-conserving sampling and weighting method for the hyper reduction of nonlinear finite element dynamic models. Int J Numer Method Eng 102:1077–1110MathSciNetCrossRefzbMATHGoogle Scholar
- 14.Chaturantabut S, Sorensen DC (2010) Nonlinear model order reduction via discrete empirical interpolation. SIAM J Sci Comput 32(5):2737–2764MathSciNetCrossRefzbMATHGoogle Scholar
- 15.Grepl MA, Maday Y, Nguyen NC, Patera AT (2007) Efficient reduced-basis treatment of non-affine and nonlinear partial differential equations. ESAIM Math Model Numer Anal 41(3):575–605CrossRefzbMATHGoogle Scholar
- 16.Maday Y, Nguyen NC, Patera AT, Pau SH (2009) A general multipurpose interpolation procedure: the magic points. CPPA 8(1):383–404MathSciNetzbMATHGoogle Scholar
- 17.Ryckelynck D (2005) A priori hypereduction method: an adaptive approach. J Comput Phys 202(1):346–366CrossRefzbMATHGoogle Scholar
- 18.Amsallem D, Farhat C (2011) An online method for interpolating linear parametric reduced-order models. SIAM J Sci Comput 33(5):2169–2198MathSciNetCrossRefzbMATHGoogle Scholar
- 19.Hernández J, Caicedo MA, Ferrer A (2017) Dimensional hyper-reduction of nonlinear finite element models via empirical cubature. Comput Method Appl Mech Eng 313:687–722MathSciNetCrossRefGoogle Scholar
- 20.Rapún ML, Terragni F, Vega JM (2017) Lupod: collocation in POD via LU decomposition. J Comput Phys 335:1–20MathSciNetCrossRefzbMATHGoogle Scholar
- 21.Kumar D, Raisee M, Lacor C (2016) An efficient non-intrusive reduced basis model for high dimensional stochastic problems in CFD. Comput Fluids 138:67–82MathSciNetCrossRefzbMATHGoogle Scholar
- 22.Prulière E, Chinesta F, Ammar A (2010) On the deterministic solution of multidimensional parametric models using the proper generalized decomposition. Math Comput Simul 81(4):791–810MathSciNetCrossRefzbMATHGoogle Scholar
- 23.Hackbusch W (2012) Tensor spaces and numerical tensor calculus, 1st edn. Springer Series in Computational Mathematics. Springer, BerlinCrossRefzbMATHGoogle Scholar
- 24.Grasedyck L, Kressner D, Tobler C (2013) A literature survey of low-rank tensor approximation techniques. GAMM-Mitt 36:53–78. arXiv:1302.7121
- 25.Kolda TG, Bader BW (2009) Tensor decompositions and applications. SIAM Rev 51(3):455–500MathSciNetCrossRefzbMATHGoogle Scholar
- 26.Ammar A, Mokdad B, Chinesta F, Keunings R (2007) A new family of solvers for some classes of multidimensional partial differential equations encountered in kinetic theory modeling of complex fluids. Part II: transient simulation using space-time separated representations. J Non-Newtonian Fluid Mech 144(2–3):98–121CrossRefzbMATHGoogle Scholar
- 27.Nouy A (2010) A priori model reduction through proper generalized decomposition for solving time-dependent partial differential equations. Comput Method Appl Mech Eng 199:1603–1626MathSciNetCrossRefzbMATHGoogle Scholar
- 28.Chinesta F, Leygue A, Bordeu F, Aguado JV, Cueto E, Gonzalez D, Alfaro I, Ammar A, Huerta A (2013) PGD-based computational vademecum for efficient design, optimization and control. Arch Comput Methods Eng 20(1):31–59MathSciNetCrossRefzbMATHGoogle Scholar
- 29.Chinesta F, Ladevèze P, Cueto E (2011) A short review on model order reduction based on proper generalized decomposition. Arch Comput Methods Eng 18(4):395–404CrossRefGoogle Scholar
- 30.Ammar A, Chinesta F, Díez P, Huerta A (2010) An error estimator for separated representations of highly multidimensional models. Comput Method Appl Mech Eng 199(25–28):1872–1880MathSciNetCrossRefzbMATHGoogle Scholar
- 31.Uschmajew A (2012) Local convergence of the alternating least squares algorithm for canonical tensor approximation. SIAM J Matrix Anal Appl 33(2):639–652MathSciNetCrossRefzbMATHGoogle Scholar
- 32.Quesada C, Alfaro I, Gonzalez D, Cueto E, Chinesta F (2014) PGD-based model reduction for surgery simulation: solid dynamics and contact detection. Lect Notes Comput Sci 8789:193–202CrossRefGoogle Scholar
- 33.Aguado JV, Borzacchiello D, Ghnatios C, Lebel F, Upadhyay R, Binetruy C, Chinesta F (2017) A simulation app based on reduced order modeling for manufacturing optimization of composite outlet guide vanes. Adv Model Simul Eng Sci 4(1):1CrossRefGoogle Scholar
- 34.Borzacchiello D, Aguado JV, Chinesta F (2016) Reduced order modelling for efficient numerical optimisation of a hot-wall Chemical Vapour Deposition reactor. Int J Numer Method Heat Fluid Flow 27(4). doi: 10.1108/HFF-04-2016-0153
- 35.Ghnatios Ch, Masson F, Huerta A, Leygue A, Cueto E, Chinesta F (2012) Proper generalized decomposition based dynamic data-driven control of thermal processes. Comput Method Appl Mech Eng 213–216:29–41CrossRefGoogle Scholar
- 36.Cohen A, DeVore R (2015) Approximation of high-dimensional parametric PDEs. Acta Numer 24:1–159MathSciNetCrossRefzbMATHGoogle Scholar
- 37.Bachmayr M, Cohen A, Dahmen W (2016) Parametric PDEs: sparse or low-rank approximations? arXiv:1607.04444
- 38.Boyd JP (2001) Chebyshev and Fourier spectral methods. Courier Corporation, Ann ArborzbMATHGoogle Scholar
- 39.Candès E, Romberg J (2007) Sparsity and incoherence in compressive sampling. Inverse Probl 23(3):969MathSciNetCrossRefzbMATHGoogle Scholar
- 40.Gilbert A, Indyk P (2010) Sparse recovery using sparse matrices. Proc IEEE 98(6):937–947CrossRefGoogle Scholar
- 41.Donoho DL (2006) Compressed sensing. IEEE Trans Inform Theor 52(4):1289–1306MathSciNetCrossRefzbMATHGoogle Scholar
- 42.Chen SS, Donoho DL, Saunders MA (2001) Atomic decomposition by basis pursuit. SIAM Rev 43(1):129–159MathSciNetCrossRefzbMATHGoogle Scholar
- 43.Tibshirani R (1996) Regression shrinkage and selection via the lasso. J R Stat Soc Series B Methodol 58:267–288MathSciNetzbMATHGoogle Scholar
- 44.Zou H, Hastie T (2005) Regularization and variable selection via the elastic net. J R Stat Soc Series B Stat Methodol 67(2):301–320MathSciNetCrossRefzbMATHGoogle Scholar
- 45.Brunton SL, Proctor JL, Kutz JN (2016) Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proc Natl Acad Sci USA 113(15):3932–3937MathSciNetCrossRefzbMATHGoogle Scholar
- 46.Bungartz HJ, Griebel M (2004) Sparse grids. Acta Numer 13:147–269MathSciNetCrossRefzbMATHGoogle Scholar
- 47.Nobile F, Tempone R, Webster CG (2008) A sparse grid stochastic collocation method for partial differential equations with random input data. SIAM J Numer Anal 46(5):2309–2345MathSciNetCrossRefzbMATHGoogle Scholar
- 48.Pflüger D, Peherstorfer B, Bungartz HJ (2010) Spatially adaptive sparse grids for high-dimensional data-driven problems. J Complex 26(5):508–522MathSciNetCrossRefzbMATHGoogle Scholar
- 49.Gerstner T, Griebel M (2003) Dimension-adaptive tensor-product quadrature. Computing 71(1):65–87MathSciNetCrossRefzbMATHGoogle Scholar
- 50.Nobile F, Tempone R, Webster CG (2008) An anisotropic sparse grid stochastic collocation method for partial differential equations with random input data. SIAM J Numer Anal 46(5):2411–2442MathSciNetCrossRefzbMATHGoogle Scholar
- 51.Golub GH, Van Loan CF (1996) Matrix computations, 3rd edn. The Johns Hopkins University Press, BaltimorezbMATHGoogle Scholar
- 52.Halko N, Martinsson PG, Tropp JA (2011) Finding structure with randomness: probabilistic algorithms for constructing approximate matrix decompositions. SIAM Rev 53(2):217–288MathSciNetCrossRefzbMATHGoogle Scholar
- 53.Harshman RA (1970) Foundations of the parafac procedure: models and conditions for an “explanatory” multi-modal factor analysis. UCLA Working Papers in PhoneticsGoogle Scholar
- 54.Carroll JD, Chang J (1970) Analysis of individual differences in multidimensional scaling via an n-way generalization of eckart-young decomposition. Psychometrika 35(3):283–319CrossRefzbMATHGoogle Scholar
- 55.Li Y (2004) On incremental and robust subspace learning. Pattern Recognit 37(7):1509–1518CrossRefzbMATHGoogle Scholar
- 56.Zhao H, Yuen PC, Kwok JT (2006) A novel incremental principal component analysis and its application for face recognition. IEEE Trans Syst Man Cybern Part B 36(4):873–886CrossRefGoogle Scholar
- 57.Sobral A, Baker CG, Bouwmans T, Zahzah E (2014) Incremental and multi-feature tensor subspace learning applied for background modeling and subtraction. In International conference image analysis and recognition. Springer, New York. pp. 94–103Google Scholar
- 58.Brand M (2002) Incremental singular value decomposition of uncertain data with missing values. Computer Vision ECCV 2002, pp. 707–720Google Scholar
- 59.Quarteroni A, Rozza G (2007) Numerical solution of parametrized navier-stokes equations by reduced basis methods. Numer Methods Partial Differ Equ 23(4):923–948MathSciNetCrossRefzbMATHGoogle Scholar
- 60.Canuto C, Hussaini MY, Quarteroni A, Zang TA Jr (2012) Spectral methods in fluid dynamics. Springer, BerlinzbMATHGoogle Scholar
- 61.De Lathauwer L, De Moor B, Vanderwalle J (2000) A multilinear singular value decomposition. SIAM J Matrix Anal Appl 21(4):1253–1278MathSciNetCrossRefzbMATHGoogle Scholar
- 62.Smoljak SA (1963) Quadrature and interpolation formulae on tensor products of certain function classes. Dokl Akad Nauk SSSR 148(5):1042–1045 (
**Transl.: Soviet Math Dokl 4:240–243, 1963**)MathSciNetGoogle Scholar - 63.Gerstner T, Griebel M (1998) Numerical integration using sparse grids. Numer Algorithm 18(3):209–232MathSciNetCrossRefzbMATHGoogle Scholar
- 64.Dauge M, Stevenson R (2010) Sparse tensor product wavelet approximation of singular functions. SIAM J Math Anal 42(5):2203–2228MathSciNetCrossRefzbMATHGoogle Scholar
- 65.Garcke J (2007) A dimension adaptive sparse grid combination technique for machine learning. ANZIAM J 48:725–740MathSciNetCrossRefzbMATHGoogle Scholar
- 66.Dũng D, Temlyakov VN, Ullrich T (2016) Hyperbolic cross approximation. arXiv:1601.03978
- 67.Quarteroni A, Rozza G, Manzoni A (2011) Certified reduced basis approximation for parametrized partial differential equations and applications. J Math Indus 1(1):3MathSciNetCrossRefzbMATHGoogle Scholar
- 68.Bordeu E (2013) Pxdmf : aA file format for separated variables problems version 1.6. Technical report, Ecole Centrale de NantesGoogle Scholar
- 69.Chen P, Quarteroni A, Rozza G (2014) Comparison between reduced basis and stochastic collocation methods for elliptic problems. J Sci Comput 59(1):187–216MathSciNetCrossRefzbMATHGoogle Scholar
- 70.Peherstorfer B, Zimmer S, Bungartz HJ (2012) Model reduction with the reduced basis method and sparse grids. Sparse grids and applications. Springer, Berlin, pp. 223–242CrossRefGoogle Scholar