Non-intrusive Sparse Subspace Learning for Parametrized Problems

  • Domenico BorzacchielloEmail author
  • José V. Aguado
  • Francisco Chinesta
Original Paper


We discuss the use of hierarchical collocation to approximate the numerical solution of parametric models. With respect to traditional projection-based reduced order modeling, the use of a collocation enables non-intrusive approach based on sparse adaptive sampling of the parametric space. This allows to recover the low-dimensional structure of the parametric solution subspace while also learning the functional dependency from the parameters in explicit form. A sparse low-rank approximate tensor representation of the parametric solution can be built through an incremental strategy that only needs to have access to the output of a deterministic solver. Non-intrusiveness makes this approach straightforwardly applicable to challenging problems characterized by nonlinearity or non affine weak forms. As we show in the various examples presented in the paper, the method can be interfaced with no particular effort to existing third party simulation software making the proposed approach particularly appealing and adapted to practical engineering problems of industrial interest.



The authors of the paper would like to acknowledge Jean-Louis Duval, Jean-Christophe Allain and Julien Charbonneaux from the ESI group for the support and data for crash and stamping simulations.

Compliance with Ethical Standards

Conflict of interest

The authors declare that they have no conflict of interest.

Informed Consent

All the authors are informed and provided their consent.

Research Involving Human and Animal Participants

The research does not involve neither human participants nor animals.


  1. 1.
    Zorriassatine F, Wykes C, Parkin R, Gindy N (2003) A survey of virtual proto-typing techniques for mechanical product development. Proc Inst Mech Eng Part B J Eng Manuf 217(4):513–530CrossRefGoogle Scholar
  2. 2.
    Oden JT, Belytschko T, Fish J, Hughes TJR, Johnson C, Keyes D, Laub A, Petzold L, Srolovitz D, Yip S (2006) Simulation-based engineering science: revolutionizing engineering science through simulation. Report of the NSF Blue Ribbon Panel on Simulation-Based Engineering Science. National Science Foundation, ArlingtonGoogle Scholar
  3. 3.
    Glotzer SC, Kim S, Cummings PT, Deshmukh A, Head-Gordon M, Karniadakis G, Petzold L, Sagui C, Shinozuka M (2009) International assessment of research and development in simulation-based engineering and science. Panel Report. World Technology Evaluation Center Inc, BaltimoreGoogle Scholar
  4. 4.
    Bellman RE (2003) Dynamic programming. Courier Dover Publications, New York (Republished edition)zbMATHGoogle Scholar
  5. 5.
    Montgomery DC (2013) Design and analysis of experiments, 8th edn. Wiley, HobokenGoogle Scholar
  6. 6.
    Chong EKP, Zak SH (2013) An introduction to optimization, 4th edn. Wiley series on discrete mathematics and optimization. Wiley, HobokenzbMATHGoogle Scholar
  7. 7.
    Antoulas A, Sorensen DC, Gugercin S (2001) A survey of model reduction methods for large-scale systems. Contemp Math 280:193–220MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Bialecki RA, Kassab AJ, Fic A (2005) Proper orthogonal decomposition and modal analysis for acceleration of transient FEM thermal analysis. Int J Numer Method Eng 62(6):774–797CrossRefzbMATHGoogle Scholar
  9. 9.
    Quarteroni A, Manzoni A, Negri E (2015) Reduced basis methods for partial differential equations: an introduction. Modeling and simulation in science, engineering and technology, 1st edn. Springer, BaselGoogle Scholar
  10. 10.
    Huynh DBP, Rozza G, Sen S, Patera AT (2007) A successive constraint linear optimization method for lower bounds of parametric coercivity and infsup stability constants. C R Math 345(8):473–478zbMATHGoogle Scholar
  11. 11.
    Daversin C, Prud’homme C (2015) Simultaneous empirical interpolation and reduced basis method for non-linear problems. C R Math 353(12):1105–1109MathSciNetzbMATHGoogle Scholar
  12. 12.
    Barrault M, Maday Y, Nguyen NC, Patera AT (2004) An “empirical interpolation method”: application to efficient reduced-basis discretization of partial differential equations. C R Math 339(9):667–672MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Farhat C, Chapman T, Avery P (2015) Structure-preserving, stability, and accuracy properties of the energy-conserving sampling and weighting method for the hyper reduction of nonlinear finite element dynamic models. Int J Numer Method Eng 102:1077–1110MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Chaturantabut S, Sorensen DC (2010) Nonlinear model order reduction via discrete empirical interpolation. SIAM J Sci Comput 32(5):2737–2764MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Grepl MA, Maday Y, Nguyen NC, Patera AT (2007) Efficient reduced-basis treatment of non-affine and nonlinear partial differential equations. ESAIM Math Model Numer Anal 41(3):575–605CrossRefzbMATHGoogle Scholar
  16. 16.
    Maday Y, Nguyen NC, Patera AT, Pau SH (2009) A general multipurpose interpolation procedure: the magic points. CPPA 8(1):383–404MathSciNetzbMATHGoogle Scholar
  17. 17.
    Ryckelynck D (2005) A priori hypereduction method: an adaptive approach. J Comput Phys 202(1):346–366CrossRefzbMATHGoogle Scholar
  18. 18.
    Amsallem D, Farhat C (2011) An online method for interpolating linear parametric reduced-order models. SIAM J Sci Comput 33(5):2169–2198MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Hernández J, Caicedo MA, Ferrer A (2017) Dimensional hyper-reduction of nonlinear finite element models via empirical cubature. Comput Method Appl Mech Eng 313:687–722MathSciNetCrossRefGoogle Scholar
  20. 20.
    Rapún ML, Terragni F, Vega JM (2017) Lupod: collocation in POD via LU decomposition. J Comput Phys 335:1–20MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Kumar D, Raisee M, Lacor C (2016) An efficient non-intrusive reduced basis model for high dimensional stochastic problems in CFD. Comput Fluids 138:67–82MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Prulière E, Chinesta F, Ammar A (2010) On the deterministic solution of multidimensional parametric models using the proper generalized decomposition. Math Comput Simul 81(4):791–810MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Hackbusch W (2012) Tensor spaces and numerical tensor calculus, 1st edn. Springer Series in Computational Mathematics. Springer, BerlinCrossRefzbMATHGoogle Scholar
  24. 24.
    Grasedyck L, Kressner D, Tobler C (2013) A literature survey of low-rank tensor approximation techniques. GAMM-Mitt 36:53–78. arXiv:1302.7121
  25. 25.
    Kolda TG, Bader BW (2009) Tensor decompositions and applications. SIAM Rev 51(3):455–500MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Ammar A, Mokdad B, Chinesta F, Keunings R (2007) A new family of solvers for some classes of multidimensional partial differential equations encountered in kinetic theory modeling of complex fluids. Part II: transient simulation using space-time separated representations. J Non-Newtonian Fluid Mech 144(2–3):98–121CrossRefzbMATHGoogle Scholar
  27. 27.
    Nouy A (2010) A priori model reduction through proper generalized decomposition for solving time-dependent partial differential equations. Comput Method Appl Mech Eng 199:1603–1626MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    Chinesta F, Leygue A, Bordeu F, Aguado JV, Cueto E, Gonzalez D, Alfaro I, Ammar A, Huerta A (2013) PGD-based computational vademecum for efficient design, optimization and control. Arch Comput Methods Eng 20(1):31–59MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Chinesta F, Ladevèze P, Cueto E (2011) A short review on model order reduction based on proper generalized decomposition. Arch Comput Methods Eng 18(4):395–404CrossRefGoogle Scholar
  30. 30.
    Ammar A, Chinesta F, Díez P, Huerta A (2010) An error estimator for separated representations of highly multidimensional models. Comput Method Appl Mech Eng 199(25–28):1872–1880MathSciNetCrossRefzbMATHGoogle Scholar
  31. 31.
    Uschmajew A (2012) Local convergence of the alternating least squares algorithm for canonical tensor approximation. SIAM J Matrix Anal Appl 33(2):639–652MathSciNetCrossRefzbMATHGoogle Scholar
  32. 32.
    Quesada C, Alfaro I, Gonzalez D, Cueto E, Chinesta F (2014) PGD-based model reduction for surgery simulation: solid dynamics and contact detection. Lect Notes Comput Sci 8789:193–202CrossRefGoogle Scholar
  33. 33.
    Aguado JV, Borzacchiello D, Ghnatios C, Lebel F, Upadhyay R, Binetruy C, Chinesta F (2017) A simulation app based on reduced order modeling for manufacturing optimization of composite outlet guide vanes. Adv Model Simul Eng Sci 4(1):1CrossRefGoogle Scholar
  34. 34.
    Borzacchiello D, Aguado JV, Chinesta F (2016) Reduced order modelling for efficient numerical optimisation of a hot-wall Chemical Vapour Deposition reactor. Int J Numer Method Heat Fluid Flow 27(4). doi: 10.1108/HFF-04-2016-0153
  35. 35.
    Ghnatios Ch, Masson F, Huerta A, Leygue A, Cueto E, Chinesta F (2012) Proper generalized decomposition based dynamic data-driven control of thermal processes. Comput Method Appl Mech Eng 213–216:29–41CrossRefGoogle Scholar
  36. 36.
    Cohen A, DeVore R (2015) Approximation of high-dimensional parametric PDEs. Acta Numer 24:1–159MathSciNetCrossRefzbMATHGoogle Scholar
  37. 37.
    Bachmayr M, Cohen A, Dahmen W (2016) Parametric PDEs: sparse or low-rank approximations? arXiv:1607.04444
  38. 38.
    Boyd JP (2001) Chebyshev and Fourier spectral methods. Courier Corporation, Ann ArborzbMATHGoogle Scholar
  39. 39.
    Candès E, Romberg J (2007) Sparsity and incoherence in compressive sampling. Inverse Probl 23(3):969MathSciNetCrossRefzbMATHGoogle Scholar
  40. 40.
    Gilbert A, Indyk P (2010) Sparse recovery using sparse matrices. Proc IEEE 98(6):937–947CrossRefGoogle Scholar
  41. 41.
    Donoho DL (2006) Compressed sensing. IEEE Trans Inform Theor 52(4):1289–1306MathSciNetCrossRefzbMATHGoogle Scholar
  42. 42.
    Chen SS, Donoho DL, Saunders MA (2001) Atomic decomposition by basis pursuit. SIAM Rev 43(1):129–159MathSciNetCrossRefzbMATHGoogle Scholar
  43. 43.
    Tibshirani R (1996) Regression shrinkage and selection via the lasso. J R Stat Soc Series B Methodol 58:267–288MathSciNetzbMATHGoogle Scholar
  44. 44.
    Zou H, Hastie T (2005) Regularization and variable selection via the elastic net. J R Stat Soc Series B Stat Methodol 67(2):301–320MathSciNetCrossRefzbMATHGoogle Scholar
  45. 45.
    Brunton SL, Proctor JL, Kutz JN (2016) Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proc Natl Acad Sci USA 113(15):3932–3937MathSciNetCrossRefzbMATHGoogle Scholar
  46. 46.
    Bungartz HJ, Griebel M (2004) Sparse grids. Acta Numer 13:147–269MathSciNetCrossRefzbMATHGoogle Scholar
  47. 47.
    Nobile F, Tempone R, Webster CG (2008) A sparse grid stochastic collocation method for partial differential equations with random input data. SIAM J Numer Anal 46(5):2309–2345MathSciNetCrossRefzbMATHGoogle Scholar
  48. 48.
    Pflüger D, Peherstorfer B, Bungartz HJ (2010) Spatially adaptive sparse grids for high-dimensional data-driven problems. J Complex 26(5):508–522MathSciNetCrossRefzbMATHGoogle Scholar
  49. 49.
    Gerstner T, Griebel M (2003) Dimension-adaptive tensor-product quadrature. Computing 71(1):65–87MathSciNetCrossRefzbMATHGoogle Scholar
  50. 50.
    Nobile F, Tempone R, Webster CG (2008) An anisotropic sparse grid stochastic collocation method for partial differential equations with random input data. SIAM J Numer Anal 46(5):2411–2442MathSciNetCrossRefzbMATHGoogle Scholar
  51. 51.
    Golub GH, Van Loan CF (1996) Matrix computations, 3rd edn. The Johns Hopkins University Press, BaltimorezbMATHGoogle Scholar
  52. 52.
    Halko N, Martinsson PG, Tropp JA (2011) Finding structure with randomness: probabilistic algorithms for constructing approximate matrix decompositions. SIAM Rev 53(2):217–288MathSciNetCrossRefzbMATHGoogle Scholar
  53. 53.
    Harshman RA (1970) Foundations of the parafac procedure: models and conditions for an “explanatory” multi-modal factor analysis. UCLA Working Papers in PhoneticsGoogle Scholar
  54. 54.
    Carroll JD, Chang J (1970) Analysis of individual differences in multidimensional scaling via an n-way generalization of eckart-young decomposition. Psychometrika 35(3):283–319CrossRefzbMATHGoogle Scholar
  55. 55.
    Li Y (2004) On incremental and robust subspace learning. Pattern Recognit 37(7):1509–1518CrossRefzbMATHGoogle Scholar
  56. 56.
    Zhao H, Yuen PC, Kwok JT (2006) A novel incremental principal component analysis and its application for face recognition. IEEE Trans Syst Man Cybern Part B 36(4):873–886CrossRefGoogle Scholar
  57. 57.
    Sobral A, Baker CG, Bouwmans T, Zahzah E (2014) Incremental and multi-feature tensor subspace learning applied for background modeling and subtraction. In International conference image analysis and recognition. Springer, New York. pp. 94–103Google Scholar
  58. 58.
    Brand M (2002) Incremental singular value decomposition of uncertain data with missing values. Computer Vision ECCV 2002, pp. 707–720Google Scholar
  59. 59.
    Quarteroni A, Rozza G (2007) Numerical solution of parametrized navier-stokes equations by reduced basis methods. Numer Methods Partial Differ Equ 23(4):923–948MathSciNetCrossRefzbMATHGoogle Scholar
  60. 60.
    Canuto C, Hussaini MY, Quarteroni A, Zang TA Jr (2012) Spectral methods in fluid dynamics. Springer, BerlinzbMATHGoogle Scholar
  61. 61.
    De Lathauwer L, De Moor B, Vanderwalle J (2000) A multilinear singular value decomposition. SIAM J Matrix Anal Appl 21(4):1253–1278MathSciNetCrossRefzbMATHGoogle Scholar
  62. 62.
    Smoljak SA (1963) Quadrature and interpolation formulae on tensor products of certain function classes. Dokl Akad Nauk SSSR 148(5):1042–1045 (Transl.: Soviet Math Dokl 4:240–243, 1963)MathSciNetGoogle Scholar
  63. 63.
    Gerstner T, Griebel M (1998) Numerical integration using sparse grids. Numer Algorithm 18(3):209–232MathSciNetCrossRefzbMATHGoogle Scholar
  64. 64.
    Dauge M, Stevenson R (2010) Sparse tensor product wavelet approximation of singular functions. SIAM J Math Anal 42(5):2203–2228MathSciNetCrossRefzbMATHGoogle Scholar
  65. 65.
    Garcke J (2007) A dimension adaptive sparse grid combination technique for machine learning. ANZIAM J 48:725–740MathSciNetCrossRefzbMATHGoogle Scholar
  66. 66.
    Dũng  D, Temlyakov VN, Ullrich T (2016) Hyperbolic cross approximation. arXiv:1601.03978
  67. 67.
    Quarteroni A, Rozza G, Manzoni A (2011) Certified reduced basis approximation for parametrized partial differential equations and applications. J Math Indus 1(1):3MathSciNetCrossRefzbMATHGoogle Scholar
  68. 68.
    Bordeu E (2013) Pxdmf : aA file format for separated variables problems version 1.6. Technical report, Ecole Centrale de NantesGoogle Scholar
  69. 69.
    Chen P, Quarteroni A, Rozza G (2014) Comparison between reduced basis and stochastic collocation methods for elliptic problems. J Sci Comput 59(1):187–216MathSciNetCrossRefzbMATHGoogle Scholar
  70. 70.
    Peherstorfer B, Zimmer S, Bungartz HJ (2012) Model reduction with the reduced basis method and sparse grids. Sparse grids and applications. Springer, Berlin, pp. 223–242CrossRefGoogle Scholar

Copyright information

© CIMNE, Barcelona, Spain 2017

Authors and Affiliations

  • Domenico Borzacchiello
    • 1
    Email author
  • José V. Aguado
    • 1
  • Francisco Chinesta
    • 1
  1. 1.High Performance Computing Institute & ESI GROUP Chair @ Ecole Centrale de NantesNantesFrance

Personalised recommendations