Analysis of parametric models

Linear methods and approximations
  • Hermann G. MatthiesEmail author
  • Roger Ohayon


Parametric models in vector spaces are shown to possess an associated linear map, leading directly to reproducing kernel Hilbert spaces and affine/linear representations in terms of tensor products. From this map, analogues of correlation operators can be formed such that the associated linear map factorises the correlation. Its spectral decomposition and the associated Karhunen-Loève- or proper orthogonal decomposition in a tensor product follow directly, including an extension to continuous spectra. It is shown that all factorisations of a certain class are unitarily equivalent, as well as that every factorisation induces a different representation, and vice versa. No particular assumptions are made on the parameter set, other than that the vector space of real valued functions on this set allows an appropriate inner product on a subspace. A completely equivalent spectral and factorisation analysis can be carried out in kernel space. The relevance of these abstract constructions is shown on a number of mostly familiar examples, thus unifying many such constructions under one theoretical umbrella. From the factorisation, one obtains tensor representations, which may be cascaded, leading to tensors of higher degree. When carried over to a discretised level in the form of a model order reduction, such factorisations allow sparse low-rank approximations which lead to very efficient computations especially in high dimensions.


Parametric models Reproducing kernel Hilbert space Correlation Factorisation Spectral decomposition Representation 

Mathematics Subject Classification (2010)

35B30 37M99 41A05 41A45 41A63 60G20 60G60 65J99 93A30 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.



  1. 1.
    Ammar, A., Chinesta, F., Falcó, A.: On the convergence of a greedy rank-one update algorithm for a class of linear systems. Arch. Computat. Methods Eng. 17, 473–486 (2010). MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Atkinson, K.E.: The Numerical Solution of Integral Equations of the Second Kind. Cambridge University Press, Cambridge (1997)CrossRefGoogle Scholar
  3. 3.
    Benner, P., Gugercin, S., Willcox, K.: A survey of projection-based model reduction methods for parametric dynamical systems. SIAM Rev. 57, 483–531 (2015). MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Benner, P., Ohlberger, M., Patera, A.T., Rozza, G., Urban, K. (eds.): Model reduction of parametrized systems, MS&A — modeling simulation & applications, vol. 17. Springer, Berlin (2017). Google Scholar
  5. 5.
    Berlinet, A., Thomas-Agnan, C.: Reproducing Kernel Hilbert Spaces in Probability and Statistics. Kluwer, Dordrecht (2004)CrossRefGoogle Scholar
  6. 6.
    Bracewell, R.N.: The Fourier Transform and Its Applications. McGraw-Hill, New York NY (1978)Google Scholar
  7. 7.
    Chinesta, F., Keunings, R., Leygue, A.: The Proper Generalized Decomposition for Advanced Numerical Simulations. Springer, Berlin (2014)CrossRefGoogle Scholar
  8. 8.
    Chinesta, F., Ladevèze, P., Cueto, E.: A short review on model order reduction based on proper generalized decomposition. Arch. Computat. Methods Eng. 18, 395–404 (2011). CrossRefGoogle Scholar
  9. 9.
    Cohen, N., Sharri, O., Shashua, A.: On the expressive power of deep learning: a tensor analysis. arXiv:1509.05009[cs.NE] (2016)
  10. 10.
    Courant, R., Hilbert, D.: Methods of Mathematical Physics. Wiley, Chichester (1989)CrossRefGoogle Scholar
  11. 11.
    Dautray, R., Lions, J.L.: Spectral Theory and Applications Mathematical Analysis and Numerical Methods for Science and Technology, vol. 3. Springer, Berlin (1990)zbMATHGoogle Scholar
  12. 12.
    Espig, M., Hackbusch, W., Litvinenko, A., Matthies, H.G., Zander, E.: Efficient analysis of high dimensional data in tensor formats. In: Garcke, J., Griebel, M. (eds.) Sparse Grids and Applications, Lecture Notes in Computational Science and Engineering., vol. 88, pp 31–56. Springer, Berlin (2013)Google Scholar
  13. 13.
    Gel’fand, I.M., Shilov, G.E.: Properties and Operations Generalized Functions, vol. 1. Academic Press, New York (1964)zbMATHGoogle Scholar
  14. 14.
    Gel’fand, I.M., Shilov, G.E.: Theory of Differential Equations Generalized Functions, vol. 3. Academic Press, New York (1967)zbMATHGoogle Scholar
  15. 15.
    Gel’fand, I.M., Shilov, G.E.: Spaces of Fundamental and Generalized Functions. Generalized Functions, vol. 2. Academic Press, New York (1968)zbMATHGoogle Scholar
  16. 16.
    Gel’fand, I.M., Vilenkin, N.Y.: Applications of Harmonic Analysis Generalized Functions, vol. 4. Academic Press, New York (1964)zbMATHGoogle Scholar
  17. 17.
    Grasedyck, L.: Hierarchical singular value decomposition of tensors. SIAM Journal on Matrix Analysis and Applications 31, 2029–2054 (2010). MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Grasedyck, L., Kressner, D., Tobler, C.: A literature survey of low-rank tensor approximation techniques. GAMM-Mitteilungen 36, 53–78 (2013). MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Gross, L.: Measurable functions on Hilbert space. Trans. Am. Math. Soc. 105 (3), 372–390 (1962). MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Hackbusch, W.: Tensor Spaces and Numerical Tensor Calculus. Springer, Berlin (2012)CrossRefGoogle Scholar
  21. 21.
    Janson, S.: Gaussian Hilbert Spaces. Cambridge Tracts in Mathematics, vol. 129. Cambridge University Press, Cambridge (1997)CrossRefGoogle Scholar
  22. 22.
    Karhunen, K.: Zur Spektraltheorie stochastischer Prozesse. Ann. Acad. Sci. Fennicae. Ser. A. I. Math.-Phys. 34, 1–7 (1946)MathSciNetzbMATHGoogle Scholar
  23. 23.
    Karhunen, K.: Über lineare Methoden in der Wahrscheinlichkeitsrechnung. Ann. Acad. Sci. Fennicae. Ser. A. I. Math.-Phys. 37, 1–79 (1947)MathSciNetzbMATHGoogle Scholar
  24. 24.
    Karhunen, K.: Über die Struktur stationärer zufälliger Funktionen. Arkiv för Matematik 1, 141–160 (1950). MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Karhunen, K., Oliva Santos, F., Ferrer Martín, S.: Métodos lineales en el cálculo de probabilidades — Über lineare Methoden in der Wahrscheinlichkeitsrechnung. In: Trabajos de Estadística Y de Investigación Operativa, pp 59–137 (1947). Spanish Tranlation — Traducción Español [23]CrossRefGoogle Scholar
  26. 26.
    Karhunen, K., Selin, I.: On linear methods in probability theory — Über lineare Methoden in der Wahrscheinlichkeitsrechnung — 1947. U.S. Air Force — Project RAND T-131, The RAND Corporation, St Monica, CA, USA. Englisch Translation [23] (1960)
  27. 27.
    Khrulkov, V., Novikov, A., Oseledets, I.: Expressive power of recurrent neural net-works. arXiv:1711.00811[cs.LG] (2018)
  28. 28.
    Krée, P., Soize, C.: Mathematics of Random Phenomena—Random Vibrations of Mechanical Structures. D. Reidel, Dordrecht (1986)CrossRefGoogle Scholar
  29. 29.
    Ladevèze, P., Chamoin, L.: On the verification of model reduction methods based on the proper generalized decomposition. Comput. Methods Appl. Mech. Eng. 200(23–24), 2032–2047 (2011). MathSciNetCrossRefzbMATHGoogle Scholar
  30. 30.
    Le Maître, O.P., Knio, O.M.: Spectral Methods for Uncertainty Quantification. Scientific Computation. Springer, Berlin (2010)CrossRefGoogle Scholar
  31. 31.
    Loève, M.: Fonctions alétoires de second ordre. C. R. Acad. Sci. 220, 295–296 (1945)Google Scholar
  32. 32.
    Loève, M.: Fonctions alétoires de second ordre. C. R. Acad. Sci. 222 (1946)Google Scholar
  33. 33.
    Loève, M.: Probability Theory II Graduate Texts in Mathematics, 4th edn., vol. 46. Springer, Berlin (1978)CrossRefGoogle Scholar
  34. 34.
    Luenberger, D.G.: Optimization by Vector Space Methods. Wiley, Chichester (1969)zbMATHGoogle Scholar
  35. 35.
    Matthies, H.G.: Uncertainty quantification with stochastic finite elements. In: Stein, E., de Borst, R., Hughes, T.J.R. (eds.) Encyclopaedia of Computational Mechanics. Part 1. Fundamentals. Encyclopaedia of Computational Mechanics, vol. 1. Wiley, Chichester (2007)
  36. 36.
    Matthies, H.G.: Uncertainty quantification and Bayesian inversion. In: Stein, E., de Borst, R., Hughes, T.J.R. (eds.) Encyclopaedia of Computational Mechanics. 2nd edn. Part 1. Fundamentals. Encyclopaedia of Computational Mechanics, vol. 1. Wiley, Chichester (2017)
  37. 37.
    Matthies, H.G., Litvinenko, A., Pajonk, O., Rosić, B.V., Zander, E.: Parametric and uncertainty computations with tensor product representations. In: Dienstfrey, A., Boisvert, R. (eds.) Uncertainty Quantification in Scientific Computing, IFIP Advances in Information and Communication Technology, vol. 377, pp 139–150. Springer, Boulder (2012). zbMATHGoogle Scholar
  38. 38.
    Nouy, A.: Proper generalized decompositions and separated representations for the numerical solution of high dimensional stochastic problems. Arch. Comput. Methods Eng. 17, 403–434 (2010). MathSciNetCrossRefzbMATHGoogle Scholar
  39. 39.
    Nouy, A., Le Maître, O.P.: Generalized spectral decomposition for stochastic nonlinear problems. J. Comput. Phys. 228(1), 202–235 (2009). MathSciNetCrossRefzbMATHGoogle Scholar
  40. 40.
    Oseledets, I.: TT-cross approximation for multidimensional arrays. Linear Algebra Appl. 432, 70–88 (2010). MathSciNetCrossRefzbMATHGoogle Scholar
  41. 41.
    Oseledets, I.V.: Tensor-train decomposition. SIAM J. Sci. Comput. 33(5), 2295–2317 (2011). MathSciNetCrossRefzbMATHGoogle Scholar
  42. 42.
    Reed, M., Simon, B.: Fourier Analysis and Self-Adjointness, Methods of Modern Mathematical Physics, vol. II. Academic Press, New York (1975)zbMATHGoogle Scholar
  43. 43.
    Reed, M., Simon, B.: Functional Analysis, Methods of Modern Mathematical Physics, vol. I. Academic Press, New York (1980)zbMATHGoogle Scholar
  44. 44.
    Segal, I.E.: Tensor algebras over Hilbert spaces I. Trans. Am. Math. Soc. 81 (1), 106–134 (1956). MathSciNetCrossRefzbMATHGoogle Scholar
  45. 45.
    Segal, I.E.: Distributions in Hilbert space and canonical systems of operators. Trans. Am. Math. Soc. 88(1), 12–41 (1958). MathSciNetCrossRefzbMATHGoogle Scholar
  46. 46.
    Segal, I.E.: Nonlinear functions of weak processes. I. J. Funct. Anal. 4(3), 404–456 (1969). MathSciNetCrossRefzbMATHGoogle Scholar
  47. 47.
    Segal, I.E., Kunze, R.A.: Integrals and Operators. Springer, Berlin (1978)CrossRefGoogle Scholar
  48. 48.
    Smith, R.C.: Uncertainty Quantification: Theory, Implementation, and Applications Computational Science & Engineering, vol. 12. SIAM, Philadelphia (2014)Google Scholar
  49. 49.
    Soize, C., Farhat, C.: A nonparametric probabilistic approach for quantifying uncertainties in low-dimensional and high-dimensional nonlinear models. Int. J. Numer. Methods Eng. 109, 837–888 (2017). MathSciNetCrossRefGoogle Scholar
  50. 50.
    Strang, G.: Introduction to Applied Mathematics. Wellesley-Cambridge Press, Wellesley (1986)zbMATHGoogle Scholar
  51. 51.
    Xiu, D.: Numerical Methods for Stochastic Computations: a Spectral Method Approach. Princeton University Press, Princeton (2010)CrossRefGoogle Scholar
  52. 52.
    Yaglom, A.M.: Correlation Theory of Stationary and Related Random Functions I. Springer, Berlin (1968)Google Scholar
  53. 53.
    Yaglom, A.M.: Correlation Theory of Stationary and Related Random Functions II. Springer, Berlin (1968)Google Scholar
  54. 54.
    Yaglom, A.M.: An introduction to the theory of stationary random functions. Dover, Mineola, NY USA (2004)Google Scholar
  55. 55.
    Yosida, K.: Functional Analysis, 6th edn. Springer, Berlin (1980)zbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Institute of Scientific ComputingTechnische Universität BraunschweigBraunschweigGermany
  2. 2.Structural Mechanics and Coupled Systems LaboratoryConservatoire National des Arts et Métiers (CNAM)ParisFrance

Personalised recommendations