A Distributed Active Subspace Method for Scalable Surrogate Modeling of Function Valued Outputs


We present a distributed active subspace method for training surrogate models of complex physical processes with high-dimensional inputs and function valued outputs. Specifically, we represent the model output with a truncated Karhunen–Loève (KL) expansion, screen the structure of the input space with respect to each KL mode via the active subspace method, and finally form an overall surrogate model of the output by combining surrogates of individual output KL modes. To ensure scalable computation of the gradients of the output KL modes, needed in active subspace discovery, we rely on adjoint-based gradient computation. The proposed method combines benefits of active subspace methods for input dimension reduction and KL expansions used for spectral representation of the output field. We provide a mathematical framework for the proposed method and conduct an error analysis of the mixed KL active subspace approach. Specifically, we provide an error estimate that quantifies errors due to active subspace projection and truncated KL expansion of the output. We demonstrate the numerical performance of the surrogate modeling approach with an application example from biotransport.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9


  1. 1.

    Alexanderian, A., Gremaud, P., Smith, R.: Variance-based sensitivity analysis for time-dependent processes. Reliab. Eng. Syst. Safety 196, 106722 (2020)

    Article  Google Scholar 

  2. 2.

    Alexanderian, A., Reese, W., Smith, R.C., Yu, M.: Model input and output dimension reduction using Karhunen–Loève expansions with application to biotransport. ASCE-ASME J. Risk Uncertain. Eng. Syst. B Mech. Eng. https://ui.adsabs.harvard.edu/#abs/2019arXiv190306314A. Accepted (2019)

  3. 3.

    Alexanderian, A., Zhu, L., Salloum, M., Ma, R., Yu, M.: Investigation of biotransport in a tumor with uncertain material properties using a non-intrusive spectral uncertainty quantification method. J. Biomech. Eng. 139, 091006-1–091006-11 (2017)

    Article  Google Scholar 

  4. 4.

    Babuška, I., Nobile, F., Tempone, R.: A stochastic collocation method for elliptic partial differential equations with random input data. SIAM J. Numer. Anal. 45(3), 1005–1034 (2007)

    MathSciNet  Article  Google Scholar 

  5. 5.

    Blatman, G., Sudret, B.: Sparse polynomial chaos expansions and adaptive stochastic finite elements using a regression approach. C. R. Mécan. 336(6), 518–523 (2008)

    Article  Google Scholar 

  6. 6.

    Blatman, G., Sudret, B.: Sparse polynomial chaos expansions of vector-valued response quantities. In: Safety, Reliability, Risk and Life-cycle Performance of Structures and Infrastructures, pp. 3245–3252 (2013)

  7. 7.

    Chen, P., Villa, U., Ghattas, O.: Taylor approximation and variance reduction for PDE-constrained optimal control under uncertainty. J. Comput. Phys. 385, 163–186 (2019)

    MathSciNet  Article  Google Scholar 

  8. 8.

    Clark, W.H.: Tumour progression and the nature of cancer. Br. J. Cancer 64, 631–44 (1991)

    Article  Google Scholar 

  9. 9.

    Clark, W.H.: Biphasic finite element model of solute transport for direct infusion into nervous tissue. Ann. Biomed. Eng. 35, 2145–2158 (2007)

    Article  Google Scholar 

  10. 10.

    Cleaves, H., Alexanderian, A., Saad, B.: Structure exploiting methods for fast uncertainty quantification in multiphase flow through heterogeneous media. Preprint https://arxiv.org/abs/2008.11274 (2020)

  11. 11.

    Cleaves, H.L., Alexanderian, A., Guy, H., Smith, R.C., Yu, M.: Derivative-based global sensitivity analysis for models with high-dimensional inputs and functional outputs. arXiv e-prints arXiv:1902.04630 (2019)

  12. 12.

    Constantine, P.: Active Subspaces: Emerging Ideas in Dimension Reduction for Parameter Studies. SIAM, Philadelphia (2015)

    Google Scholar 

  13. 13.

    Constantine, P.G., Diaz, P.: Global sensitivity metrics from active subspaces. Reliab. Eng. Syst. Saf. 162, 1–13 (2017)

    Article  Google Scholar 

  14. 14.

    Constantine, P.G., Doostan, A.: Time-dependent global sensitivity analysis with active subspaces for a lithium ion battery model. Stat. Anal. Data Min. ASA Data Sci. J. 10, 243–262 (2017)

    MathSciNet  Article  Google Scholar 

  15. 15.

    Constantine, P.G., Dow, E., Wang, Q.: Active subspace methods in theory and practice: applications to kriging surfaces. SIAM J. Sci. Comput. 36(4), A1500–A1524 (2014)

    MathSciNet  Article  Google Scholar 

  16. 16.

    Constantine, P.G., Emory, M., Larsson, J., Iaccarino, G.: Exploiting active subspaces to quantify uncertainty in the numerical simulation of the Hyshot II scramjet. J. Comput. Phys. 302, 1–20 (2015)

    MathSciNet  Article  Google Scholar 

  17. 17.

    Debbage, P.: Targeted drugs and nanomedicine: present and future. Curr. Pharm. Des. 15, 153–72 (2009)

    Article  Google Scholar 

  18. 18.

    Doostan, A., Ghanem, R.G., Red-Horse, J.: Stochastic model reduction for chaos representations. Comput. Methods Appl. Mech. Eng. 196(37–40), 3951–3966 (2007)

    MathSciNet  Article  Google Scholar 

  19. 19.

    Doostan, A., Owhadi, H.: A non-adapted sparse approximation of PDEs with stochastic inputs. J. Comput. Phys. 230(8), 3015–3034 (2011)

    MathSciNet  Article  Google Scholar 

  20. 20.

    Elman, H.: Solution algorithms for stochastic Galerkin discretizations of differential equations with random data. In: Handbook of Uncertainty Quantification, pp. 1–16 (2017)

  21. 21.

    Friedman, J.: Fast MARS. Technical Report 110, Laboratory for Computational Statistics, Department of Statistics, Stanford University (1993)

  22. 22.

    Gamboa, F., Janon, A., Klein, T., Lagnoux, A., et al.: Sensitivity analysis for multidimensional and functional outputs. Electron. J. Stat. 8(1), 575–603 (2014)

    MathSciNet  Article  Google Scholar 

  23. 23.

    Ghanem, R.: Probabilistic characterization of transport in heterogeneous media. Comput. Methods Appl. Mech. Eng. 158(3), 199–220 (1998). https://doi.org/10.1016/S0045-7825(97)00250-8

    Article  MATH  Google Scholar 

  24. 24.

    Ghanem, R.G., Spanos, P.D.: Stochastic Finite Elements: A Spectral Approach. Springer, New York (1991). https://doi.org/10.1007/978-1-4612-3094-6

    Google Scholar 

  25. 25.

    Graham, I.G., Kuo, F.Y., Nichols, J.A., Scheichl, R., Schwab, C., Sloan, I.H.: Quasi-Monte Carlo finite element methods for elliptic PDEs with lognormal random coefficients. Numer. Math. 131(2), 329–368 (2015)

    MathSciNet  Article  Google Scholar 

  26. 26.

    Gunzburger, M.: Perspectives in Flow Control and Optimization, vol. 5. SIAM, Philadelphia (2003)

    Google Scholar 

  27. 27.

    Hsing, T., Eubank, R.: Theoretical Foundations of Functional Data Analysis, with an Introduction to Linear Operators. Wiley, Hoboken (2015)

    Google Scholar 

  28. 28.

    Iooss, B., Saltelli, A.: Introduction to sensitivity analysis. In: Ghanem, R., Higdon, D., Owhadi, H. (eds.) Handbook of Uncertainty Quantification, pp. 1103–1122. Springer, Berlin (2017)

    Google Scholar 

  29. 29.

    Jefferson, J., Gilbert, J., Constantine, P., Maxwell, R.: Active subspaces for sensitivity analysis and dimension reduction of an integrated hydrologic model. Comput. Geosci. 83, 127–138 (2015)

    Article  Google Scholar 

  30. 30.

    Ji, W., Wang, J., Zahm, O., Marzouk, Y.M., Yang, B., Ren, Z., Law, C.K.: Shared low-dimensional subspaces for propagating kinetic uncertainty to multiple outputs. Combust. Flame 190, 146–157 (2018)

    Article  Google Scholar 

  31. 31.

    Kucherenko, S., Iooss, B.: Derivative-based global sensitivity measures. In: Ghanem, R., Higdon, D., Owhadi, H. (eds.) Handbook of Uncertainty Quantification. Springer, Berlin (2017)

    Google Scholar 

  32. 32.

    Le Maıtre, O., Knio, O., Najm, H., Ghanem, R.: Uncertainty propagation using Wiener-Haar expansions. J. Comput. Phys. 197(1), 28–57 (2004)

    MathSciNet  Article  Google Scholar 

  33. 33.

    Le Maître, O.P., Knio, O.M.: Spectral Methods for Uncertainty Quantification: With Applications to Computational Fluid Dynamics. Scientific Computation. Springer, New York (2010). https://doi.org/10.1007/978-90-481-3520-2

    Google Scholar 

  34. 34.

    Le Maître, O.P., Reagan, M.T., Najm, H.N., Ghanem, R.G., Knio, O.M.: A stochastic projection method for fluid flow: II. Random process. J. Comput. Phys. 181(1), 9–44 (2002)

    MathSciNet  Article  Google Scholar 

  35. 35.

    Li, G., Iskandarani, M., Le Hénaff, M., Winokur, J., Le Maître, O.P., Knio, O.M.: Quantifying initial and wind forcing uncertainties in the Gulf of Mexico. Comput. Geosci. 20(5), 1133–1153 (2016)

    MathSciNet  Article  Google Scholar 

  36. 36.

    Loève, M.: Probability Theory: I. Graduate Texts in Mathematics, vol. 45, 4th edn. Springer, New York (1977)

    Google Scholar 

  37. 37.

    Lukaczyk, T.W., Palacios, F., Alonso, J.J., Constantine, P.: Active subspaces for shape optimization. In: The 10th AIAA Multidisciplinary Design Optimization Conference. AIAA-2014-1171. National Harbor, Maryland (2014)

  38. 38.

    Ma, R., Su, D., Zhu, L.: Multiscale simulation of nanopartical transport in deformable tissue during an infusion process in hyperthermia treatments of cancers. In: Minkowycz, W.J., Sparrow, E., Abraham, J.P. (eds.) Nanoparticle Heat Transfer and Fluid Flow, Computational and Physical Processes in Mechanics and Thermal Science Series, vol. 4. CRC Press, Taylor & Francis Group, Boca Raton (2012)

    Google Scholar 

  39. 39.

    Mangado, N., Piella, G., Noailly, J., Pons-Prats, J., Ballester, M.A.G.: Analysis of uncertainty and variability in finite element computational models for biomedical engineering: characterization and propagation. Front. Bioeng. Biotechnol. 4, 85 (2016)

    Article  Google Scholar 

  40. 40.

    Matthies, H.G., Keese, A.: Galerkin methods for linear and nonlinear elliptic stochastic partial differential equations. Comput. Methods Appl. Mech. Eng. 194(12–16), 1295–1331 (2005)

    MathSciNet  Article  Google Scholar 

  41. 41.

    Prieur, C., Tarantola, S.: Variance-based sensitivity analysis: theory and estimation algorithms. In: Ghanem, R., Higdon, D., Owhadi, H. (eds.) Handbook of Uncertainty Quantification, pp. 1217–1239. Springer, Berlin (2017)

    Google Scholar 

  42. 42.

    Rasmussen, C.E., Williams, C.: Gaussian Processes for Machine Learning. The MIT Press, Cambridge (2006)

    Google Scholar 

  43. 43.

    Russi, T.M.: Uncertainty quantification with experimental data and complex system models. Ph.D. thesis, University of California, Berkeley (2010)

  44. 44.

    Saad, G., Ghanem, R.: Characterization of reservoir simulation models using a polynomial chaos-based ensemble Kalman filter. Water Resour. Res. 45(4) (2009)

  45. 45.

    Salloum, M., Ma, R., Weeks, D., Zhu, L.: Controlling nanoparticle delivery in magnetic nanoparticle hyperthermia for cancer treatment: experimental study in agarose gel. Int. J. Hyperth. 24, 337–345 (2008)

    Article  Google Scholar 

  46. 46.

    Sobol, I.: Estimation of the sensitivity of nonlinear mathematical models. Matematicheskoe Modelirovanie 2(1), 112–118 (1990)

    MathSciNet  MATH  Google Scholar 

  47. 47.

    Sobol, I.: Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates. Math. Comput. Simul. 55(1–3), 271–280 (2001)

    MathSciNet  Article  Google Scholar 

  48. 48.

    Sobol’, I., Kucherenko, S.: Derivative based global sensitivity measures and their link with global sensitivity indices. Math. Comput. Simul. 79(10), 3009–3017 (2009)

    MathSciNet  Article  Google Scholar 

  49. 49.

    Vohra, M., Alexanderian, A., Guy, H., Mahadevan, S.: Active subspace-based dimension reduction for chemical kinetics applications with epistemic uncertainty. Combust. Flame 204, 152–161 (2019)

    Article  Google Scholar 

  50. 50.

    Wang, B., Yu, M.: Analysis of wake structures behind an oscillating square cylinder using dynamic mode decomposition. In: 46th AIAA Fluid Dynamics Conference, p. 3779 (2016)

  51. 51.

    Winokur, J., Kim, D., Bisetti, F., Le Maître, O.P., Knio, O.M.: Sparse pseudo spectral projection methods with directional adaptation for uncertainty quantification. J. Sci. Comput. 68(2), 596–623 (2016)

    MathSciNet  Article  Google Scholar 

  52. 52.

    Xiu, D., Karniadakis, G.E.: Modeling uncertainty in flow simulations via generalized polynomial chaos. J. Comput. Phys. 187(1), 137–167 (2003). https://doi.org/10.1016/S0021-9991(03)00092-5

    MathSciNet  Article  MATH  Google Scholar 

  53. 53.

    Yabansu, Y.C., Steinmetz, P., Hötzer, J., Kalidindi, S.R., Nestler, B.: Extraction of reduced-order process-structure linkages from phase-field simulations. Acta Mater. 124, 182–194 (2017)

    Article  Google Scholar 

  54. 54.

    Zahm, O., Constantine, P., Prieur, C., Marzouk, Y.: Gradient-based dimension reduction of multivariate vector-valued functions. arXiv preprint arXiv:1801.07922 (2018)

Download references

Author information



Corresponding author

Correspondence to Meilin Yu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.



Proof of Theorem 1

Note that,

$$\begin{aligned} e(\varvec{x},\varvec{\xi }) =\bigg | \sum _{k=1}^N (F_k(\varvec{\xi }) -{G}_k(\varvec{W}_{k,1}^T\varvec{\xi }))\phi _k(\varvec{x}) \bigg | \le \sum _{k=1}^N |F_k(\varvec{\xi }) -{G}_k(\varvec{W}_{k,1}^T\varvec{\xi })||\phi _k(\varvec{x})|. \end{aligned}$$


$$\begin{aligned} {\mathbb {E}}\{{\bar{e}}\}&= {\mathbb {E}} \bigg \{\int _{\mathcal {X}}\sum _{k=1}^N |F_k(\varvec{\xi }) -{G}_k(\varvec{W}_{k,1}^T\varvec{\xi })||\phi _k(\varvec{x})| d\varvec{x} \bigg \}\\&= \sum _{k=1}^N \bigg (\int _{\mathcal {X}}|\phi _k(\varvec{x})| d\varvec{x} \bigg ) {\mathbb {E}}\{|F_k(\varvec{\xi }) -{G}_k(\varvec{W}_{k,1}^T\varvec{\xi })| \} \\&\le \displaystyle \sum _{k=1}^N \left[ \int _{\mathcal {X}}|\phi _k(\varvec{x})|^2 d\varvec{x}\right] ^\frac{1}{2}\left[ \int _{\mathcal {X}}1^2 d\varvec{x}\right] ^\frac{1}{2} {\mathbb {E}}\{|F_k(\varvec{\xi }) -{G}_k(\varvec{W}_{k,1}^T\varvec{\xi })|^2\}^{1/2} \le |{\mathcal {X}}|^\frac{1}{2} \displaystyle \sum _{k=1}^N \delta _k \end{aligned}$$

where we have used Cauchy–Schwarz inequlity and Lemma 2(a). \(\square \)

Proof of Theorem 2

We have

$$\begin{aligned} {\bar{E}}(\varvec{\xi }) = \int _{\mathcal {X}}E(\varvec{x},\varvec{\xi }) d\varvec{x}&= \int _{\mathcal {X}}|f(\varvec{x},\varvec{\xi })-f_N(\varvec{x},\varvec{\xi })+ f_N(\varvec{x},\varvec{\xi })-{\hat{f}}_N(\varvec{x},\varvec{\xi })| \\&\le \int _{\mathcal {X}}|f(\varvec{x},\varvec{\xi })-f_N(\varvec{x},\varvec{\xi })|d\varvec{x} + \int _{\mathcal {X}}|f_N(\varvec{x},\varvec{\xi })-{\hat{f}}_N(\varvec{x},\varvec{\xi })|d\varvec{x} \\&\le \bigg \{\int _{\mathcal {X}}|f(\varvec{x},\varvec{\xi })-f_N(\varvec{x},\varvec{\xi })|^2 d\varvec{x}\bigg \}^{1/2}|{\mathcal {X}}|^\frac{1}{2} + {\bar{e}}(\varvec{\xi }) \\&= \bigg \{ \sum _{k=N+1}^\infty F_k(\varvec{\xi })^2 \bigg \}^{1/2}|{\mathcal {X}}|^\frac{1}{2} + {\bar{e}}(\varvec{\xi }). \end{aligned}$$

Now we consider

$$\begin{aligned} {\mathbb {E}}\{{\bar{E}}(\varvec{\xi })\}&\le {\mathbb {E}}\Bigg (\bigg \{ \sum _{k=N+1}^\infty F_k(\varvec{\xi })^2 \bigg \}^{1/2}\Bigg )|{\mathcal {X}}|^\frac{1}{2} + {\mathbb {E}}\{{\bar{e}}(\varvec{\xi })\} \\&\le |{\mathcal {X}}|^\frac{1}{2} \Bigg [{\mathbb {E}}\Bigg (\bigg \{\sum _{k=N+1}^\infty F_k(\varvec{\xi })^2\bigg \}^{1/2}\Bigg ) +\sum _{k=1}^N \delta _k \Bigg ] \le |{\mathcal {X}}|^\frac{1}{2}\Bigg [{\mathbb {E}}\Bigg (\sum _{k=N+1}^\infty F_k(\varvec{\xi })^2\Bigg )^{1/2}+\sum _{k=1}^N \delta _k \Bigg ]. \end{aligned}$$

Note that \({\mathbb {E}}\Bigg (\sum _{k=N+1}^\infty F_k(\varvec{\xi })^2\Bigg ) = \sum _{k=N+1}^\infty {\mathbb {E}}\big (F_k(\varvec{\xi })^2\big ) = \sum _{k=N+1}^\infty \lambda _k(C_f)\). Therefore, we have our desired result

$$\begin{aligned} \begin{aligned} {\mathbb {E}}\{{\bar{E}}(\varvec{\xi })\}&\le |{\mathcal {X}}|^\frac{1}{2}\Bigg [\Bigg (\sum _{k=N+1}^\infty \lambda _k(C_f)\Bigg )^{1/2} + \sum _{k=1}^N \delta _k\Bigg ]\\&= |{\mathcal {X}}|^\frac{1}{2}\Bigg [\Bigg (\sum _{k=N+1}^\infty \lambda _k(C_f)\Bigg )^{1/2} + \sum _{k=1}^N \left( \sum _{j = r_k+1}^{N_p}\lambda _j(\mathbf {{S}}_k)\right) ^{1/2}\Bigg ]. \end{aligned}. \end{aligned}$$

\(\square \)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Guy, H., Alexanderian, A. & Yu, M. A Distributed Active Subspace Method for Scalable Surrogate Modeling of Function Valued Outputs. J Sci Comput 85, 36 (2020). https://doi.org/10.1007/s10915-020-01346-2

Download citation


  • Distributed active subspace
  • Karhunen–Loève expansion
  • Dimension reduction
  • Function valued outputs
  • Porous medium flow
  • Biotransport