Inverse Problems in a Bayesian Setting

Part of the Computational Methods in Applied Sciences book series (COMPUTMETHODS, volume 41)


In a Bayesian setting, inverse problems and uncertainty quantification (UQ)—the propagation of uncertainty through a computational (forward) model—are strongly connected. In the form of conditional expectation the Bayesian update becomes computationally attractive. We give a detailed account of this approach via conditional approximation, various approximations, and the construction of filters. Together with a functional or spectral approach for the forward UQ there is no need for time-consuming and slowly convergent Monte Carlo sampling. The developed sampling-free non-linear Bayesian update in form of a filter is derived from the variational problem associated with conditional expectation. This formulation in general calls for further discretisation to make the computation possible, and we choose a polynomial approximation. After giving details on the actual computation in the framework of functional or spectral approximations, we demonstrate the workings of the algorithm on a number of examples of increasing complexity. At last, we compare the linear and nonlinear Bayesian update in form of a filter on some examples.


Posterior Distribution Monte Carlo Kalman Filter Conditional Expectation Uncertainty Quantification 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This work was partially supported by the German research foundation “Deutsche Forschungsgemeinschaft” (DFG).


  1. 1.
    Blanchard, E.D., Sandu, A., Sandu, C.: A polynomial chaos-based Kalman filter approach for parameter estimation of mechanical systems. Journal of Dynamic Systems, Measurement, and Control 132(6), 061404 (2010). doi: 10.1115/1.4002481 Google Scholar
  2. 2.
    Bobrowski, A.: Functional Analysis for Probability and Stochastic Processes. Cambridge University Press, Cambridge (2005)Google Scholar
  3. 3.
    Bosq, D.: Linear Processes in Function Spaces. Theory and Applications., Lecture Notes in Statistics, vol. 149. Springer, Berlin (2000). Contains definition of strong or \(L\)-orthogonality for vector valued random variablesGoogle Scholar
  4. 4.
    Bosq, D.: General linear processes in Hilbert spaces and prediction. Journal of Statistical Planning and Inference 137, 879–894 (2007). doi: 10.1016/j.jspi.2006.06.014 Google Scholar
  5. 5.
    Engl, H.W., Hanke, M., Neubauer, A.: Regularization of inverse problems. Kluwer, Dordrecht (2000)Google Scholar
  6. 6.
    Evensen, G.: Data Assimilation — The Ensemble Kalman Filter. Springer, Berlin (2009)Google Scholar
  7. 7.
    Evensen, G.: The ensemble Kalman filter for combined state and parameter estimation. IEEE Control Systems Magazine 29, 82–104 (2009). doi: 10.1109/MCS.2009.932223 Google Scholar
  8. 8.
    Galvis, J., Sarkis, M.: Regularity results for the ordinary product stochastic pressure equation. SIAM Journal on Mathematical Analysis 44, 2637–2665 (2012). doi: 10.1137/110826904 Google Scholar
  9. 9.
    Gamerman, D., Lopes, H.F.: Markov Chain Monte Carlo: Stochastic Simulation for Bayesian Inference. Chapman & Hall, Boca Raton, FL (2006)Google Scholar
  10. 10.
    Ghanem, R., Spanos, P.D.: Stochastic finite elements—A spectral approach. Springer, Berlin (1991)Google Scholar
  11. 11.
    Goldstein, M., Wooff, D.: Bayes Linear Statistics—Theory and Methods. Wiley Series in Probability and Statistics. John Wiley & Sons, Chichester (2007)Google Scholar
  12. 12.
    Hackbusch, W.: Tensor Spaces and Numerical Tensor Calculus. Springer, Berlin (2012)Google Scholar
  13. 13.
    Hida, T., Kuo, H.H., Potthoff, J., Streit, L.: White Noise—An Infinite Dimensional Calculus. Kluwer, Dordrecht (1999)Google Scholar
  14. 14.
    Holden, H., Øksendal, B., Ubøe, J., Zhang, T.S.: Stochastic Partial Differential Equations. Birkhäuser, Basel (1996)Google Scholar
  15. 15.
    Janson, S.: Gaussian Hilbert spaces. Cambridge Tracts in Mathematics, 129. Cambridge University Press, Cambridge (1997)Google Scholar
  16. 16.
    Jaynes, E.T.: Probability Theory, The Logic of Science. Cambridge University Press, Cambridge (2003)Google Scholar
  17. 17.
    Kálmán, R.E.: A new approach to linear filtering and prediction problems. Transactions of the ASME—J. of Basic Engineering (Series D) 82, 35–45 (1960)Google Scholar
  18. 18.
    Kučerová, A., Matthies, H.G.: Uncertainty updating in the description of heterogeneous materials. Technische Mechanik 30(1–3), 211–226 (2010)Google Scholar
  19. 19.
    Law, K.H.J., Litvinenko, A., Matthies, H.G.: Nonlinear evolution, observation, and update (2015)Google Scholar
  20. 20.
    Luenberger, D.G.: Optimization by Vector Space Methods. John Wiley & Sons, Chichester (1969)Google Scholar
  21. 21.
    Madras, N.: Lectures on Monte Carlo Methods. American Mathematical Society, Providence, RI (2002)Google Scholar
  22. 22.
    Malliavin, P.: Stochastic Analysis. Springer, Berlin (1997)Google Scholar
  23. 23.
    Marzouk, Y.M., Najm, H.N., Rahn, L.A.: Stochastic spectral methods for efficient Bayesian solution of inverse problems. Journal of Computational Physics 224(2), 560–586 (2007). doi: 10.1016/ Google Scholar
  24. 24.
    Matthies, H.G.: Uncertainty quantification with stochastic finite elements. In: E. Stein, R. de Borst, T.J.R. Hughes (eds.) Encyclopaedia of Computational Mechanics. John Wiley & Sons, Chichester (2007). doi: 10.1002/0470091355.ecm071
  25. 25.
    Matthies, H.G., Keese, A.: Galerkin methods for linear and nonlinear elliptic stochastic partial differential equations. Computer Methods in Applied Mechanics and Engineering 194(12–16), 1295–1331 (2005)Google Scholar
  26. 26.
    Matthies, H.G., Litvinenko, A., Pajonk, O., Rosić, B.V., Zander, E.: Parametric and uncertainty computations with tensor product representations. In: A. Dienstfrey, R. Boisvert (eds.) Uncertainty Quantification in Scientific Computing, IFIP Advances in Information and Communication Technology, vol. 377, pp. 139–150. Springer, Berlin (2012). doi: 10.1007/978-3-642-32677-6 Google Scholar
  27. 27.
    Moselhy, T.A., Marzouk, Y.M.: Bayesian inference with optimal maps. Journal of Computational Physics 231, 7815–7850 (2012). doi: 10.1016/ Google Scholar
  28. 28.
    Pajonk, O., Rosić, B.V., Litvinenko, A., Matthies, H.G.: A deterministic filter for non-Gaussian Bayesian estimation — applications to dynamical system estimation with noisy measurements. Physica D 241, 775–788 (2012). doi: 10.1016/j.physd.2012.01.001 Google Scholar
  29. 29.
    Pajonk, O., Rosić, B.V., Matthies, H.G.: Sampling-free linear Bayesian updating of model state and parameters using a square root approach. Computers and Geosciences 55, 70–83 (2013). doi: 10.1016/j.cageo.2012.05.017 Google Scholar
  30. 30.
    Papoulis, A.: Probability, Random Variables, and Stochastic Processes, third edn. McGraw-Hill Series in Electrical Engineering. McGraw-Hill, New York (1991)Google Scholar
  31. 31.
    Parno, M., Moselhy, T., Marzouk, Y.: A multiscale strategy for Bayesian inference using transport maps. arXiv:1507.07024v1 [stat:CO] (2015)
  32. 32.
    Rao, M.M.: Conditional Measures and Applications. CRC Press, Boca Raton, FL (2005)Google Scholar
  33. 33.
    Roman, L., Sarkis, M.: Stochastic Galerkin method for elliptic SPDEs: A white noise approach. Discrete Cont. Dyn. Syst. Ser. B 6, 941–955 (2006)Google Scholar
  34. 34.
    Rosić, B.V., Kučerová, A., Sýkora, J., Pajonk, O., Litvinenko, A., Matthies, H.G.: Parameter identification in a probabilistic setting. Engineering Structures 50, 179–196 (2013). doi: 10.1016/j.engstruct.2012.12.029 Google Scholar
  35. 35.
    Rosić, B.V., Litvinenko, A., Pajonk, O., Matthies, H.G.: Sampling-free linear Bayesian update of polynomial chaos representations. Journal of Computational Physics 231, 5761–5787 (2012). doi: 10.1016/ Google Scholar
  36. 36.
    Rosić, B.V., Matthies, H.G.: Identification of properties of stochastic elastoplastic systems. In: M. Papadrakakis, G. Stefanou, V. Papadopoulos (eds.) Computational Methods in Stochastic Dynamics, Computational Methods in Applied Sciences, vol. 26, pp. 237–253. Springer, Berlin (2013). doi: 10.1007/978-94-007-5134-7 Google Scholar
  37. 37.
    Saad, G., Ghanem, R.: Characterization of reservoir simulation models using a polynomial chaos-based ensemble Kalman filter. Water Resources Research 45, W04,417 (2009). doi: 10.1029/2008WR007148
  38. 38.
    Sanz-Alonso, D., Stuart, A.M.: Long-time asymptotics of the filtering distribution for partially observed chaotic dynamical systems. arXiv:1411.6510v1 [math.DS] (2014)
  39. 39.
    Segal, I.E., Kunze, R.A.: Integrals and Operators. Springer, Berlin (1978)Google Scholar
  40. 40.
    Stuart, A.M.: Inverse problems: A Bayesian perspective. Acta Numerica 19, 451–559 (2010). doi: 10.1017/S0962492910000061 Google Scholar
  41. 41.
    Tarantola, A.: Inverse Problem Theory and Methods for Model Parameter Estimation. SIAM, Philadelphia, PA (2004)Google Scholar
  42. 42.
    Tarn, T.J., Rasis, Y.: Observers for nonlinear stochastic systems. IEEE Transactions on Automatic Control 21, 441–448 (1976)Google Scholar
  43. 43.
    Wiener, N.: The homogeneous chaos. American Journal of Mathematics 60(4), 897–936 (1938)Google Scholar
  44. 44.
    Xiu, D., Karniadakis, G.E.: The Wiener-Askey polynomial chaos for stochastic differential equations. SIAM Journal of Scientific Computing 24, 619–644 (2002)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.TU BraunschweigBrunswickGermany
  2. 2.KAUSTThuwalSaudi Arabia
  3. 3.ElektrobitBraunschweigGermany
  4. 4.Schlumberger Information Solutions ASKjellerNorway

Personalised recommendations