Computational Geosciences

, Volume 16, Issue 3, pp 593–611 | Cite as

Inverse problems with non-trivial priors: efficient solution through sequential Gibbs sampling

  • Thomas Mejer Hansen
  • Knud Skou Cordua
  • Klaus Mosegaard
Original Paper


Markov chain Monte Carlo methods such as the Gibbs sampler and the Metropolis algorithm can be used to sample solutions to non-linear inverse problems. In principle, these methods allow incorporation of prior information of arbitrary complexity. If an analytical closed form description of the prior is available, which is the case when the prior can be described by a multidimensional Gaussian distribution, such prior information can easily be considered. In reality, prior information is often more complex than can be described by the Gaussian model, and no closed form expression of the prior can be given. We propose an algorithm, called sequential Gibbs sampling, allowing the Metropolis algorithm to efficiently incorporate complex priors into the solution of an inverse problem, also for the case where no closed form description of the prior exists. First, we lay out the theoretical background for applying the sequential Gibbs sampler and illustrate how it works. Through two case studies, we demonstrate the application of the method to a linear image restoration problem and to a non-linear cross-borehole inversion problem. We demonstrate how prior information can reduce the complexity of an inverse problem and that a prior with little information leads to a hard inverse problem, practically unsolvable except when the number of model parameters is very small. Considering more complex and realistic prior information thus not only makes realizations from the posterior look more realistic but it can also reduce the computation time for the inversion dramatically. The method works for any statistical model for which sequential simulation can be used to generate realizations. This applies to most algorithms developed in the geostatistical community.


Inverse problem theory Geostatistics Geology Prior information 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Besag, J.: On the statistical analysis of dirty pictures. J. R. Stat. Soc., B 48(3), 259–302 (1986)MathSciNetzbMATHGoogle Scholar
  2. 2.
    Bosch, M.: Lithologic tomography: from plural geophysical data to lithology estimation. J. Geophys. Res. 104(B1), 749–766 (1999)CrossRefGoogle Scholar
  3. 3.
    Box, G., Tiao, G.: Bayesian Inference in Statistical Analysis. Wiley, New York (1973)zbMATHGoogle Scholar
  4. 4.
    Brooks, S., Gelman, A.: General methods for monitoring convergence of iterative simulations. J. Comput. Graph. Stat. 7, 434–455 (1998)MathSciNetGoogle Scholar
  5. 5.
    Buland, A., Omre, H.: Bayesian linearized AVO inversion. Geophysics 68, 185 (2003)CrossRefGoogle Scholar
  6. 6.
    Caers, J., Hoffman, T.: The probability perturbation method: a new look at Bayesian inverse modeling. Math. Geol. 38(1), 81–100 (2006)MathSciNetzbMATHCrossRefGoogle Scholar
  7. 7.
    Cowles, M., Carlin, B.: Markov chain Monte Carlo convergence diagnostics: a comparative review. J. Am. Stat. Assoc. 91, 883–904 (1996)MathSciNetzbMATHGoogle Scholar
  8. 8.
    Deutsch, C.V., Tran, T.T., Xie, Y.: An Approach to Ensure Histogram Reproduction in Direct Sequential Simulation. Tech. rep. Centre for Computational Geostatistics, University of Alberta, Edmonton, Alberta (2000)Google Scholar
  9. 9.
    Fu, J., Gomez-Hernandez, J.: Preserving spatial structure for inverse stochastic simulation using blocking Markov chain Monte Carlo method. Inverse Probl. Sci. Eng. 16(7), 865–884 (2008)MathSciNetzbMATHCrossRefGoogle Scholar
  10. 10.
    Fu, J., Gómez-Hernández, J.: A blocking Markov chain Monte Carlo method for inverse stochastic hydrogeological modeling. Math. Geosci. 41(2), 105–128 (2009)zbMATHCrossRefGoogle Scholar
  11. 11.
    Gelman, A., Roberts, G., Gilks, W.: Efficient metropolis jumping rules. In: Bernardo, J., Berger, K., Dawid, A., Smith, A. (eds.) Bayesian Statistics, vol. 5, pp. 599–608. Clarendon, Oxford. (1996)Google Scholar
  12. 12.
    Gelman, A., Rubin, D.: Inference from iterative simulation using multiple sequences. Stat. Sci. 7, 457–472 (1992)CrossRefGoogle Scholar
  13. 13.
    Geman, S., Geman, D.: Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Mach. Intell. 6, 721–741 (1984)zbMATHCrossRefGoogle Scholar
  14. 14.
    Gomez-Hernandez, J., Journel, A.: Joint sequential simulation of multi-Gaussian fields. In: Soares, A. (ed.) Geostatistics Troia, vol. 92, pp. 85–94. Kluwer Academic, Dordrecht (1993)CrossRefGoogle Scholar
  15. 15.
    Goovaerts, P.: Geostatistics for Natural Resources Evaluation. Applied Geostatistics Series. Oxford University Press, Oxford (1997)Google Scholar
  16. 16.
    Guardiano, F., Srivastava, R.: Multivariate geostatistics: beyond bivariate moments. In: Soares, A. (ed.) Geostatistics-Troia, vol. 1, pp. 133–144. Kluwer Academic, Dordrecht (1993)Google Scholar
  17. 17.
    Hansen, T.M., Journel, A.G., Tarantola, A., Mosegaard, K.: Linear inverse Gaussian theory and geostatistics. Geophysics 71(6), R101–R111 (2006)CrossRefGoogle Scholar
  18. 18.
    Hansen, T.M., Mosegaard, K.: VISIM: sequential simulation for linear inverse problems. Comput Geosci 34(1), 53–76 (2008)CrossRefGoogle Scholar
  19. 19.
    Hansen, T.M., Mosegaard, K., Cordua, K.C.: Using geostatistics to describe complex a priori information for inverse problems. In: Ortiz, J.M., Emery, X. (eds.) VIII International Geostatistics Congress, vol. 1, pp. 329–338. Mining Engineering Department, University of Chile, Santiago (2008)Google Scholar
  20. 20.
    Hastings, W.: Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57(1), 97 (1970)zbMATHCrossRefGoogle Scholar
  21. 21.
    Irving, J., Singha, K.: Stochastic inversion of tracer test and electrical geophysical data to estimate hydraulic conductivities. Water Resour. Res 46 (2010)Google Scholar
  22. 22.
    Jaynes, E.: Highly informative priors. In: Bernardo, J., DeGroot, M., Lindley, D., Smith, A. (eds.) Bayesian Statistics, vol. 2, pp. 329–360. Elsevier, Amsterdam (1985)Google Scholar
  23. 23.
    Jaynes, E., Bretthorst, G.: Probability Theory: the Logic of Science. Cambridge University Press, Cambridge (2003)zbMATHCrossRefGoogle Scholar
  24. 24.
    Journel, A., Zhang, T.: The necessity of a multiple-point prior model. Math. Geol. 38(5), 591–610 (2006)zbMATHCrossRefGoogle Scholar
  25. 25.
    Journel, A.G.: Modeling uncertainty: some conceptual thoughts. In: Dimitrakopoulos, R. (ed.) Geostatistics for the Next Century, pp. 30–43. Kluwer, Dordrecht (1994)CrossRefGoogle Scholar
  26. 26.
    Journel, A.G., Huijbregts, C.J.: Mining Geostatistics. Academic, Amsterdam (1978)Google Scholar
  27. 27.
    Khan, A., Mosegaard, K.: An inquiry into the lunar interior: a nonlinear inversion of the Apollo lunar seismic data. J. Geophys. Res 107(E6), 19–44 (2002)CrossRefGoogle Scholar
  28. 28.
    Kirkpatrick, S., Jr., D., Vecchi, M.: Optimization by simulated annealing. Science 220(4598), 671–680 (1983)MathSciNetzbMATHCrossRefGoogle Scholar
  29. 29.
    Le Ravalec-Dupin, M., Noetinger, B.: Optimization with the gradual deformation method. Math. Geol. 34(2), 125–142 (2002)zbMATHCrossRefGoogle Scholar
  30. 30.
    Mariethoz, G., Renard, P., Caers, J.: Bayesian inverse problem and optimization with iterative spatial resampling. Water Resour. Res. 46(11), W11,530 (2010)CrossRefGoogle Scholar
  31. 31.
    Metropolis, N., Rosenbluth, M., Rosenbluth, A., Teller, A., Teller, E.: Equation of state calculations by fast computing machines. J. Chem. Phys. 21, 1087–1092 (1963)CrossRefGoogle Scholar
  32. 32.
    Mosegaard, K.: Resolution analysis of general inverse problems through inverse Monte Carlo sampling. Inverse Probl. 14, 405 (1998)MathSciNetzbMATHCrossRefGoogle Scholar
  33. 33.
    Mosegaard, K.: Monte Carlo Analysis of Inverse Problems. University of Copenhagen, Copenhagen (2006)Google Scholar
  34. 34.
    Mosegaard, K.: Quest for consistency, symmetry and simplicity—the legacy of Albert Tarantola. Geophysics 76, W51–W61 (2011)CrossRefGoogle Scholar
  35. 35.
    Mosegaard, K., Sambridge, M.: Monte carlo analysis of inverse problems. Inverse Probl. 18(3), 29–54 (2002)MathSciNetCrossRefGoogle Scholar
  36. 36.
    Mosegaard, K., Singh, S., Snyder, D., Wagner, H.: Monte Carlo analysis of seismic reflector from Moho and the W reflector. J. Geophys. Res. 102(B2), 2969–2981 (1997)CrossRefGoogle Scholar
  37. 37.
    Mosegaard, K., Tarantola, A.: Monte Carlo sampling of solutions to inverse problems. J. Geophys. Res. 100(B7), 12431–12447 (1995)CrossRefGoogle Scholar
  38. 38.
    Mustapha, H., Dimitrakopoulos, R.: HOSIM: a high-order stochastic simulation algorithm for generating three-dimensional complex geological patterns. Comput Geosci (2010)Google Scholar
  39. 39.
    Oz, B., Deutsch, C.V., Tran, T.T., Xie, Y.: DSSIM-HR: a FORTRAN 90 program for direct sequential simulation with histogram reproduction. Comput. Geosci. 29(1), 39–51 (2003). doi: 10.1016/S0098-3004(02)00071-7 CrossRefGoogle Scholar
  40. 40.
    Remy, N., Boucher, A., Wu, J.: Applied Geostatistics with SGeMS: A User’s Guide. Cambridge University Press, Cambridge (2008)Google Scholar
  41. 41.
    Scales, J., Sneider, R.: To Bayes or not to Bayes? Geophysics 62(4), 1045–1046 (1997)Google Scholar
  42. 42.
    Scales, J., Tenorio, L.: Prior information and uncertainty in inverse problems. Geophysics 66(2), 389 (2001)CrossRefGoogle Scholar
  43. 43.
    Soares, A.: Direct sequential simulation and cosimulation. Math. Geol 33(8), 911–926 (2001)MathSciNetzbMATHCrossRefGoogle Scholar
  44. 44.
    Strebelle, S.: Sequential simulation drawing structures from training images. Ph.D. thesis, Stanford University (2000)Google Scholar
  45. 45.
    Strebelle, S.: Conditional simulation of complex geological structures using multiple-point statistics. Math. Geol 34(1), 1–20 (2002)MathSciNetzbMATHCrossRefGoogle Scholar
  46. 46.
    Tarantola, A.: Inverse Problem Theory and Methods for Model Parameter Estimation. SIAM, Philadelphia (2005)zbMATHCrossRefGoogle Scholar
  47. 47.
    Tarantola, A., Valette, B.: Inverse problems = quest for information. J. Geophys 50(3), 150–170 (1982)Google Scholar
  48. 48.
    Tjelmeland, H., Besag, J.: Markov random fields with higher-order interactions. Scand. J. Statist. 25(3), 415–433 (1998)MathSciNetzbMATHCrossRefGoogle Scholar
  49. 49.
    Voss, P., Mosegaard, K., Gregersen, S.: The Tornquist zone, a north east inclining lithospheric transition at the south western margin of the Baltic shield: revealed through a nonlinear teleseismic tomographic inversion. Tectonophysics 416(1–4), 151–166 (2006)CrossRefGoogle Scholar
  50. 50.
    Wu, J., Boucher, A., Zhang, T.: A SGeMS code for pattern simulation of continuous and categorical variables: FILTERSIM. Comput Geosci 34(12), 1863–1876 (2008)CrossRefGoogle Scholar
  51. 51.
    Zelt, C., Barton, P.: Three-dimensional seismic refraction tomography—a comparison of two methods applied to data from the Faeroe Basin. J. Geophys. Res. 103(B4), 7187–7210 (1998)CrossRefGoogle Scholar
  52. 52.
    Zhang, T.: Incorporating geological conceptual models and interpretations into reservoir modeling using multiple-point geostatistics. Earth Sci Front 15(1), 26–35 (2008). doi: 10.1016/S1872-5791(08)60016-0, CrossRefGoogle Scholar
  53. 53.
    Zhang, T., Lu, D., Li, D.: Porous media reconstruction using a cross-section image and multiple-point geostatistics. Advanced Computer Control, International Conference, pp 24–29 (2009). doi:

Copyright information

© Springer Science+Business Media B.V. 2012

Authors and Affiliations

  • Thomas Mejer Hansen
    • 1
  • Knud Skou Cordua
    • 1
  • Klaus Mosegaard
    • 1
  1. 1.Center for Energy Resources Engineering, DTU InformaticsTechnical University of DenmarkLyngbyDenmark

Personalised recommendations