Skip to main content

Fast Bayesian inversion for high dimensional inverse problems

Abstract

We investigate the use of learning approaches to handle Bayesian inverse problems in a computationally efficient way when the signals to be inverted present a moderately high number of dimensions and are in large number. We propose a tractable inverse regression approach which has the advantage to produce full probability distributions as approximations of the target posterior distributions. In addition to provide confidence indices on the predictions, these distributions allow a better exploration of inverse problems when multiple equivalent solutions exist. We then show how these distributions can be used for further refined predictions using importance sampling, while also providing a way to carry out uncertainty level estimation if necessary. The relevance of the proposed approach is illustrated both on simulated and real data in the context of a physical model inversion in planetary remote sensing. The approach shows interesting capabilities both in terms of computational efficiency and multimodal inference.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

References

  • Adler, J., Öktem, O.: Solving ill-posed inverse problems using iterative deep neural networks. Inverse Prob. 33(12), 124007 (2017)

    MathSciNet  MATH  Article  Google Scholar 

  • Arridge, S., Maass, P., Öktem, O., Schönlieb, C.B.: Solving inverse problems using data-driven models. Acta Numer. 28, 1–174 (2019)

    MathSciNet  MATH  Article  Google Scholar 

  • Balsiger, F., Konar, A.S., Chikop, S., Chandran, V., Scheidegger, O., Geethanath, S., Reyes, M.: Magnetic resonance fingerprinting reconstruction via spatiotemporal convolutional neural networks. In: Knoll, F., Maier, A.K., Rueckert, D. (eds.) Machine Learning for Medical Image Reconstruction—First International Workshop, MLMIR 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 16, 2018, Proceedings, Lecture Notes in Computer Science, vol. 11074, pp. 39–46. Springer (2018)

  • Barbieri, M., Brizi, L., Giampieri, E., Solera, F., Castellani, G., Testa, C., Remondini, D.: Circumventing the Curse of Dimensionality in Magnetic Resonance Fingerprinting through a Deep Learning Approach. arXiv:1811.11477 [physics] (2018)

  • Bardenet, R., Doucet, A., Holmes, C.: Towards scaling up Markov chain Monte Carlo: an adaptive subsampling approach. In: International Conference on Machine Learning (ICML). Proceedings of the 31st International Conference on Machine Learning (ICML), Beijing, China, pp. 405–413 (2014)

  • Bernard-Michel, C., Douté, S., Gardes, L., Girard, S.: Estimation of Mars surface physical properties from hyperspectral images using sliced inverse regression (2007)

  • Bernard-Michel, C., Douté, S., Fauvel, M., Gardes, L., Girard, S.: Retrieval of Mars surface physical properties from OMEGA hyperspectral images using regularized sliced inverse regression. J. Geophys. Res. Planets 114(E6), E06005 (2009)

    Article  Google Scholar 

  • Bertrand, C., Ohmi, M., Suzuki, R., Kado, H.: A probabilistic solution to the MEG inverse problem via MCMC methods: the reversible jump and parallel tempering algorithms. IEEE Trans. Biomed. Eng. 48(5), 533–542 (2001)

    Article  Google Scholar 

  • Bezanson, J., Edelman, A., Karpinski, S., Shah, V.: Julia: a fresh approach to numerical computing. SIAM Rev. 59(1), 65–98 (2017)

    MathSciNet  MATH  Article  Google Scholar 

  • Carreira-Perpinan, M.: Mode-finding for mixtures of Gaussian distributions. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1318–1323 (2000)

    Article  Google Scholar 

  • Ceamanos, X., Douté, S., Fernando, J., Schmidt, F., Pinet, P., Lyapustin, A.: Surface reflectance of Mars observed by CRISM/MRO: 1. Multi-angle approach for retrieval of surface reflectance from CRISM observations (MARS-ReCO). J. Geophys. Res. Planets 118(3), 514–533 (2013)

    Article  Google Scholar 

  • Chiancone, A., Forbes, F., Girard, S.: Student sliced inverse regression. Comput. Stat. Data Anal. 113, 441–456 (2017)

    MathSciNet  MATH  Article  Google Scholar 

  • Cohen, O., Zhu, B., Rosen, M.S.: MR fingerprinting deep reconstruction network (DRONE). Magn. Reson. Med. 80(3), 885–894 (2018)

    Article  Google Scholar 

  • Cook, R.D., Forzani, L.: Partial least squares prediction in high-dimensional regression. Ann. Stat. 47(2), 884–908 (2019)

    MathSciNet  MATH  Article  Google Scholar 

  • Darvishzadeh, R., Matkan, A.A., Ahangar, A.D.: Inversion of a radiative transfer model for estimation of rice canopy chlorophyll content using a lookup-table approach. IEEE J. Select. Top. Appl. Earth Observ. Remote Sens. 5(4), 1222–1230 (2012)

    Article  Google Scholar 

  • Deleforge, A., Forbes, F., Ba, S., Horaud, R.: Hyper-spectral image analysis with partially-latent regression and spatial Markov dependencies. IEEE J. Select. Top. Signal Process. 9(6), 1037–1048 (2015)

    Article  Google Scholar 

  • Deleforge, A., Forbes, F., Horaud, R.: High-dimensional regression with gaussian mixtures and partially-latent response variables. Stat. Comput. 25(5), 893–911 (2015)

    MathSciNet  MATH  Article  Google Scholar 

  • Douté, S., Pilorget, C.: Physical state and temporal evolution of icy surfaces in the Mars South Pole. Eur. Planet. Sci. Congress 11, EPSC2017-491 (2017)

    Google Scholar 

  • Fernando, J., Schmidt, F., Douté, S.: Martian surface microtexture from orbital CRISM multi-angular observations: a new perspective for the characterization of the geological processes. Planet Space Sci. 128, 30–51 (2016)

    Article  Google Scholar 

  • Frau-Pascual, A., Vincent, T., Sloboda, J., Ciuciu, P., Forbes, F.: Physiologically Informed Bayesian Analysis of ASL fMRI Data. In: Cardoso, M.J., Simpson, I., Arbel, T., Precup, D. Ribbens, A. (eds.) Bayesian and Graphical Models for Biomedical Imaging, Lecture Notes in Computer Science, pp. 37–48. Springer (2014)

  • Giovannelli, J.F., Idier, J.: Regularization and Bayesian Methods for Inverse Problems in Signal and Image Processing. Wiley, New York (2015)

    Book  Google Scholar 

  • Golbabaee, M., Chen, D., Gómez, P.A., Menzel, M.I., Davies, M.: Geometry of deep learning for magnetic resonance fingerprinting. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), vol. 2019, pp. 7825–7829 (2019)

  • Hapke, B.: Bidirectional reflectance spectroscopy. I—Theory. J. Geophys. Res. 86, 3039–3054 (1981)

    Article  Google Scholar 

  • Hapke, B.: Bidirectional reflectance spectroscopy. III—Correction for macroscopic roughness. Icarus 59, 41–59 (1984)

    Article  Google Scholar 

  • Hapke, B.: Bidirectional reflectance spectroscopy: 4. The extinction coefficient and the opposition effect. Icarus 67(2), 264–280 (1986)

    Article  Google Scholar 

  • Hennig, C.: Methods for merging Gaussian mixture components. Adv. Data Anal. Classif. 4(1), 3–34 (2010)

    MathSciNet  MATH  Article  Google Scholar 

  • Hoffman, M.D., Gelman, A.: The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo. arXiv:1111.4246 [cs, stat] (2011)

  • Hoppe, E., Körzdörfer, G., Würfl, T., Wetzl, J., Lugauer, F., Pfeuffer, J., Maier, A.: Deep learning for magnetic resonance fingerprinting: a new approach for predicting quantitative parameter values from time series. Stud. Health Technol. Inf. 243, 202–206 (2017)

    Google Scholar 

  • Hovorka, R., Canonico, V., Chassin, L.J., Haueter, U., Massi-Benedetti, M., Federici, M.O., Pieber, T.R., Schaller, H.C., Schaupp, L., Vering, T., Wilinska, M.E.: Nonlinear model predictive control of glucose concentration in subjects with type 1 diabetes. Physiol. Meas. 25(4), 905–920 (2004)

    Article  Google Scholar 

  • Ingrassia, S., Minotti, S.C., Vittadini, G.: Local statistical modeling via a cluster-weighted approach with elliptical distributions. J. Classif. 29(3), 363–401 (2012)

    MathSciNet  MATH  Article  Google Scholar 

  • Izbicki, R., Lee, A.B., Pospisil, T.: ABC-CDE: toward approximate bayesian computation with complex high-dimensional data and limited simulations. J. Comput. Graph. Stat. 28(3), 481–492 (2019)

    MathSciNet  MATH  Article  Google Scholar 

  • Lathuiliere, S., Juge, R., Mesejo, P., Munoz-Salinas, R., Horaud, R.: Deep mixture of linear inverse regressions applied to head-pose estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4817–4825 (2017)

  • Lemasson, B., Pannetier, N., Coquery, N., Boisserand, L.S.B., Collomb, N., Schuff, N., Moseley, M., Zaharchuk, G., Barbier, E.L., Christen, T.: MR vascular fingerprinting in stroke and brain tumors models. Sci. Rep. 6, 37071 (2016)

    Article  Google Scholar 

  • Li, K.C.: Sliced inverse regression for dimension reduction. J. Am. Stat. Assoc. 86(414), 316–327 (1991)

    MathSciNet  MATH  Article  Google Scholar 

  • Ma, D., Gulani, V., Seiberlich, N., Liu, K., Sunshine, J.L., Duerk, J.L., Griswold, M.A.: Magnetic resonance fingerprinting. Nature 495(7440), 187–192 (2013)

    Article  Google Scholar 

  • Martin, J., Wilcox, L.C., Burstedde, C., Ghattas, O.: A stochastic newton MCMC method for large-scale statistical inverse problems with application to seismic inversion. SIAM J. Sci. Comput. 34(3), A1460–A1487 (2012)

    MathSciNet  MATH  Article  Google Scholar 

  • Mesejo, P., Saillet, S., David, O., Bénar, C., Warnking, J.M., Forbes, F.: A differential evolution-based approach for fitting a nonlinear biophysical model to fMRI BOLD data. IEEE J. Select. Top. Signal Process. 10(2), 416–427 (2016)

    Article  Google Scholar 

  • Murchie, S.L., Seelos, F.P., Hash, C.D., Humm, D.C., Malaret, E., McGovern, J.A., Choo, T.H., Seelos, K.D., Buczkowski, D.L., Morgan, M.F., Barnouin-Jha, O.S., Nair, H., Taylor, H.W., Patterson, G.W., Harvel, C.A., Mustard, J.F., Arvidson, R.E., McGuire, P., Smith, M.D., Wolff, M.J., Titus, T.N., Bibring, J.P., Poulet, F.: Compact reconnaissance imaging spectrometer for Mars investigation and data set from the mars reconnaissance orbiters primary science phase. J. Geophys. Res. Planets 114(E2), E00D07 (2009)

    Google Scholar 

  • Nataraj, G., Nielsen, J.F., Scott, C., Fessler, J.A.: Dictionary-free MRI PERK: parameter estimation via regression with kernels. IEEE Trans. Med. Imaging 37(9), 2103–2114 (2018)

    Article  Google Scholar 

  • Nguyen, H.D., Chamroukhi, F., Forbes, F.: Approximation results regarding the multiple-output Gaussian gated mixture of linear experts model. Neurocomputing 366, 208–214 (2019)

    Article  Google Scholar 

  • Perthame, E., Forbes, F., Deleforge, A.: Inverse regression approach to robust nonlinear high-to-low dimensional mapping. J. Multivar. Anal. 163, 1–14 (2018)

    MathSciNet  MATH  Article  Google Scholar 

  • Pilorget, C., Fernando, J., Ehlmann, B.L., Schmidt, F., Hiroi, T.: Wavelength dependence of scattering properties in the VIS-NIR and links with grain-scale physical and compositional properties. Icarus 267, 296–314 (2016)

    Article  Google Scholar 

  • Potin, S., Beck, P., Schmitt, B., Moynier, F.: Some things special about NEAs: geometric and environmental effects on the optical signatures of hydration. Icarus 333, 415–428 (2019)

    Article  Google Scholar 

  • Raftery, A.E., Bao, L.: Estimating and projecting trends in HIV/AIDS generalized epidemics using incremental mixture importance sampling. Biometrics 66(4), 1162–1173 (2010)

    MathSciNet  MATH  Article  Google Scholar 

  • Ray, S., Lindsay, B.G.: The topography of multivariate normal mixtures. Ann. Stat. 33(5), 2042–2065 (2005)

    MathSciNet  MATH  Article  Google Scholar 

  • Robert, C., Casella, G.: Monte Carlo Statistical Methods. Springer Texts in Statistics, 2nd edn. Springer, New York (2004)

    MATH  Book  Google Scholar 

  • Runnalls, A.: Kullback–Leibler approach to Gaussian mixture reduction. IEEE Trans. Aerosp. Electron. Syst. 43, 989–999 (2007)

    Article  Google Scholar 

  • Schmidt, F., Fernando, J.: Realistic uncertainties on Hapke model parameters from photometric measurement. Icarus 260, 73–93 (2015)

    Article  Google Scholar 

  • Sisson, S.A., Fan, Y., Beaumont, M.: Handbook of Approximate Bayesian Computation. CRC Press, Boca Raton (2018)

    MATH  Book  Google Scholar 

  • Sobol, I.M.: On the distribution of points in a cube and the approximate evaluation of integrals. USSR Comput. Math. Math. Phys. 7(4), 86–112 (1967)

    MathSciNet  MATH  Article  Google Scholar 

  • Song, P., Eldar, Y.C., Mazor, G., Rodrigues, M.R.D.: HYDRA: hybrid deep magnetic resonance fingerprinting. Med. Phys. 46(11), 4951–4969 (2019)

    Article  Google Scholar 

  • Steele, R.J., Raftery, A.E., Emond, M.J.: Computing normalizing constants for finite mixture models via incremental mixture importance sampling (IMIS). J. Comput. Graph. Stat. 15(3), 712–734 (2006)

    MathSciNet  Article  Google Scholar 

  • Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., Fergus, R.: Intriguing properties of neural networks. In: 2nd International Conference on Learning Representations, ICLR 2014 (2014)

  • Tarantola, A.: Inverse Problem Theory and Methods for Model Parameter Estimation. Other Titles in Applied Mathematics. Society for Industrial and Applied Mathematics (2005)

  • Tarantola, A., Valette, B., et al.: Inverse problems quest for information. J. Geophys. 50(1), 159–170 (1982)

    Google Scholar 

  • Tu, C.C., Forbes, F., Lemasson, B., Wang, N.: Prediction with high dimensional regression via hierarchically structured Gaussian mixtures and latent variables. J. R. Stat. Soc. Ser. C Appl. Stat. 68(5), 1485–1507 (2019)

    MathSciNet  Article  Google Scholar 

  • Virtue, P., Yu, S.X., Lustig, M.: Better than real: complex-valued neural nets for MRI fingerprinting. In: 2017 IEEE International Conference on Image Processing (ICIP), pp. 3953–3957 (2017)

  • Zhao, B., Setsompop, K., Ye, H., Cauley, S.F., Wald, L.L.: Maximum likelihood reconstruction for magnetic resonance fingerprinting. IEEE Trans. Med. Imaging 35(8), 1812–1823 (2016)

    Article  Google Scholar 

Download references

Acknowledgements

This article was developed in the framework of the Grenoble Alpes Data Institute, supported by the French National Research Agency under the “Investissements d’avenir” program (ANR-15-IDEX-02).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Florence Forbes.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

Computation times

The simulations ran on a laptop with 4 cores (at 2.5 Ghz). Table 6 shows computation times. For each experiment, the time is divided in two parts, the time for the learning step (GLLiM inference) and the time for the prediction step, which consists either of mixture merging, mode-finding and importance sampling (Examples 1 to 6) or of noise level estimation via the EM algorithm. Most experiments run in few minutes. The complexity of the forward model and the way it is implemented can take an important part in the resulting running time. This appears in the comparison of examples 2 and 3 which mainly differ in the choice of F. The Hapke model (Example 3) benefits from a more efficient implementation which explains running time twice smaller for similar settings. For equivalent forward model implementations, the time depends mainly on the size and dimensionality of the learning set and on the number of inversions to be performed. Learning sets have equivalent complexity in our experiments, except for the high dimensional example (Example 4, \(D=336\)). Higher computation times are observed in case of massive inversions. In particular, Example 6 with 156,100 inversions takes few hours. We believe it is the first time that such spatial and spectral parametric maps are obtained due to the intractability of other methods in this setting.

Fig. 10
figure 10

Inversion of Nontronite laboratory observations. \(\bar{{\mathbf {x}}}_{IMIS-G}\) is in red, \(\bar{{\mathbf {x}}}_{IMIS-centroid,1}\) and \(\bar{{\mathbf {x}}}_{IMIS-centroid,2}\) are in blue. Relative reconstruction errors are shown with a logarithmic scale in the top right plot

Centroids predictions for the Nontronite dataset

Figure 10 provides a complementary analysis to the results in Fig. 5, for Example 5. The centroids predictions show that, for parameters \({\bar{\theta }}, b,c\), the marginal posteriors are not concentrated around their means, yielding a large range of high probability predictions including the centroids, as shown by the reconstruction errors.

Massive inversion of spatial and spectral Mars data

Figure 11 shows the maps for parameters b and c after inversion of the real Mars data described in Sect. 7.2.5.

Fig. 11
figure 11

Mars South pole dataset. Parameters b (top) and c (bottom) averaged over spectral dimension, predicted using \(\bar{{\mathbf {x}}}_{IMIS-G}\) (left) or \(\hat{{\mathbf {x}}}_{best}\) (right)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Kugler, B., Forbes, F. & Douté, S. Fast Bayesian inversion for high dimensional inverse problems. Stat Comput 32, 31 (2022). https://doi.org/10.1007/s11222-021-10019-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11222-021-10019-5

Keywords

  • Inverse problems
  • Bayesian analysis
  • Mixtures of Gaussians
  • Importance sampling
  • Remote sensing
  • Planetary science