Skip to main content
SpringerLink
Log in

Robust inversion, dimensionality reduction, and randomized sampling

  • Full Length Paper
  • Series B
  • Published:
Mathematical Programming Submit manuscript

Abstract

We consider a class of inverse problems in which the forward model is the solution operator to linear ODEs or PDEs. This class admits several dimensionality-reduction techniques based on data averaging or sampling, which are especially useful for large-scale problems. We survey these approaches and their connection to stochastic optimization. The data-averaging approach is only viable, however, for a least-squares misfit, which is sensitive to outliers in the data and artifacts unexplained by the forward model. This motivates us to propose a robust formulation based on the Student’s t-distribution of the error. We demonstrate how the corresponding penalty function, together with the sampling approach, can obtain good results for a large-scale seismic inverse problem with 50 % corrupted data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Aravkin, A.: Robust methods with applications to Kalman smoothing and Bundle adjustment. PhD thesis, University of Washington, Seattle (2010)

  2. Aravkin, A., van Leeuwen, T., Friedlander, M.P.: Robust inversion via semistochastic dimensionality reduction. In: Proceedings of ICASSP, IEEE (2012)

  3. Aravkin, A., van Leeuwen, T., Herrmann, F.: Robust full waveform inversion with students t-distribution. In: Proceedings of the SEG, Society for Exploration Geophysics, San Antonio, Texas (2011)

  4. Avron H., Toledo S.: Randomized algorithms for estimating the trace of an implicit symmetric positive semi-definite matrix. J. ACM 58, 8–1834 (2011)

    Article  MathSciNet  Google Scholar 

  5. Bertsekas D., Tsitsiklis J.: Neuro-dynamic Programming. Athena Scientific, Belmont (1996)

    MATH  Google Scholar 

  6. Bertsekas D.P., Tsitsiklis J.N.: Gradient convergence in gradient methods with errors. SIAM J. Optim. 10, 627–642 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  7. Brossier R., Operto S., Virieux J.: Which data residual norm for robust elastic frequency-domain full waveform inversion?. Geophysics 75, R37–R46 (2010)

    Google Scholar 

  8. Bube K.P., Langan R.T.: Hybrid ℓ 1/ℓ 2 minimization with applications to tomography. Geophysics 62, 1183–1195 (1997)

    Google Scholar 

  9. Bube K.P., Nemeth T.: Fast line searches for the robust solution of linear systems in the hybrid ℓ 1/ℓ 2 and huber norms. Geophysics 72, A13–A17 (2007)

    Google Scholar 

  10. Byrd R.H., Chin G.M., Neveitt W., Nocedal J.: On the use of stochastic hessian information in optimization methods for machine learning. SIAM J. Optim. 21, 977–995 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  11. Cochran W.G.: Sampling Techniques. 3rd edn. Wiley, New York (1977)

    MATH  Google Scholar 

  12. Fahrmeir L., Kunstler R.: Penalized likelihood smoothing in robust state space models. Metrika 49, 173–191 (1998)

    Article  MathSciNet  Google Scholar 

  13. Farquharson C.G., Oldenburg D.W.: Non-linear inversion using general measures of data misfit and model structure. Geophys. J. Intern. 134, 213–227 (1998)

    Article  Google Scholar 

  14. Friedlander, M.P., Schmidt, M.: Hybrid deterministic-stochastic methods for data fitting. Technical report, University of British Columbia, April 2011, revised September 2011 (2011)

  15. Golub G.H., von Matt U.: Quadratically constrained least squares and quadratic problems. Numer. Math. 59, 561–580 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  16. Guitton A., Symes W.W.: Robust inversion of seismic data using the huber norm. Geophysics 68, 1310–1319 (2003)

    Google Scholar 

  17. Haber, E., Chung, M., Herrmann, F.J.: An effective method for parameter estimation with pde constraints with multiple right hand sides. Technical report TR-2010-4, UBC-Earth and Ocean Sciences Department. SIAM J. Optim. (2010, to appear)

  18. Hampel, F.R., Ronchetti, E.M., Rousseeuw, P.J., Stahel, W.A.: Robust Statistics: The Approach Based on Influence Functions. Wiley (1986)

  19. Herrmann, F., Friedlander, M.P., Yılmaz, O.: Fighting the curse of dimensionality: compressive sensing in exploration seismology. IEEE Signal Proc. Magazine 29(3), 88–100 (2012)

    Google Scholar 

  20. Huber P.J.: Robust Statistics. Wiley, New York (1981)

    Book  MATH  Google Scholar 

  21. Huber P.J., Ronchetti E.M.: Robust Statistics. 2nd edn. Wiley, New York (2009)

    Book  MATH  Google Scholar 

  22. Hutchinson M.: A stochastic estimator of the trace of the influence matrix for laplacian smoothing splines. Commun. Stat. Simul. Computation 19, 433–450 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  23. Kaczmarz S.: Angenäherte auflösung von systemen linearer gleichungen. Bull. Int. Acad. Polon. Sci. A 355, 357 (1937)

    Google Scholar 

  24. Krebs J.R., Anderson J.E., Hinkley D., Neelamani R., Lee S., Baumstein A., Lacasse M.-D.: Fast full-wavefield seismic inversion using encoded sources. Geophysics 74, WCC177–WCC188 (2009)

    Google Scholar 

  25. Lange K.L., Little K.L., Taylor J.M.G.: Robust statistical modeling using the t distribution. J. Am. Stat. Assoc. 84, 881–896 (1989)

    MathSciNet  Google Scholar 

  26. Lohr S.L.: Sampling: Design and Analysis. Duxbury Press, Pacific Grove (1999)

    MATH  Google Scholar 

  27. Maronna, R.A., Martin, D., Yohai: Robust Statistics. Wiley, New York (2006)

  28. Nedic, A., Bertsekas, D.: Convergence rate of incremental subgradient algorithms. In: Uryasev, S., Pardalos, P.M. (eds.) Stochastic optimization: algorithms and applications, pp. 263–304. Kluwer Academic Publishers, Dordrecht (2000)

  29. Nemirovski, A.: Efficient methods in convex programming, Lecture notes (1994)

  30. Nemirovski A., Juditsky A., Lan G., Shapiro A.: Robust stochastic approximation approach to stochastic programming. SIAM J. Optim. 19, 1574–1609 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  31. Nocedal J., Wright S.J.: Numerical Optimization. Springer, Berlin (1999)

    Book  MATH  Google Scholar 

  32. Pratt R., Worthington M.: Inverse theory applied to multi-source cross-hole tomography. Part I: acoustic wave-equation method. Geophys. Prospect. 38, 287–310 (1990)

    Article  Google Scholar 

  33. Prékopa A.: Logarithmic concave measures with application to stochastic programming. Acta Sci. Math. (Szeged) 32, 301–316 (1971)

    MathSciNet  MATH  Google Scholar 

  34. Rockafellar R.T.: Convex Analysis. Priceton Landmarks in Mathematics. Princeton University Press, Princeton (1970)

    Google Scholar 

  35. Shevlyakov G., Morgenthaler S., Shurygin A.: Redescending m-estimators. J. Stat. Plan. Inference 138, 2906–2917 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  36. Solodov M.: Incremental gradient algorithms with stepsizes bounded away from zero. Comput. Optim. Appl. 11, 23–35 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  37. Strohmer T., Vershynin R.: A randomized Kaczmarz algorithm with exponential convergence. J. Fourier Anal. Appl. 15, 262–278 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  38. Tarantola A.: Inversion of seismic reflection data in the acoustic approximation. Geophysics 49, 1259–1266 (1984)

    Google Scholar 

  39. Tseng P.: An incremental gradient(-projection) method with momentum term and adaptive stepsize rule. SIAM J. Optim. 8, 506–531 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  40. van den Doel, K., Ascher, U.M.: Adaptive and stochastic algorithms for electrical impedance tomography and dc resistivity problems with piecewise constant solutions and many measurements. SIAM J. Sci. Comput. 34, A185–A205 (2012)

    Google Scholar 

  41. van Leeuwen, T., Aravkin, A., Herrmann, F.: Seismic waveform inversion by stochastic optimization. Int. J. Geophys. (2011), (p. ID 689041)

  42. Virieux J., Operto S.: An overview of full-waveform inversion in exploration geophysics. Geophysics 74, 127–152 (2009)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Earth and Ocean Sciences, University of British Columbia, Vancouver, BC, Canada

    Aleksandr Aravkin, Felix J. Herrmann & Tristan van Leeuwen

  2. Department of Computer Science, University of British Columbia, Vancouver, BC, Canada

    Michael P. Friedlander

Authors
  1. Aleksandr Aravkin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael P. Friedlander
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Felix J. Herrmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tristan van Leeuwen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael P. Friedlander.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Aravkin, A., Friedlander, M.P., Herrmann, F.J. et al. Robust inversion, dimensionality reduction, and randomized sampling. Math. Program. 134, 101–125 (2012). https://doi.org/10.1007/s10107-012-0571-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10107-012-0571-6

Keywords

Mathematics Subject Classification

Search

Navigation