Numerical Algorithms for Non-smooth Optimization Applicable to Seismic Recovery

  • Ignace Loris
Living reference work entry

Latest version View entry history


Inverse problems in seismic tomography are often cast in the form of an optimization problem involving a cost function composed of a data misfit term and regularizing constraint or penalty. Depending on the noise model that is assumed to underlie the data acquisition, these optimization problems may be non-smooth. Another source of lack of smoothness (differentiability) of the cost function may arise from the regularization method chosen to handle the ill-posed nature of the inverse problem. A numerical algorithm that is well suited to handle minimization problems involving two non-smooth convex functions and two linear operators is studied. The emphasis lies on the use of some simple proximity operators that allow for the iterative solution of non-smooth convex optimization problems. Explicit formulas for several of these proximity operators are given and their application to seismic tomography is demonstrated.


Convex Function Minimization Problem Iterative Algorithm Convex Optimization Noise Model 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



I.L. is a research associate of the Fonds de la Recherche Scientifique (FNRS).


  1. Afonso MV, Bioucas-Dias JM, Figueiredo MAT (2010) Fast image recovery using variable splitting and constrained optimization. IEEE Trans Image Process 19(9):2345–2356MathSciNetCrossRefGoogle Scholar
  2. Beck A, Teboulle M (2009) Fast gradient-based algorithms for constrained total variation image denoising and deblurring problems. IEEE Trans Image Process 18:2419–2434MathSciNetCrossRefGoogle Scholar
  3. Beck A, Teboulle M (2009) A fast iterative shrinkage-threshold algorithm for linear inverse problems. SIAM J Imaging Sci 2:183–202MathSciNetCrossRefzbMATHGoogle Scholar
  4. Bonesky T (2009) Morozov’s discrepancy principle and Tikhonov-type functionals. Inverse Probl 25(1):015015 (11pp)Google Scholar
  5. Boyd S, Vandenberghe L (2004) Convex optimization. Cambridge University PressCrossRefzbMATHGoogle Scholar
  6. Bredies K (2009) A forward-backward splitting algorithm for the minimization of non-smooth convex functionals in Banach space. Inverse Probl 25:015005MathSciNetCrossRefGoogle Scholar
  7. Bruckstein AM, Donoho DL, Elad M (2009) From sparse solutions of systems of equations to sparse modeling of signals and images. SIAM Rev 51(1):34–81MathSciNetCrossRefzbMATHGoogle Scholar
  8. Candès EJ, Romberg J (2004) Practical signal recovery from random projections. In: Wavelet applications in signal and image processing XI, Proceedings of the SPIE Conference, vol 5914, San Diego CA, USAGoogle Scholar
  9. Chambolle A, Pock T (2011) A first-order primal-dual algorithm for convex problems with applications to imaging. J Math Imaging Vis 40:120–145 hal-00490826Google Scholar
  10. Chen P, Huang J, Zhang X (2013) A primal-dual fixed point algorithm for convex separable minimization with applications to image restoration. Inverse Probl 29(2):025011 (33pp)Google Scholar
  11. Claerbout JF, Muir F (1973) Robust modelling with erratic data. Geophysics 38(5):826–844CrossRefGoogle Scholar
  12. Combettes PL, Pesquet J-C (2011) Proximal splitting methods in signal processing, chapter 1. In: Bauschke HH, Burachik RS, Combettes PL, Elser V, Luke DR, Wolkowicz H (eds) Fixed-point algorithms for inverse problems in science and engineering. Springer-Verlag, pp 185–212Google Scholar
  13. Combettes PL, Wajs VR (2005) Signal recovery by proximal forward-backward splitting. Multiscale Model Simul 4(4):1168–1200MathSciNetCrossRefzbMATHGoogle Scholar
  14. Daubechies I, Defrise M, De Mol C (2004) An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Commun Pure Appl Math 57(11):1413–1457CrossRefzbMATHGoogle Scholar
  15. Daubechies I, Teschke G, Vese L (2007) Iteratively solving linear inverse problems under general convex constraints. Inverse Probl Imaging 1:29–46MathSciNetCrossRefzbMATHGoogle Scholar
  16. Daubechies I, Fornasier M, Loris I (2008) Accelerated projected gradient method for linear inverse problems with sparsity constraints. J Fourier Anal Appl 14(5–6):764–792MathSciNetCrossRefzbMATHGoogle Scholar
  17. Defrise M, Vanhove C, Liu X (2011) An algorithm for total variation regularization in high-dimensional linear problems. Inverse Probl 27(6):065002MathSciNetCrossRefGoogle Scholar
  18. Engl HW, Hanke M, Neubauer A (2000) Regularization of inverse problems. Kluwer Academic Publishers, Dordrecht (The Netherlands)Google Scholar
  19. Esser JE (2010) Primal dual algorithms for convex models and applications to image restoration, registration and nonlocal inpainting. PhD thesis, University of CaliforniaGoogle Scholar
  20. Esser E, Zhang X, Chan TF (2010) A general framework for a class of first order primal-dual algorithms for convex optimization in imaging science. SIAM J Imaging Sci 3(4):1015–1046MathSciNetCrossRefzbMATHGoogle Scholar
  21. Figueiredo MAT, Nowak RD, Wright SJ (2007) Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems. IEEE J Sel Top Signal Process (Special Issue Convex Optim Methods Signal Process) 1:586–598CrossRefGoogle Scholar
  22. Fornasier M, Rauhut H (2008) Recovery algorithms for vector-valued data with joint sparsity constraints. SIAM J Numer Anal 46:577–613MathSciNetCrossRefzbMATHGoogle Scholar
  23. Gholami A, Siahkoohi HR (2010) Regularization of linear and non-linear geophysical ill-posed problems with joint sparsity constraints. Geophys J Int 180(2):871–882CrossRefGoogle Scholar
  24. Goldstein AA (1964) Convex programming in Hilbert space. Bull Am Math Soc 70:709–710CrossRefzbMATHGoogle Scholar
  25. Grasmair M, Haltmeier M, Scherzer O (2011) The residual method for regularizing ill-posed problems. Appl Math Comput 218(6):2693–2710MathSciNetCrossRefzbMATHGoogle Scholar
  26. Hale E, Yin W, Zhang Y (2008) Fixed-point continuation for 1-minimization: Methodology and convergence. SIAM J Optim 19(3):1107–1130MathSciNetCrossRefzbMATHGoogle Scholar
  27. Hansen C (1992) Analysis of discrete ill-posed problems by means of the L-curve. SIAM Rev 34(4):561–580MathSciNetCrossRefzbMATHGoogle Scholar
  28. Hennenfent G, van den Berg E, Friedlander MP, Herrmann FJ (2008) New insights into one-norm solvers from the Pareto curve. Geophysics 73(4):A23–A26CrossRefGoogle Scholar
  29. Herrmann FJ, Hennenfent G (2008) Non-parametric seismic data recovery with curvelet frames. Geophys J Int 173(1):233–248CrossRefGoogle Scholar
  30. Herrmann FJ, Moghaddam PP, Stolk C (2008) Sparsity- and continuity-promoting seismic image recovery with curvelet frames. Appl Comput Harmonic Anal 24(2):150–173MathSciNetCrossRefzbMATHGoogle Scholar
  31. Ivanov VV (1976) The theory of approximate methods and their application to the numerical solution of singular integral equations. Nordhoff International, LeydenzbMATHGoogle Scholar
  32. Kim S-J, Koh K, Lustig M, Boyd S, Gorinevsky D (2007) An interior-point method for large-scale 1-regularized least squares. IEEE J Sel Top Signal Process 1(4):606–617CrossRefGoogle Scholar
  33. Levitin ES, Polyak BT (1966) Constrained minimization problems. USSR Comput Math Math Phys 6:1–50CrossRefGoogle Scholar
  34. Loris I, Verhoeven C (2011) On a generalization of the iterative soft-thresholding algorithm for the case of non-separable penalty. Inverse Probl 27(12):125007MathSciNetCrossRefGoogle Scholar
  35. Loris I, Verhoeven C (2012) Iterative algorithms for total variation-like reconstructions in seismic tomography. Int J Geomath 3(2):179–208MathSciNetCrossRefzbMATHGoogle Scholar
  36. Loris I, Nolet G, Daubechies I, Dahlen FA (2007) Tomographic inversion using 1-norm regularization of wavelet coefficients. Geophys J Int 170(1):359–370CrossRefGoogle Scholar
  37. Loris I, Bertero M, De Mol C, Zanella R, Zanni L (2009) Accelerating gradient projection methods for 1-constrained signal recovery by steplength selection rules. Appl Comput Harmon Anal 27(2):247–254MathSciNetCrossRefzbMATHGoogle Scholar
  38. Loris I, Douma H, Nolet G, Daubechies I, Regone C (2010) Nonlinear regularization techniques for seismic tomography. J Comput Phys 229(3):890–905MathSciNetCrossRefzbMATHGoogle Scholar
  39. Mi T, Li S, Liu Y (2012) Fast thresholding algorithms with feedbacks for sparse signal recovery. ArXiv: abs/1204.3700Google Scholar
  40. Moreau JJ (1965) Proximité et dualité dans un espace hilbertien. Bull Soc Math France 93:273–299MathSciNetzbMATHGoogle Scholar
  41. Morozov VA (1966) On the solution of functional equations by the method of regularization. Soviet Math Dokl 7:414–417MathSciNetzbMATHGoogle Scholar
  42. Nikazad T, Davidi R, Herman GT (2012) Accelerated perturbation-resilient block-iterative projection methods with application to image reconstruction. Inverse Probl 28(3):035005 (19pp)Google Scholar
  43. Nolet G (2008) A breviary of seismic tomography. Cambridge University Press, CambridgeCrossRefzbMATHGoogle Scholar
  44. Polyak BT (1987) Introduction to optimization. Optimization Software, New YorkGoogle Scholar
  45. Rezghi M, Hosseini SM (2009) A new variant of L-curve for Tikhonov regularization. J Comput Appl Math 231(2):914–924MathSciNetCrossRefzbMATHGoogle Scholar
  46. Rockafellar RT (1970) Convex analysis. Princeton University Press, PrincetonzbMATHGoogle Scholar
  47. Rudin LI, Osher S, Fatemi E (1992) Nonlinear total variation based noise removal algorithms. Phys D Nonlin Phenom 60(1–4):259–268CrossRefzbMATHGoogle Scholar
  48. Santosa F, Symes WW (1986) Linear inversion of band-limited reflection seismograms. SIAM J Sci Stat Comput 7:1307–1330MathSciNetCrossRefzbMATHGoogle Scholar
  49. Strong D, Chan T (2003) Edge-preserving and scale-dependent properties of total variation regularization. Inverse Probl 19(6):S165MathSciNetCrossRefzbMATHGoogle Scholar
  50. Taylor HL, Banks SC, McCoy JF (1979) Deconvolution with the 1 norm. Geophysics 44:39–52CrossRefGoogle Scholar
  51. Tikhonov AN (1963) Solution of incorrectly formulated problems and the regularization method. Soviet Math Dokl 4:1035–1038Google Scholar
  52. Trampert J, Woodhouse JH (1995) Global phase-velocity maps of Love and Rayleigh-waves between 40 and 150 seconds. Geophys J Int 122(2):675–690CrossRefGoogle Scholar
  53. Trampert J, Woodhouse JH (1996) High resolution global phase velocity distributions. Geophys Res Lett 23(1):21–24CrossRefGoogle Scholar
  54. Trampert J, Woodhouse JH (2001) Assessment of global phase velocity models. Geophys J Int 144(1):165–174. doi: 10.1046/j.1365-246x.2001.00307.xCrossRefGoogle Scholar
  55. van den Berg E, Friedlander MP (2007) SPGL1: a solver for large-scale sparse reconstruction
  56. van den Berg E, Friedlander MP (2008) Probing the Pareto frontier for basis pursuit solutions. SIAM J Sci Comput 31(2):890–912MathSciNetCrossRefzbMATHGoogle Scholar
  57. van den Berg E, Friedlander MP (2011) Sparse optimization with least-squares constraints. SIAM J Optim 21(4):1201–1229MathSciNetCrossRefzbMATHGoogle Scholar
  58. Zhang X, Burger M, Osher S (2011) A unified primal-dual algorithm framework based on Bregman iteration. J Sci Comput 46:20–46MathSciNetCrossRefzbMATHGoogle Scholar
  59. Zhu M, Chan T (2008) An efficient primal-dual hybrid gradient algorithm for total variation image restoration. Technical report, UCLAGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  1. 1.Université libre de BruxellesBruxellesBelgium

Personalised recommendations