Skip to main content
SpringerLink
Log in

A New Steplength Selection for Scaled Gradient Methods with Application to Image Deblurring

  • Published:
Journal of Scientific Computing Aims and scope Submit manuscript

Abstract

Gradient methods are frequently used in large scale image deblurring problems since they avoid the onerous computation of the Hessian matrix of the objective function. Second order information is typically sought by a clever choice of the steplength parameter defining the descent direction, as in the case of the well-known Barzilai and Borwein rules. In a recent paper, a strategy for the steplength selection approximating the inverse of some eigenvalues of the Hessian matrix has been proposed for gradient methods applied to unconstrained minimization problems. In the quadratic case, this approach is based on a Lanczos process applied every \(m\) iterations to the matrix of the gradients computed in the previous \(m\) iterations, but the idea can be extended to a general objective function. In this paper we extend this rule to the case of scaled gradient projection methods applied to constrained minimization problems, and we test the effectiveness of the proposed strategy in image deblurring problems in both the presence and the absence of an explicit edge-preserving regularization term.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Acar, R., Vogel, C.R.: Analysis of bounded variation penalty methods for ill-posed problems. Inverse Probl. 10(6), 1217–1229 (2004)

    Article  MathSciNet  Google Scholar 

  2. Bardsley, J.M., Goldes, J.: Regularization parameter selection methods for ill-posed Poisson maximum likelihood estimation. Inverse Probl. 25(9), 095005 (2009)

    Article  MathSciNet  Google Scholar 

  3. Barzilai, J., Borwein, J.M.: Two-point step size gradient methods. IMA J. Numer. Anal. 8(1), 141–148 (1988)

    Article  MATH  MathSciNet  Google Scholar 

  4. Bertero, M., Boccacci, P., Talenti, G., Zanella, R., Zanni, L.: A discrepancy principle for Poisson data. Inverse Probl. 26(10), 105004 (2010)

    Article  MathSciNet  Google Scholar 

  5. Bertero, M., Lantéri, H., Zanni, L.: Iterative image reconstruction: a point of view. In: Censor, Y., Jiang, M., Louis, A.K. (eds.) Mathematical Methods in Biomedical Imaging and Intensity-Modulated Radiation Therapy, pp. 37–63. Edizioni della Normale, Pisa (2008)

    Google Scholar 

  6. Bertsekas, D.: Nonlinear Programming. Athena Scientific, Belmont (1999)

    MATH  Google Scholar 

  7. Bertsekas, D.: Convex Optimization Theory. Supplementary Chapter 6 on Convex Optimization Algorithms. Athena Scientific, Belmont (2009)

    Google Scholar 

  8. Birgin, E.G., Martinez, J.M., Raydan, M.: Inexact spectral projected gradient methods on convex sets. IMA J. Numer. Anal. 23(4), 539–559 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  9. Bonettini, S., Landi, G., Loli Piccolomini, E., Zanni, L.: Scaling techniques for gradient projection-type methods in astronomical image deblurring. Int. J. Comput. Math. 90(1), 9–29 (2013)

    Article  MATH  MathSciNet  Google Scholar 

  10. Bonettini, S., Prato, M.: Nonnegative image reconstruction from sparse Fourier data: a new deconvolution algorithm. Inverse Probl. 26(9), 095001 (2010)

    Article  MathSciNet  Google Scholar 

  11. Bonettini, S., Prato, M.: Accelerated gradient methods for the X-ray imaging of solar flares. Inverse Probl. 30(5), 055004 (2014)

    Article  MathSciNet  Google Scholar 

  12. Bonettini, S., Prato, M.: A new general framework for gradient projection methods (2014). arXiv:1406.6601

  13. Bonettini, S., Ruggiero, V.: An alternating extragradient method for total variation based image restoration from Poisson data. Inverse Probl. 27(9), 095001 (2011)

    Article  MathSciNet  Google Scholar 

  14. Bonettini, S., Ruggiero, V.: On the convergence of primal–dual hybrid gradient algorithms for total variation image restoration. J. Math. Imaging Vis. 44(3), 236–253 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  15. Bonettini, S., Zanella, R., Zanni, L.: A scaled gradient projection method for constrained image deblurring. Inverse Probl. 25(1), 015002 (2009)

    Article  MathSciNet  Google Scholar 

  16. Carlavan, M., Blanc-Féraud, L.: Regularizing parameter estimation for Poisson noisy image restoration. In: International ICST Workshop on New Computational Methods for Inverse Problems, May 2011, Paris, France

  17. Chambolle, A.: An algorithm for total variation minimization and applications. J. Math. Imaging Vis. 20(1–2), 89–97 (2004)

    MathSciNet  Google Scholar 

  18. Chambolle, A., Pock, T.: A first-order primal–dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40(1), 120–145 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  19. Coleman, T.F., Li, Y.: An interior trust region approach for nonlinear minimization subject to bounds. SIAM J. Optim. 6(2), 418–445 (1996)

    Article  MATH  MathSciNet  Google Scholar 

  20. Cornelio, A., Porta, F., Prato, M., Zanni, L.: On the filtering effect of iterative regularization algorithms for discrete inverse problems. Inverse Probl. 29(12), 125013 (2013)

    Article  MathSciNet  Google Scholar 

  21. Dai, Y.H., Yuan, Y.X.: Alternate minimization gradient method. IMA J. Numer. Anal. 23(3), 377–393 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  22. Daube-Witherspoon, M.E., Muehllener, G.: An iterative image space reconstruction algorithm suitable for volume ECT. IEEE Trans. Med. Imaging 5(2), 61–66 (1986)

    Article  Google Scholar 

  23. De Asmundis, R., Di Serafino, D., Riccio, F., Toraldo, G.: On spectral properties of steepest descent methods. IMA J. Numer. Anal. 33(4), 1416–1435 (2013)

    Article  MATH  MathSciNet  Google Scholar 

  24. De Asmundis, R., Di Serafino, D., Hager, W.W., Toraldo, G., Zhang, H.: An efficient gradient method using the Yuan steplength. Comput. Optim. Appl. 59(3), 541–563 (2014)

    Article  MATH  MathSciNet  Google Scholar 

  25. Fletcher, R.: A limited memory steepest descent method. Math. Program. 135(1–2), 413–436 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  26. Frassoldati, G., Zanghirati, G., Zanni, L.: New adaptive stepsize selections in gradient methods. J. Ind. Manage. Optim. 4(2), 299–312 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  27. Golub, G.H., Van Loan, C.F.: Matrix Computations, 3rd edn. John Hopkins University Press, Baltimore (1996)

    MATH  Google Scholar 

  28. Grippo, L., Lampariello, F., Lucidi, S.: A nonmonotone line search technique for Newton’s method. SIAM J. Numer. Anal. 23(4), 707–716 (1986)

    Article  MATH  MathSciNet  Google Scholar 

  29. Hansen, P.C.: Rank-Deficient and Discrete Ill-Posed Problems. SIAM, Philadelphia (1997)

    MATH  Google Scholar 

  30. Hansen, P.C., Nagy, J.G., O’Leary, D.P.: Deblurring Images: Matrices, Spectra and Filtering. SIAM, Philadelphia (2006)

    Book  Google Scholar 

  31. Harmany, Z.T., Marcia, R.F., Willett, R.M.: This is spiral-tap: sparse Poisson intensity reconstruction algorithms–theory and practice. IEEE Trans. Image Process. 3(21), 1084–1096 (2012)

    Article  MathSciNet  Google Scholar 

  32. Lantéri, H., Roche, M., Aime, C.: Penalized maximum likelihood image restoration with positivity constraints: multiplicative algorithms. Inverse Probl. 18(5), 1397–1419 (2002)

    Article  MATH  Google Scholar 

  33. Lantéri, H., Roche, M., Cuevas, O., Aime, C.: A general method to devise maximum likelihood signal restoration multiplicative algorithms with non-negativity constraints. Signal Process. 81(5), 945–974 (2001)

    Article  MATH  Google Scholar 

  34. Lucy, L.: An iterative technique for the rectification of observed distributions. Astron. J. 79(6), 745–754 (1974)

    Article  Google Scholar 

  35. Nocedal, J., Wright, S.J.: Numerical Optimization, 2nd edn. Springer, New York (2006)

    MATH  Google Scholar 

  36. Porta, F., Zanella, R., Zanghirati, G., Zanni, L.: Limited-memory scaled gradient projection methods for real-time image deconvolution in microscopy. Commun. Nonlinear Sci. Numer. Simul. 21, 112–127 (2015)

    Article  MathSciNet  Google Scholar 

  37. Prato, M., Cavicchioli, R., Zanni, L., Boccacci, P., Bertero, M.: Efficient deconvolution methods for astronomical imaging: algorithms and IDL-GPU codes. Astron. Astrophys. 539, A133 (2012)

    Article  Google Scholar 

  38. Prato, M., La Camera, A., Bonettini, S., Bertero, M.: A convergent blind deconvolution method for post-adaptive-optics astronomical imaging. Inverse Probl. 29(6), 065017 (2013)

    Article  Google Scholar 

  39. Richardson, W.H.: Bayesian based iterative method of image restoration. J. Opt. Soc. Am. 62(1), 55–59 (1972)

    Article  Google Scholar 

  40. Rudin, L., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. D 60(1–4), 259–268 (1992)

    Article  MATH  Google Scholar 

  41. Ruggiero, V., Zanni, L.: A modified projection algorithm for large strictly-convex quadratic programs. J. Optim. Theory Appl. 104(2), 281–299 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  42. Setzer, S., Steidl, G., Teuber, T.: Deblurring Poissonian images by split Bregman techniques. J. Vis. Commun. Image Represent. 21(3), 193–199 (2010)

    Article  MathSciNet  Google Scholar 

  43. Vogel, C.R.: Computational Methods for Inverse Problems. SIAM, Philadelphia (2002)

    Book  MATH  Google Scholar 

  44. Yuan, Y.: A new stepsize for the steepest descent method. J. Comput. Math. 24, 149–156 (2006)

    MATH  MathSciNet  Google Scholar 

  45. Zanella, R., Boccacci, P., Zanni, L., Bertero, M.: Efficient gradient projection methods for edge-preserving removal of Poisson noise. Inverse Probl. 25(4), 045010 (2009)

    Article  MathSciNet  Google Scholar 

  46. Zanella, R., Zanghirati, G., Cavicchioli, R., Zanni, L., Boccacci, P., Bertero, M., Vicidomini, G.: Towards real-time image deconvolution: application to confocal and sted microscopy. Sci. Rep. 3, 2523 (2013)

    Article  Google Scholar 

  47. Zhou, B., Gao, L., Dai, Y.H.: Gradient methods with adaptive step-sizes. Comput. Optim. Appl. 35(1), 69–86 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  48. Zhu, M., Wright, S.J., Chan, T.F.: Duality-based algorithms for total-variation-regularized image restoration. Comput. Optim. Appl. 47(3), 377–400 (2008)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgments

This work has been partially supported by the Italian Spinner 2013 Ph.D. Project “High-complexity inverse problems in biomedical applications and social systems” and by MIUR (Italian Ministry for University and Research), under the projects FIRB—Futuro in Ricerca 2012, contract RBFR12M3AC, and PRIN 2012, contract 2012MTE38N. The Italian GNCS—INdAM (Gruppo Nazionale per il Calcolo Scientifico—Istituto Nazionale di Alta Matematica) is also acknowledged.

Author information

Authors and Affiliations

  1. Dipartimento di Scienze Fisiche, Informatiche e Matematiche, Università degli Studi di Modena e Reggio Emilia, Via Campi 213/b, 41125, Modena, Italy

    Federica Porta, Marco Prato & Luca Zanni

Authors
  1. Federica Porta
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marco Prato
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Luca Zanni
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marco Prato.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Porta, F., Prato, M. & Zanni, L. A New Steplength Selection for Scaled Gradient Methods with Application to Image Deblurring. J Sci Comput 65, 895–919 (2015). https://doi.org/10.1007/s10915-015-9991-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10915-015-9991-9

Keywords

Mathematics Subject Classification

Search

Navigation