Skip to main content
SpringerLink
Log in

Optimization methods for regularization-based ill-posed problems: a survey and a multi-objective framework

  • Review Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

Ill-posed problems are widely existed in signal processing. In this paper, we review popular regularization models such as truncated singular value decomposition regularization, iterative regularization, variational regularization. Meanwhile, we also retrospect popular optimization approaches and regularization parameter choice methods. In fact, the regularization problem is inherently a multi-objective problem. The traditional methods usually combine the fidelity term and the regularization term into a single-objective with regularization parameters, which are difficult to tune. Therefore, we propose a multi-objective framework for ill-posed problems, which can handle complex features of problem such as non-convexity, discontinuity. In this framework, the fidelity term and regularization term are optimized simultaneously to gain more insights into the ill-posed problems. A case study on signal recovery shows the effectiveness of the multi-objective framework for ill-posed problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Hadamard J. Sur les Problemes aux Derivees Partielles et Leur Signification Physique. Princeton University Bulletin, 1902, 13: 49–52

    MathSciNet  Google Scholar 

  2. Kabanikhin S I. Inverse and Ill-Posed Problems: Theory and Applications. Berlin: Water De Gruyter, 2011

    Book  Google Scholar 

  3. Zhang B Y, Xu D H, Liu T W. Stabilized algorithms for ill-posed problems in signal processing. In: Proceedings of the IEEE International Conferences on Info-tech and Info-net. 2001, 1: 375–380

    Article  Google Scholar 

  4. Scherzer O. Handbook of Mathematical Methods in Imaging. Springer Science & Business Media, 2011

    Book  MATH  Google Scholar 

  5. Groetsch C W. Inverse problems in the mathematical sciences. Mathematics of Computation, 1993, 63(5): 799–811

    MATH  Google Scholar 

  6. Rudin L I, Osher S, Fatemi E. Nonlinear total variation based noise removal algorithms. Physica D: Nonlinear Phenomena, 1992, 60(1): 259–268

    Article  MathSciNet  MATH  Google Scholar 

  7. Tikhonov A N. Solution of incorrectly formulated problems and the regularization method. Soviet Math, 1963, 4: 1035–1038

    MATH  Google Scholar 

  8. Tikhonov A N, Arsenin V I. Solutions of Ill-posed Problems. Washington, DC: V. H. Winston & Sons, 1977

    MATH  Google Scholar 

  9. Landweber L. An iteration formula for Fredholm integral equations of the first kind. American Journal of Mathematics, 1951, 73(3): 615–624

    Article  MathSciNet  MATH  Google Scholar 

  10. Hestenes M R, Stiefel E. Methods of conjugate gradients for solving linear systems. Journal of Research of the National Bureau of Standards, 1952, 49(6): 409–436

    Article  MathSciNet  MATH  Google Scholar 

  11. Vogel C R. Computational Methods for Inverse Problems. Society for Industrial and Applied Mathematics, 2002, 23

    Google Scholar 

  12. Hansen P C. The truncated SVD as a method for regularization. Bit Numerical Mathematics, 1987, 27(4): 534–553

    Article  MathSciNet  MATH  Google Scholar 

  13. Honerkamp J, Weese J. Tikhonovs regularization method for ill-posed problems. Continuum Mechanics and Thermodynamics, 1990, 2(1): 17–30

    Article  MathSciNet  MATH  Google Scholar 

  14. Zhang X Q, Burger M, Bresson X, Osher S. Bregmanized nonlocal regularization for deconvolution and sparse reconstruction. SIAM Journal on Imaging Sciences, 2010, 3(3): 253–276

    Article  MathSciNet  MATH  Google Scholar 

  15. Deb K. Multi-Objective Optimization Using Evolutionary Algorithms. New York: John Wiley & Sons, 2001, 16

    MATH  Google Scholar 

  16. Fonseca CM, Fleming P J. An overview of evolutionary algorithms in multiobjective optimization. Evolutionary Computation, 1995, 3(1): 1–16

    Article  Google Scholar 

  17. Coello C A C, Van Veldhuizen D A, Lamont G B. Evolutionary Algorithms for Solving Multi-objective Problems. New York: Kluwer Academic, 2002

    Book  MATH  Google Scholar 

  18. Tan K C, Khor E F, Lee T H. Multiobjective Evolutionary Algorithms and Applications. Springer Science & Business Media, 2005

    Google Scholar 

  19. Knowles J, Corne D, Deb K. Multiobjective Problem Solving from Nature: from Concepts to Applications. Springer Science & Business Media, 2008

    Book  MATH  Google Scholar 

  20. Raquel C, Yao X. Dynamic multi-objective optimization: a survey of the state-of-the-art. In: Yang S X, Yao X, eds. Evolutionary Computation for Dynamic Optimization Problems. Springer Berlin Heidelberg, 2013, 85–106

    Chapter  Google Scholar 

  21. Lücken C V, Barán B, Brizuela C. A survey on multi-objective evolutionary algorithms for many-objective problems. Computational Optimization and Applications, 2014, 58(3): 707–756

    MathSciNet  MATH  Google Scholar 

  22. Hwang C L, Masud A S M. Multiple Objective Decision Making- Methods and Applications. Springer Science & Business Media, 1979, 164

    Book  Google Scholar 

  23. Girosi F, Jones M B, Poggio T. Regularization theory and neural networks architectures. Neural Computation, 1995, 7(2): 219–269

    Article  Google Scholar 

  24. Belge M, Kilmer M E, Miller E L. Efficient determination of multiple regularization parameters in a generalized L-curve framework. Inverse Problems, 2002, 18(4): 1161

    Article  MathSciNet  MATH  Google Scholar 

  25. Hansen P C. Rank-deficient and discrete ill-posed problems: numerical aspects of linear inversion. American Mathematical Monthly, 1997, 4(5): 491

    Google Scholar 

  26. Eriksson P, Jiménez C, Buehler S A. Qpack, a general tool for instrument simulation and retrieval work. Journal of Quantitative Spectroscopy and Radiative Transfer, 2005, 91(1): 47–64

    Article  Google Scholar 

  27. Giusti E. Minimal Surfaces and Functions of Bounded Variation. Springer Science & Business Media, 1984, 80

    Book  MATH  Google Scholar 

  28. Catté F, Coll T. Image selective smoothing and edge detection by nonlinear diffusion. SIAM Journal on Numerical Analysis, 1992, 29(1): 182–193

    Article  MathSciNet  MATH  Google Scholar 

  29. Björck A. Numerical Methods for Least Squares Problems. Society for Industrial and Applied Mathematics, 1996

    Google Scholar 

  30. Groetsch C W. The Theory of Tikhonov Regularization for Fredholm Equations of the First Kind. Pitman Advanced Publishing Program, 1984

    MATH  Google Scholar 

  31. Hanson R J. A numerical method for solving Fredholm integral equations of the first kind using singular values. SIAM Journal on Numerical Analysis, 1971, 8(3): 616–622

    Article  MathSciNet  MATH  Google Scholar 

  32. Stewart G W. Rank degeneracy. SIAM Journal on Scientific and Statistical Computing, 1984, 5(2): 403–413

    Article  MathSciNet  MATH  Google Scholar 

  33. Hansen P C, Sekii T, Shibahashi H. The modified truncated SVD method for regularization in general form. SIAM Journal on Scientific and Statistical Computing, 1992, 13(5): 1142–1150

    Article  MathSciNet  MATH  Google Scholar 

  34. Van Loan C F. Generalizing the singular value decomposition. SIAM Journal on Numerical Analysis, 1976, 13(1): 76–83

    Article  MathSciNet  MATH  Google Scholar 

  35. Hansen P C. Regularization, GSVD and truncated GSVD. BIT Numerical Mathematics, 1989, 29(3): 491–504

    Article  MathSciNet  MATH  Google Scholar 

  36. Paige C C. Computing the generalized singular value decomposition. SIAM Journal on Scientific and Statistical Computing, 1986, 7(4): 1126–1146

    Article  MathSciNet  MATH  Google Scholar 

  37. Morigi S, Reichel L, Sgallari F. A truncated projected SVD method for linear discrete ill-posed problems. Numerical Algorithms, 2006, 43(3): 197–213

    Article  MathSciNet  MATH  Google Scholar 

  38. Fernando K V, Hammarling S. A product induced singular value decomposition (ΠSVD) for two matrices and balanced realization. In: Proceedings of SIAM Conference on Linear Algebra in Signals, Systems and Control. 1988, 128–140

    Google Scholar 

  39. Zha H Y. The restricted singular value decomposition of matrix triplets. SIAM Journal on Matrix Analysis and Applications, 1991, 12(1): 172–194

    Article  MathSciNet  MATH  Google Scholar 

  40. De Moor B, Golub G H. The restricted singular value decomposition: properties and applications. SIAM Journal on Matrix Analysis and Applications, 1991, 12(3): 401–425

    Article  MathSciNet  MATH  Google Scholar 

  41. De Moor B, Zha H Y. A tree of generalizations of the ordinary singular value decomposition. Linear Algebra and Its Applications, 1991, 147: 469–500

    Article  MathSciNet  MATH  Google Scholar 

  42. De Moor B. Generalizations of the OSVD: structure, properties and applications. In: Vaccaro R J, ed. SVD & Signal Processing, II: Algorithms, Analysis & Applications. 1991, 83–98

    Google Scholar 

  43. Noschese S, Reichel L. A modified TSVD method for discrete ill posed problems. Numerical Linear Algebra with Applications, (in press)

  44. Dykes L, Noschese S, Reichel L. Rescaling the GSVD with application to ill-posed problems. Numerical Algorithms, 2015, 68(3): 531–545

    Article  MathSciNet  MATH  Google Scholar 

  45. Edo L, Franco W, Martinsson P G, Rokhlin V, Tygert M. Randomized algorithms for the low-rank approximation of matrices. Proceedings of the National Academy of Sciences, 2007, 104(51): 20167–20172

    Article  MathSciNet  MATH  Google Scholar 

  46. Woolfe F, Liberty E, Rokhlin V, Tygert M. A fast randomized algorithm for the approximation of matrices. Applied & Computational Harmonic Analysis, 2008, 25(3): 335–366

    Article  MathSciNet  MATH  Google Scholar 

  47. Sifuentes J, Gimbutas Z, Greengard L. Randomized methods for rankdeficient linear systems. Electronic Transactions on Numerical Analysis, 2015, 44: 177–188

    MathSciNet  MATH  Google Scholar 

  48. Liu Y G, Lei Y J, Li C G, Xu W Z, Pu Y F. A random algorithm for low-rank decomposition of large-scale matrices with missing entries. IEEE Transactions on Image Processing, 2015, 24(11): 4502–4511

    Article  MathSciNet  Google Scholar 

  49. Sekii T. Two-dimensional inversion for solar internal rotation. Publications of the Astronomical Society of Japan, 1991, 43: 381–411

    Google Scholar 

  50. Scales J A. Uncertainties in seismic inverse calculations. In: Jacobsen B H, Mosegaard K, Sibani P, eds. Inverse Methods. Berlin: Springer-Verlag, 1996, 79–97

    Chapter  Google Scholar 

  51. Lawless J F, Wang P. A simulation study of ridge and other regression estimators. Communications in Statistics-Theory and Methods, 1976, 5(4): 307–323

    Article  MATH  Google Scholar 

  52. Dempster A P, Schatzoff M, Wermuth N. A simulation study of alternatives to ordinary least squares. Journal of the American Statistical Association, 1977, 72(357): 77–91

    Article  MATH  Google Scholar 

  53. Hansen P C, O’Leary D P. The use of the L-curve in the regularization of discrete ill-posed problems. SIAM Journal on Scientific Computing, 1993, 14(6): 1487–1503

    Article  MathSciNet  MATH  Google Scholar 

  54. Hansen P C. Analysis of discrete ill-posed problems by means of the L-curve. SIAM Review, 1992, 34(4): 561–580

    Article  MathSciNet  MATH  Google Scholar 

  55. Xu P L. Truncated SVD methods for discrete linear ill-posed problems. Geophysical Journal International, 1998, 135(2): 505–514

    Article  Google Scholar 

  56. Wu Z M, Bian S F, Xiang C B, Tong Y D. A new method for TSVD regularization truncated parameter selection. Mathematical Problems in Engineering, 2013

    Google Scholar 

  57. Chicco D, Masseroli M. A discrete optimization approach for SVD best truncation choice based on ROC curves. In: Proceedings of the 13th IEEE International Conference on Bioinformatics and Bioengineering. 2013: 1–4

    Chapter  Google Scholar 

  58. Golub G H, Heath M, Wahba G. Generalized cross-validation as a method for choosing a good ridge parameter. Technometrics, 1979, 21(2): 215–223

    Article  MathSciNet  MATH  Google Scholar 

  59. Jbilou K, Reichel L, Sadok H. Vector extrapolation enhanced TSVD for linear discrete ill-posed problems. Numerical Algorithms, 2009, 51(2): 195–208

    Article  MathSciNet  MATH  Google Scholar 

  60. Bouhamidi A, Jbilou K, Reichel L, Sadok H, Wang Z. Vector extrapolation applied to truncated singular value decomposition and truncated iteration. Journal of Engineering Mathematics, 2015, 93(1): 99–112

    Article  MathSciNet  MATH  Google Scholar 

  61. Vogel C R. Computational Methods for Inverse Problems. Society for Industrial and Applied Mathematics, 2002

    Google Scholar 

  62. Doicu A, Trautmann T, Schreier F. Numerical Regularization for Atmospheric Inverse Problems. Springer Science & Business Media, 2010

    Book  MATH  Google Scholar 

  63. Bakushinsky A B, Goncharsky A V. Iterative Methods for the Solution of Incorrect Problems. Moscow: Nauka, 1989

    Google Scholar 

  64. Rieder A. Keine Probleme mit Inversen Problemen: Eine Einführung in ihre stabile Lösung. Berlin: Springer-Verlag, 2013

    Google Scholar 

  65. Nemirovskiy A S, Polyak B T. Iterative methods for solving linear ill posed problems under precise information. Engineering Cybernetics, 1984, 22(4): 50–56

    MathSciNet  MATH  Google Scholar 

  66. Brakhage H. On ill-posed problems and the method of conjugate gradients. Inverse and Ill-posed Problems, 1987, 4: 165–175

    Article  MathSciNet  MATH  Google Scholar 

  67. Hanke M. Accelerated Landweber iterations for the solution of ill posed equations. Numerische Mathematik, 1991, 60(1): 341–373

    Article  MathSciNet  MATH  Google Scholar 

  68. Barzilai J, Borwein J M. Two-point step size gradient methods. IMA Journal of Numerical Analysis, 1988, 8(1): 141–148

    Article  MathSciNet  MATH  Google Scholar 

  69. Axelsson O. Iterative Solution Methods. Cambridge: Cambridge University Press, 1996

    MATH  Google Scholar 

  70. Van der Sluis A, Van der Vorst H A. The rate of convergence of conjugate gradients. Numerische Mathematik, 1986, 48(5): 543–560

    Article  MathSciNet  MATH  Google Scholar 

  71. Scales J A, Gersztenkorn A. Robust methods in inverse theory. Inverse Problems, 1988, 4(4): 1071–1091

    Article  MathSciNet  MATH  Google Scholar 

  72. Björck Å, Eldén L. Methods in numerical algebra for ill-posed problems. Technical Report LiTH-MAT-R-33-1979. 1979

    Google Scholar 

  73. Trefethen L N, Bau D. Numerical Linear Algebra. Society for Industrial and Applied Mathematics, 1997

    Google Scholar 

  74. Calvetti D, Lewis B, Reichel L. On the regularizing properties of the GMRES method. Numerische Mathematik, 2002, 91(4): 605–625

    Article  MathSciNet  MATH  Google Scholar 

  75. Calvetti D, Lewis B, Reichel L. Alternating Krylov subspace image restoration methods. Journal of Computational and Applied Mathematics, 2012, 236(8): 2049–2062

    Article  MathSciNet  MATH  Google Scholar 

  76. Brianzi P, Favati P, Menchi O, Romani F. A framework for studying the regularizing properties of Krylov subspace methods. Inverse Problems, 2006, 22(3): 1007–1021

    Article  MathSciNet  MATH  Google Scholar 

  77. Sonneveld P, Van Gijzen M B. IDR(s): a family of simple and fast algorithms for solving large nonsymmetric systems of linear equations. SIAM Journal on Scientific Computing, 2008, 31(2): 1035–1062

    Article  MathSciNet  MATH  Google Scholar 

  78. Fong D C L, Saunders M. LSMR: an iterative algorithm for sparse least-squares problems. SIAM Journal on Scientific Computing, 2011, 33(5): 2950–2971

    Article  MathSciNet  MATH  Google Scholar 

  79. Zhao C, Huang T Z, Zhao X L, Deng L J. Two new efficient iterative regularization methods for image restoration problems. Abstract & Applied Analysis, 2013

    Google Scholar 

  80. Perez A, Gonzalez R C. An iterative thresholding algorithm for image segmentation. IEEE Transactions on Pattern Analysis & Machine Intelligence, 1987, 9(6): 742–751

    Article  Google Scholar 

  81. Beck A, Teboulle M. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM Journal on Imaging Sciences, 2009, 2(1): 183–202

    Article  MathSciNet  MATH  Google Scholar 

  82. Bioucas-Dias J M, Figueiredo M A T. Two-step algorithms for linear inverse problems with non-quadratic regularization. In: Proceedings of the IEEE International Conference on Image Processing. 2007, 105–108

    Google Scholar 

  83. Bioucas-Dias J M, Figueiredo M A T. A new TwIST: two-step iterative shrinkage/thresholding algorithms for image restoration. IEEE Transactions on Image Processing. 2007, 16(12): 2992–3004

    Article  MathSciNet  Google Scholar 

  84. Bayram I, Selesnick I W. A subband adaptive iterative shrinkage/ thresholding algorithm. IEEE Transactions on Signal Processing, 2010, 58(3): 1131–1143

    Article  MathSciNet  Google Scholar 

  85. Yamagishi M, Yamada I. Over-relaxation of the fast iterative shrinkage-thresholding algorithm with variable stepsize. Inverse Problems, 2011, 27(10): 105008–105022

    Article  MathSciNet  MATH  Google Scholar 

  86. Bhotto M Z A, Ahmad M O, Swamy M N S. An improved fast iterative shrinkage thresholding algorithm for image deblurring. SIAM Journal on Imaging Sciences, 2015, 8(3): 1640–1657

    Article  MathSciNet  MATH  Google Scholar 

  87. Zhang Y D, Dong Z C, Phillips P, Wang S H, Ji G L, Yang J Q. Exponential wavelet iterative shrinkage thresholding algorithm for compressed sensing magnetic resonance imaging. Information Sciences, 2015, 322: 115–132

    Article  MathSciNet  Google Scholar 

  88. Zhang Y D, Wang S H, Ji G L, Dong Z C. Exponential wavelet iterative shrinkage thresholding algorithm with random shift for compressed sensing magnetic resonance imaging. IEEJ Transactions on Electrical and Electronic Engineering, 2015, 10(1): 116–117

    Article  Google Scholar 

  89. Wu G M, Luo S Q. Adaptive fixed-point iterative shrinkage/ thresholding algorithm for MR imaging reconstruction using compressed sensing. Magnetic Resonance Imaging, 2014, 32(4): 372–378

    Article  Google Scholar 

  90. Fang E X, Wang J J, Hu D F, Zhang J Y, Zou W, Zhou Y. Adaptive monotone fast iterative shrinkage thresholding algorithm for fluorescence molecular tomography. IET Science Measurement Technology, 2015, 9(5): 587–595

    Article  Google Scholar 

  91. Zuo W M, Meng D Y, Zhang L, Feng X C, Zhang D. A generalized iterated shrinkage algorithm for non-convex sparse coding. In: Proceedings of the IEEE International Conference on Computer Vision. 2013, 217–224

    Google Scholar 

  92. Krishnan D, Fergus R. Fast image deconvolution using hyperlaplacian priors. In: Bengio Y, Schuurmans D, Lafferty J D, et al., eds. Advances in Neural Information Processing Systems 22. 2009, 1033–1041

    Google Scholar 

  93. Chartrand R, Yin W. Iteratively reweighted algorithms for compressive sensing. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing. 2008, 3869–3872

    Google Scholar 

  94. She Y Y. An iterative algorithm for fitting nonconvex penalized generalized linear models with grouped predictors. Computational Statistics & Data Analysis, 2012, 56(10): 2976–2990

    Article  MathSciNet  MATH  Google Scholar 

  95. Gong P H, Zhang C S, Lu Z S, Huang J Z, Ye J P. A general iterative shrinkage and thresholding algorithm for non-convex regularized optimization problems. In: Proceedings of International Conference on Machine Learning. 2013, 37–45

    Google Scholar 

  96. Bredies K, Lorenz D A. Linear convergence of iterative softthresholding. Journal of Fourier Analysis and Applications, 2008, 14(5–6): 813–837

    Article  MathSciNet  MATH  Google Scholar 

  97. Kowalski M. Thresholding rules and iterative shrinkage/thresholding algorithm: a convergence study. In: Proceedings of the IEEE International Conference on Image Processing. 2014, 4151–4155

    Google Scholar 

  98. Chambolle A, Dossal C. On the convergence of the iterates of the “fast iterative shrinkage/thresholding algorithm”. Journal of Optimization Theory & Applications, 2015, 166(3): 968–982

    Article  MathSciNet  MATH  Google Scholar 

  99. Johnstone P R, Moulin P. Local and global convergence of an inertial version of forward-backward splitting. Advances in Neural Information Processing Systems, 2014, 1970–1978

    Google Scholar 

  100. Morozov V A. On the solution of functional equations by the method of regularization. Soviet Mathematics Doklady, 1966, 7(11): 414–417

    MathSciNet  MATH  Google Scholar 

  101. Vainikko G M. The discrepancy principle for a class of regularization methods. USSR Computational Mathematics and Mathematical Physics, 1982, 22(3): 1–19

    Article  MATH  Google Scholar 

  102. Vainikko G M. The critical level of discrepancy in regularization methods. USSR Computational Mathematics and Mathematical Physics, 1983, 23(6): 1–9

    Article  Google Scholar 

  103. Plato R. On the discrepancy principle for iterative and parametric methods to solve linear ill-posed equations. Numerische Mathematik, 1996, 75(1): 99–120

    Article  MathSciNet  MATH  Google Scholar 

  104. Borges L S, Bazán F S V, Cunha M C C. Automatic stopping rule for iterative methods in discrete ill-posed problems. Computational & Applied Mathematics, 2015, 34(3): 1175–1197

    Article  MathSciNet  MATH  Google Scholar 

  105. Dziwoki G, Izydorczyk J. Stopping criteria analysis of the OMP algorithm for sparse channels estimation. In: Proceedings of the International Conference on Computer Networks. 2015, 250–259

    Google Scholar 

  106. Favati P, Lotti G, Menchi O, Romani F. Stopping rules for iterative methods in nonnegatively constrained deconvolution. Applied Numerical Mathematics, 2014, 75: 154–166

    Article  MathSciNet  MATH  Google Scholar 

  107. Engl H W, Hanke M, Neubauer A. Regularization of Inverse Problems. Springer Science & Business Media, 1996

    Book  MATH  Google Scholar 

  108. Amster P. Iterative Methods. Universitext, 2014, 53–82

    Google Scholar 

  109. Waseem M. On some iterative methods for solving system of nonlinear equations. Dissertation for the Doctoral Degree. Islamabad: COMSATS Institute of Information Technology, 2012

    Google Scholar 

  110. Burger M, Osher S. A guide to the TV zoo. In: Burger M, Mennucci A C G, Osher S, et al., eds. Level Set and PDE Based Reconstruction Methods in Imaging. Springer International Publishing, 2013, 1–70

    Chapter  Google Scholar 

  111. Tikhonov A N. Regularization of incorrectly posed problems. Soviet Mathematics Doklady, 1963, 4(1): 1624–1627

    MATH  Google Scholar 

  112. Nikolova M. Analysis of the recovery of edges in images and signals by minimizing nonconvex regularized least-squares. Multiscale Modeling & Simulation, 2005, 4(3): 960–991

    Article  MathSciNet  MATH  Google Scholar 

  113. Burger M, Osher S. Convergence rates of convex variational regularization. Inverse Problems, 2004, 20(5): 1411–1421

    Article  MathSciNet  MATH  Google Scholar 

  114. Hofmann B, Kaltenbacher B, Pöschl C, Scherzer O. A convergence rates result for Tikhonov regularization in Banach spaces with nonsmooth operators. Inverse Problems, 2007, 23(3): 987–1010

    Article  MathSciNet  MATH  Google Scholar 

  115. Resmerita E. Regularization of ill-posed problems in Banach spaces:convergence rates. Inverse Problems, 2005, 21(4): 1303–1314

    Article  MathSciNet  MATH  Google Scholar 

  116. Resmerita E, Scherzer O. Error estimates for non-quadratic regularization and the relation to enhancement. Inverse Problems, 2006, 22(3): 801–814

    Article  MathSciNet  MATH  Google Scholar 

  117. Engl H W. Discrepancy principles for Tikhonov regularization of ill posed problems leading to optimal convergence rates. Journal of Optimization Theory and Applications, 1987, 52(2): 209–215

    Article  MathSciNet  MATH  Google Scholar 

  118. Gfrerer H. An a posteriori parameter choice for ordinary and iterated Tikhonov regularization of ill-posed problems leading to optimal convergence rates. Mathematics of Computation, 1987, 49(180): 507–522

    Article  MathSciNet  MATH  Google Scholar 

  119. Natterer F. Error bounds for Tikhonov regularization in Hilbert scales. Applicable Analysis, 1984, 18(1–2): 29–37

    Article  MathSciNet  MATH  Google Scholar 

  120. Neubauer A. An a posteriori parameter choice for Tikhonov regularization in the presence of modeling error. Applied Numerical Mathematics, 1988, 4(6): 507–519

    Article  MathSciNet  MATH  Google Scholar 

  121. Engl H W, Kunisch K, Neubauer A. Convergence rates for Tikhonov regularisation of non-linear ill-posed problems. Inverse Problems, 1989, 5(4): 523–540

    Article  MathSciNet  MATH  Google Scholar 

  122. Scherzer O, Engl H W, Kunisch K. Optimal a posteriori parameter choice for Tikhonov regularization for solving nonlinear ill-posed problems. SIAM Journal on Numerical Analysis, 1993, 30(6): 1796–1838

    Article  MathSciNet  MATH  Google Scholar 

  123. Varah J M. On the numerical solution of ill-conditioned linear systems with applications to ill-posed problems. SIAM Journal on Numerical Analysis, 1973, 10(2): 257–267

    Article  MathSciNet  MATH  Google Scholar 

  124. Vinod H D, Ullah A. Recent Advances in Regression Methods. Danbury: Marcel Dekker Incorporated, 1981

    MATH  Google Scholar 

  125. O’Sullivan F. A statistical perspective on ill-posed inverse problems. Statistical Science, 1986, 1(4): 502–518

    Article  MathSciNet  MATH  Google Scholar 

  126. Grafarend EW, Schaffrin B. Ausgleichungsrechnung in linearen modellen. BI Wissenschaftsverlag Mannheim, 1993

    Google Scholar 

  127. Rodgers C D. Inverse Methods for Atmospheric Sounding: Theory and Practice. Singapore: World Scientific, 2000

    Book  MATH  Google Scholar 

  128. Ceccherini S. Analytical determination of the regularization parameter in the retrieval of atmospheric vertical profiles. Optics Letters, 2005, 30(19): 2554–2556

    Article  Google Scholar 

  129. Mallows C L. Some comments on C p. Technometrics, 1973, 15(4): 661–675

    MATH  Google Scholar 

  130. Rice J. Choice of smoothing parameter in deconvolution problems. Contemporary Mathematics, 1986, 59: 137–151

    Article  MATH  Google Scholar 

  131. Hanke M, Raus T. A general heuristic for choosing the regularization parameter in ill-posed problems. SIAM Journal on Scientific Computing, 1996, 17(4): 956–972

    Article  MathSciNet  MATH  Google Scholar 

  132. Wu L M. A parameter choice method for Tikhonov regularization. Electronic Transactions on Numerical Analysis, 2003, 16: 107–128

    MathSciNet  MATH  Google Scholar 

  133. Gao W, Yu K P. A new method for determining the Tikhonov regularization parameter of load identification. In: Proceedings of the International Symposium on Precision Engineering Measurement and Instrumentation. 2015

    Google Scholar 

  134. Ito K, Jin B, Takeuchi T. Multi-parameter Tikhonov regularizationan augmented approach. Chinese Annals of Mathematics, Series B, 2014, 35(03): 383–398

    Article  MathSciNet  MATH  Google Scholar 

  135. Jin B, Lorenz D A. Heuristic parameter-choice rules for convex variational regularization based on error estimates. SIAM Journal on Numerical Analysis, 2010, 48(3): 1208–1229

    Article  MathSciNet  MATH  Google Scholar 

  136. Pazos F, Bhaya A. Adaptive choice of the Tikhonov regularization parameter to solve ill-posed linear algebraic equations via Liapunov optimizing control. Journal of Computational and Applied Mathematics, 2015, 279: 123–132

    Article  MathSciNet  MATH  Google Scholar 

  137. Hämarik U, Palm R, Raus T. A family of rules for parameter choice in Tikhonov regularization of ill-posed problems with inexact noise level. Journal of Computational and Applied Mathematics, 2012, 236(8): 2146–2157

    Article  MathSciNet  MATH  Google Scholar 

  138. Reichel L, Rodriguez G. Old and new parameter choice rules for discrete ill-posed problems. Numerical Algorithms, 2013, 63(1): 65–87

    Article  MathSciNet  MATH  Google Scholar 

  139. Kryanev A V. An iterative method for solving incorrectly posed problems. USSR Computational Mathematics and Mathematical Physics, 1974, 14(1): 24–35

    Article  MathSciNet  Google Scholar 

  140. King J T, Chillingworth D. Approximation of generalized inverses by iterated regularization. Numerical Functional Analysis & Optimization, 1979, 1(5): 499–513

    Article  MathSciNet  MATH  Google Scholar 

  141. Fakeev A G. A class of iterative processes for solving degenerate systems of linear algebraic equations. USSR Computational Mathematics and Mathematical Physics, 1981, 21(3): 15–22

    Article  MathSciNet  MATH  Google Scholar 

  142. Brill M, Schock E. Iterative solution of ill-posed problems: a survey. In: Proceedings of the 4th International Mathematical Geophysics Seminar. 1987

    Google Scholar 

  143. Hanke M, Groetsch CW. Nonstationary iterated Tikhonov regularization. Journal of Optimization Theory and Applications, 1998, 98(1): 37–53

    Article  MathSciNet  MATH  Google Scholar 

  144. Lampe J, Reichel L, Voss H. Large-scale Tikhonov regularization via reduction by orthogonal projection. Linear Algebra and Its Applications, 2012, 436(8): 2845–2865

    Article  MathSciNet  MATH  Google Scholar 

  145. Reichel L, Yu X B. Tikhonov regularization via flexible Arnoldi reduction. BIT Numerical Mathematics, 2015, 55(4): 1145–1168

    Article  MathSciNet  MATH  Google Scholar 

  146. Huang G, Reichel L, Yin F. Projected nonstationary iterated Tikhonov regularization. BIT Numerical Mathematics, 2016, 56(2): 467–487

    Article  MathSciNet  MATH  Google Scholar 

  147. Ambrosio L, Fusco N, Pallara D. Functions of Bounded Variation and Free Discontinuity Problems. Oxford: Oxford University Press, 2000

    MATH  Google Scholar 

  148. Acar R, Vogel C R. Analysis of bounded variation penalty methods for ill-posed problems. Inverse Problems, 1997, 10(6): 1217–1229

    Article  MathSciNet  MATH  Google Scholar 

  149. Hunt B R. The application of constrained least squares estimation to image restoration by digital computer. IEEE Transactions on Computers, 1973, 100(9): 805–812

    Article  Google Scholar 

  150. Demoment G. Image reconstruction and restoration: overview of common estimation structures and problems. IEEE Transactions on Acoustics, Speech and Signal Processing, 1989, 37(12): 2024–2036

    Article  Google Scholar 

  151. Katsaggelos A K. Iterative image restoration algorithms. Optical Engineering, 1989, 28(7): 735–748

    Article  Google Scholar 

  152. Katsaggelos A K, Biemond J, Schafer R W, Mersereau R M. A regularized iterative image restoration algorithm. IEEE Transactions on Signal Processing, 1991, 39(4): 914–929

    Article  Google Scholar 

  153. Babacan S D, Molina R, Katsaggelos A K. Parameter estimation in TV image restoration using variational distribution approximation. IEEE Transactions on Image Processing, 2008, 17(3): 326–339

    Article  MathSciNet  Google Scholar 

  154. Wen Y W, Chan R H. Parameter selection for total-variation-based image restoration using discrepancy principle. IEEE Transactions on Image Processing, 2012, 21(4): 1770–1781

    Article  MathSciNet  Google Scholar 

  155. Chen A, Huo B M, Wen CW. Adaptive regularization for color image restoration using discrepancy principle. In: Proceedings of the IEEE International Conference on Signal processing, Comminications and Computing. 2013, 1–6

    Google Scholar 

  156. Lin Y, Wohlberg B, Guo H. UPRE method for total variation parameter selection. Signal Processing, 2010, 90(8): 2546–2551

    Article  MATH  Google Scholar 

  157. Stein C M. Estimation of the mean of a multivariate normal distribution. Annals of Statistics, 1981, 9(6): 1135–1151

    Article  MathSciNet  MATH  Google Scholar 

  158. Ramani S, Blu T, Unser M. Monte-Carlo SURE: a black-box optimization of regularization parameters for general denoising algorithms. IEEE Transactions on Image Processing, 2008, 17(9): 1540–1554

    Article  MathSciNet  Google Scholar 

  159. Palsson F, Sveinsson J R, Ulfarsson M O, Benediktsson J A. SAR image denoising using total variation based regularization with surebased optimization of regularization parameter. In: Proceedings of the IEEE International Conference on Geoscience and Remote Sensing Symposium. 2012, 2160–2163

    Google Scholar 

  160. Liao H Y, Li F, Ng MK. Selection of regularization parameter in total variation image restoration. Journal of the Optical Society of America A, 2009, 26(11): 2311–2320

    Article  MathSciNet  Google Scholar 

  161. Bertalmío M, Caselles V, Rougé B, Solé A. TV based image restoration with local constraints. Journal of Scientific Computing, 2003, 19(1–3): 95–122

    Article  MathSciNet  MATH  Google Scholar 

  162. Almansa A, Ballester C, Caselles V, Haro G. A TV based restoration model with local constraints. Journal of Scientific Computing, 2008, 34(3): 209–236

    Article  MathSciNet  MATH  Google Scholar 

  163. Vogel C R, Oman ME. Iterative methods for total variation denoising. SIAM Journal on Scientific Computing, 1997, 17(1): 227–238

    Article  MathSciNet  MATH  Google Scholar 

  164. Chan T F, Golub G H, Mulet P. A nonlinear primal-dual method for total variation-based image restoration. Lecture Notes in Control & Information Sciences, 1995, 20(6): 1964–1977

    MathSciNet  MATH  Google Scholar 

  165. Chambolle A. An algorithm for total variation minimization and applications. Journal of Mathematical Imaging & Vision, 2004, 20(1–2): 89–97

    MathSciNet  Google Scholar 

  166. Huang Y M, Ng M K, Wen Y W. A fast total variation minimization method for image restoration. SIAM Journal on Multiscale Modeling & Simulation, 2008, 7(2): 774–795

    Article  MathSciNet  MATH  Google Scholar 

  167. Bresson X, Chan T F. Fast dual minimization of the vectorial total variation norm and applications to color image processing. Inverse Problems & Imaging, 2008, 2(4): 455–484

    Article  MathSciNet  MATH  Google Scholar 

  168. Ng M K, Qi L Q, Yang Y F, Huang Y M. On semismooth Newton’s methods for total variation minimization. Journal of Mathematical Imaging & Vision, 2007, 27(3): 265–276

    Article  MathSciNet  Google Scholar 

  169. Zhu M Q, Chan T F. An efficient primal-dual hybrid gradient algorithm for total variation image restoration. UCLA CAM Report. 2008, 8–34

    Google Scholar 

  170. Zhu M Q, Wright S J, Chan T F. Duality-based algorithms for total-variation-regularized image restoration. Computational Optimization and Applications, 2010, 47(3): 377–400

    Article  MathSciNet  MATH  Google Scholar 

  171. Krishnan D, Lin P, Yip A M. A primal-dual active-set method for non-negativity constrained total variation deblurring problems. IEEE Transactions on Image Processing, 2007, 16(11): 2766–2777

    Article  MathSciNet  Google Scholar 

  172. Krishnan D, Pham Q V, Yip A M. A primal-dual active-set algorithm for bilaterally constrained total variation deblurring and piecewise constant Mumford-Shah segmentation problems. Advances in Computational Mathematics, 2009, 31(1–3): 237–266

    Article  MathSciNet  MATH  Google Scholar 

  173. Osher S, Burger M, Goldfarb D, Xu J J, Yin WT. An iterative regularization method for total variation-based image restoration. Multiscale Modeling & Simulation, 2005, 4(2): 460–489

    Article  MathSciNet  MATH  Google Scholar 

  174. Goldstein T, Osher S. The split Bregman method for l 1-regularized problems. SIAM Journal on Imaging Sciences, 2009, 2(2): 323–343

    Article  MathSciNet  MATH  Google Scholar 

  175. Glowinski R, Le Tallec P. Augmented Lagrangian and Operator-Splitting Methods in Nonlinear Mechanics. Society for Industrial and Applied Mathematics, 1989

    Google Scholar 

  176. Wu C C, Tai X C. Augmented Lagrangian method, dual methods, and split Bregman iteration for ROF, vectorial TV, and high order models. SIAM Journal on Imaging Sciences, 2010, 3(3): 300–339

    Article  MathSciNet  MATH  Google Scholar 

  177. Darbon J, Sigelle M. Image restoration with discrete constrained total variation part I: fast and exact optimization. Journal of Mathematical Imaging & Vision, 2006, 26(3): 261–276

    Article  MathSciNet  Google Scholar 

  178. Duan Y P, Tai X C. Domain decomposition methods with graph cuts algorithms for total variation minimization. Advances in Computational Mathematics, 2012, 36(2): 175–199

    Article  MathSciNet  MATH  Google Scholar 

  179. Fu H Y, Ng M K, Nikolova M, Barlow J L. Efficient minimization methods of mixed l2-l 1 and l 1-l 1 norms for image restoration. SIAM Journal on Scientific Computing, 2005, 27(6): 1881–1902

    Article  MATH  Google Scholar 

  180. Goldfarb D, Yin W T. Second-order cone programming methods for total variation-based image restoration. SIAM Journal on Scientific Computing, 2005, 27(2): 622–645

    Article  MathSciNet  MATH  Google Scholar 

  181. Oliveira J P, Bioucas-Dias J M, Figueiredo M A T. Adaptive total variation image deblurring: a majorization-minimization approach. Signal Processing, 2009, 89(9): 1683–1693

    Article  MATH  Google Scholar 

  182. Bioucas-Dias J M, Figueiredo M A T, Oliveira J P. Total variationbased image deconvolution: a majorization-minimization approach, In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing. 2006, 861–864

    Google Scholar 

  183. Chan T F, Esedoglu S. Aspects of total variation regularized l 1 function approximation. SIAM Journal on Applied Mathematics, 2004, 65(5): 1817–1837

    Article  MathSciNet  MATH  Google Scholar 

  184. He L, Burger M, Osher S. Iterative total variation regularization with non-quadratic fidelity. Journal of Mathematical Imaging & Vision, 2006, 26(1–2): 167–184

    Article  MathSciNet  Google Scholar 

  185. Jonsson E, Huang S C, Chan T F. Total variation regularization in positron emission tomography. CAM Report. 1998

    Google Scholar 

  186. Panin V Y, Zeng G L, Gullberg G T. Total variation regulated EM algorithm. IEEE Transactions on Nuclear Science, 1999, 46(6): 2202–2210

    Article  Google Scholar 

  187. Le T, Chartrand R, Asaki T J. A variational approach to reconstructing images corrupted by Poisson noise. Journal of Mathematical Imaging & Vision, 2007, 27(3): 257–263

    Article  MathSciNet  Google Scholar 

  188. Rudin L, Lions P L, Osher S. Multiplicative denoising and deblurring: theory and algorithms. In: Osher S, Paragios N, eds. Geometric Level Set Methods in Imaging, Vision, and Graphics. New York: Springer, 2003, 103–119

    Chapter  Google Scholar 

  189. Huang Y M, Ng M K, Wen Y W. A new total variation method for multiplicative noise removal. SIAM Journal on Imaging Sciences, 2009, 2(1): 20–40

    Article  MathSciNet  MATH  Google Scholar 

  190. Bonesky T, Kazimierski K S, Maass P, Schöpfer F, Schuster T. Minimization of Tikhonov functionals in Banach spaces. Abstract & Applied Analysis, 2008, 2008(1): 1563–1569

    MathSciNet  MATH  Google Scholar 

  191. Meyer Y. Oscillating Patterns in Image Processing and Nonlinear Evolution Equations, University Lecture Series. Rhode Island: American Mathematical Society, 2002

    Google Scholar 

  192. Blomgren P, Chen T F. Color TV: total variation methods for restoration of vector valued images. IEEE Transactions on Image Processing, 1970, 7(3): 304–309

    Article  Google Scholar 

  193. Setzer S, Steidl G, Popilka B, Burgeth B. Variational methods for denoising matrix fields. In: Laidlaw D, Weickert J, eds. Visualization and Processing of Tensor Fields. Berlin: Springer Berlin Heidelberg, 2009, 341–360

    Chapter  Google Scholar 

  194. Esedoglu S, Osher S. Decomposition of images by the anisotropic Rudin-Osher-Fatemi model. Communications on Pure and Applied Mathematics, 2004, 57(12): 1609–1626

    Article  MathSciNet  MATH  Google Scholar 

  195. Shi Y Y, Chang Q S. Efficient algorithm for isotropic and anisotropic total variation deblurring and denoising. Journal of Applied Mathematics, 2013

    Google Scholar 

  196. Marquina A, Osher S. Explicit algorithms for a new time dependent model based on level set motion for nonlinear deblurring and noise removal. SIAM Journal on Scientific Computing, 2000, 22(2): 387–405

    Article  MathSciNet  MATH  Google Scholar 

  197. Chan T F, Marquina A, Mulet P. High-order total variation-based image restoration. SIAM Journal on Scientific Computing, 2000, 22(2): 503–516

    Article  MathSciNet  MATH  Google Scholar 

  198. Gilboa G, Osher S. Nonlocal operators with applications to image processing. SIAM Journal on Multiscale Modeling & Simulation, 2008, 7(3): 1005–1028

    Article  MathSciNet  MATH  Google Scholar 

  199. Kindermann S, Osher S, Jones P W. Deblurring and denoising of images by nonlocal functionals. SIAM Journal on Multiscale Modeling & Simulation, 2005, 4(4): 1091–1115

    Article  MathSciNet  MATH  Google Scholar 

  200. Hu Y, Jacob M. Higher degree total variation (HDTV) regularization for image recovery. IEEE Transactions on Image Processing, 2012, 21(5): 2559–2571

    Article  MathSciNet  Google Scholar 

  201. Yang J S, Yu H Y, Jiang M, Wang G. High-order total variation minimization for interior SPECT. Inverse Problems, 2012, 28(1): 15001–15024

    Article  MathSciNet  MATH  Google Scholar 

  202. Liu X W, Huang L H. A new nonlocal total variation regularization algorithm for image denoising. Mathematics and Computers in Simulation, 2014, 97: 224–233

    Article  MathSciNet  Google Scholar 

  203. Ren Z M, He C J, Zhang Q F. Fractional order total variation regularization for image super-resolution. Signal Processing, 2013, 93(9): 2408–2421

    Article  Google Scholar 

  204. Oh S, Woo H, Yun S, Kang M. Non-convex hybrid total variation for image denoising. Journal of Visual Communication & Image Representation, 2013, 24(3): 332–344

    Article  Google Scholar 

  205. Donoho D L. Compressed sensing. IEEE Transactions on Information Theory, 2006, 52(4): 1289–1306

    Article  MathSciNet  MATH  Google Scholar 

  206. Candè E J, Wakin M B. An introduction to compressive sampling. IEEE Signal Processing Magazine, 2008, 25(2): 21–30

    Article  Google Scholar 

  207. Tsaig Y, Donoho D L. Extensions of compressed sensing. Signal Processing, 2006, 86(3): 549–571

    Article  MATH  Google Scholar 

  208. Candès E J, Romberg J, Tao T. Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Transactions on Information Theory, 2006, 52(2): 489–509

    Article  MathSciNet  MATH  Google Scholar 

  209. Candès E J, Tao T. Near-optimal signal recovery from random projections: Universal encoding strategies?. IEEE Transactions on Information Theory, 2006, 52(12): 5406–5425

    Article  MathSciNet  MATH  Google Scholar 

  210. Donoho D L, Elad M. Optimally sparse representation in general (nonorthogonal) dictionaries via l 1 minimization. Proceedings of National Academy of Sciences, 2003, 100(5): 2197–2202

    Article  MathSciNet  MATH  Google Scholar 

  211. Wright J, Ma Y. Dense error correction via l 1-minimization. IEEE Transactions on Information Theory, 2010, 56(7): 3540–3560

    Article  MathSciNet  Google Scholar 

  212. Yang J F, Zhang Y. Alternating direction algorithms for ℓ1-problems in compressive sensing. SIAM Journal on Scientific Computing, 2011, 33(1): 250–278.

    Article  MathSciNet  MATH  Google Scholar 

  213. Natarajan B K. Sparse approximate solutions to linear systems. SIAM Journal on Computing, 1995, 24(2): 227–234

    Article  MathSciNet  MATH  Google Scholar 

  214. Mallat S G, Zhang Z. Matching pursuits with time-frequency dictionaries. IEEE Transactions on Signal Processing, 1993, 41(12): 3397–3415

    Article  MATH  Google Scholar 

  215. Tropp J, Gilbert A C. Signal recovery from random measurements via orthogonal matching pursuit. IEEE Transactions on Information Theory, 2007, 53(12): 4655–4666

    Article  MathSciNet  MATH  Google Scholar 

  216. Blumensath T, Davies M E. Iterative thresholding for sparse approximations. Journal of Fourier Analysis and Applications, 2008, 14(5–6): 629–654

    Article  MathSciNet  MATH  Google Scholar 

  217. Gorodnitsky I F, Rao B D. Sparse signal reconstruction from limited data using FOCUSS: a re-weighted minimum norm algorithm. IEEE Transactions on Signal Processing, 1997, 45(3): 600–616

    Article  Google Scholar 

  218. Bao C L, Ji H, Quan Y H, Shen Z W. l 0 norm based dictionary learning by proximal methods with global convergence. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2014, 3858–3865

    Google Scholar 

  219. Foucart S, Lai M J. Sparsest solutions of underdetermined linear systems via l q-minimization for 0 < q ≤ 1. Applied and Computational Harmonic Analysis, 2009, 26(3): 395–407

    Article  MathSciNet  MATH  Google Scholar 

  220. Cai T T, Wang L, Xu G. Shifting inequality and recovery of sparse signals. IEEE Transactions on Signal Processing, 2010, 58(3): 1300–1308

    Article  MathSciNet  Google Scholar 

  221. Cai T T, Wang L, Xu G. New bounds for restricted isometry constants. IEEE Transactions on Information Theory, 2010, 56(9): 4388–4394

    Article  MathSciNet  Google Scholar 

  222. Chen S S, Donoho D L, Saunders M A. Atomic decomposition by basis pursuit. SIAM Journal on Scientific Computing, 1998, 20(1): 33–61

    Article  MathSciNet  MATH  Google Scholar 

  223. Efron B, Hastie T, Johnstone I, Tibshirani R. Least angle regression. The Annals of Statistics, 2004, 32(2): 407–499

    Article  MathSciNet  MATH  Google Scholar 

  224. Figueiredo M A T, Nowak R D. An EM algorithm for wavelet-based image restoration. IEEE Transactions on Image Processing, 2002, 12(8): 906–916

    Article  MathSciNet  MATH  Google Scholar 

  225. Starck J L, Mai K N, Murtagh F. Wavelets and curvelets for image deconvolution: a combined approach. Signal Processing, 2003, 83(10): 2279–2283

    Article  MATH  Google Scholar 

  226. Herrholz E, Teschke G. Compressive sensing principles and iterative sparse recovery for inverse and ill-posed problems. Inverse Problems, 2010, 26(12): 125012–125035

    Article  MathSciNet  MATH  Google Scholar 

  227. Jin B, Lorenz D, Schiffler S. Elastic-net regularization: error estimates and active set methods. Inverse Problems, 2009, 25(11): 1595–1610

    Article  MathSciNet  MATH  Google Scholar 

  228. Figueiredo M A T, Nowak R D, Wright S J. Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems. IEEE Journal of Selected Topics in Signal Processing, 2007, 1(4): 586–597

    Article  Google Scholar 

  229. Kim S J, Koh K, Lustig M, Boyd S, Gorinevsky D. An interior-point method for large-scale l 1-regularized least squares. IEEE Journal of Selected Topics in Signal Processing, 2007, 1(4): 606–617

    Article  Google Scholar 

  230. Donoho D L, Tsaig Y. Fast solution of l 1-norm minimization problems when the solution may be sparse. IEEE Transactions on Information Theory, 2008, 54(11): 4789–4812

    Article  MathSciNet  MATH  Google Scholar 

  231. Combettes P L, Wajs E R. Signal recovery by proximal forwardbackward splitting. SIAM Journal on Multiscale Modeling & Simulation, 2005, 4(4): 1168–1200

    Article  MATH  Google Scholar 

  232. Becker S, Bobin J, Candés E J. NESTA: a fast and accurate firstorder method for sparse recovery. SIAM Journal on Imaging Sciences, 2011, 4(1): 1–39

    Article  MathSciNet  MATH  Google Scholar 

  233. Osborne M R, Presnell B, Turlach B A. A new approach to variable selection in least squares problems. IMA Journal of Numerical Analysis, 1999, 20(3): 389–403

    Article  MathSciNet  MATH  Google Scholar 

  234. Li L, Yao X, Stolkin R, Gong M G, He S. An evolutionary multiobjective approach to sparse reconstruction. IEEE Transactions on Evolutionary Computation, 2014, 18(6): 827–845

    Article  Google Scholar 

  235. Chartrand R. Exact reconstruction of sparse signals via nonconvex minimization. IEEE Signal Processing Letters, 2007, 14(10): 707–710

    Article  Google Scholar 

  236. Candes E J, Tao T. Decoding by linear programming. IEEE Transactions on Information Theory, 2005, 51(12): 4203–4215

    Article  MathSciNet  MATH  Google Scholar 

  237. Saab R, Chartrand R, Yilmaz Ö. Stable sparse approximations via nonconvex optimization. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing. 2008, 3885–3888

    Google Scholar 

  238. Tibshirani R. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society, Series B, 1996, 58(1): 267–288

    MathSciNet  MATH  Google Scholar 

  239. Zhang C H. Nearly unbiased variable selection under minimax concave penalty. The Annals of Statistics, 2010, 38(2): 894–942

    Article  MathSciNet  MATH  Google Scholar 

  240. Fan J Q, Li R. Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 2001, 96(456): 1348–1360

    Article  MathSciNet  MATH  Google Scholar 

  241. Nikolova M, Ng M K, Zhang S, Ching W K. Efficient reconstruction of piecewise constant images using nonsmooth nonconvex minimization. SIAM Journal on Imaging Sciences, 2008, 1(1): 2–25

    Article  MathSciNet  MATH  Google Scholar 

  242. Frank L E, Friedman J H. A statistical view of some chemometrics regression tools. Technometrics, 1993, 35(2): 109–135

    Article  MATH  Google Scholar 

  243. Fu W J. Penalized regressions: the bridge versus the lasso. Journal of Computational and Graphical Statistics, 1998, 7(3): 397–416

    MathSciNet  Google Scholar 

  244. Lyu Q, Lin Z C, She Y Y, Zhang C. A comparison of typical l p minimization algorithms. Neurocomputing, 2013, 119: 413–424

    Article  Google Scholar 

  245. Candes E J, Wakin M B, Boyd S P. Enhancing sparsity by reweighted l 1 minimization. Journal of Fourier Analysis and Applications, 2008, 14(5–6): 877–905

    Article  MathSciNet  MATH  Google Scholar 

  246. Rao B D, Kreutz-Delgado K. An affine scaling methodology for best basis selection. IEEE Transactions on Signal Processing, 1999, 47(1): 187–200

    Article  MathSciNet  MATH  Google Scholar 

  247. She Y Y. Thresholding-based iterative selection procedures for model selection and shrinkage. Electronic Journal of Statistics, 2009, 3: 384–415

    Article  MathSciNet  MATH  Google Scholar 

  248. Xu Z B, Zhang H, Wang Y, Chang X Y, Liang Y. L 1/2 regularization. Science China Information Sciences, 2010, 53(6): 1159–1169

    Article  MathSciNet  Google Scholar 

  249. Xu Z B, Guo H L, Wang Y, Zhang H. Representative of L 1/2 regularization among l q (0 < q ≤ 1) regularizations: an experimental study based on phase diagram. Acta Automatica Sinica, 2012, 38(7): 1225–1228

    MathSciNet  Google Scholar 

  250. Candes E J, Plan Y. Matrix completion with noise. Proceedings of the IEEE, 2009, 98(6): 925–936

    Article  Google Scholar 

  251. Cai J F, Candes E J, Shen Z. A singular value thresholding algorithm for matrix completion. SIAM Journal on Optimization, 2010, 20(4): 1956–1982

    Article  MathSciNet  MATH  Google Scholar 

  252. Boyd S, Parikh N, Chu E, Peleato B, Eckstein J. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations & Trends in Machine Learning, 2011, 3(1): 1–122

    Article  MATH  Google Scholar 

  253. Qian J J, Yang J, Zhang F L, Lin Z C. Robust low-rank regularized regression for face recognition with occlusion. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2014, 21–26

    Google Scholar 

  254. Liu Y J, Sun D, Toh K C. An implementable proximal point algorithmic framework for nuclear norm minimization. Mathematical Programming, 2012, 133(1–2): 399–436

    Article  MathSciNet  MATH  Google Scholar 

  255. Yang J F, Yuan X M. Linearized augmented Lagrangian and alternating direction methods for nuclear norm minimization. Mathematics of Computation, 2013, 82(281): 301–329

    Article  MathSciNet  MATH  Google Scholar 

  256. Li T, Wang W W, Xu L, Feng X C. Image denoising using lowrank dictionary and sparse representation. In: Proceedings of the 10th IEEE International Conference on Computational Intelligence and Security. 2014, 228–232

    Google Scholar 

  257. Waters A E, Sankaranarayanan A C, Baraniuk R G. SpaRCS: recovering low-rank and sparse matrices from compressive measurements. In: Proceedings of the Neural Information Processing Systems Conference. 2011, 1089–1097

    Google Scholar 

  258. Li Q, Lu Z B, Lu Q B, Li H Q, Li WP. Noise reduction for hyperspectral images based on structural sparse and low-rank matrix decomposition. In: Proceedings of the IEEE International on Geoscience and Remote Sensing Symposium. 2013, 1075–1078

    Google Scholar 

  259. Zhou T Y, Tao D C. Godec: randomized low-rank & sparse matrix decomposition in noisy case. In: Proceedings of the 28th International Conference on Machine Learning. 2011, 33–40

    Google Scholar 

  260. Zhang H Y, He W, Zhang L P, Shen H F, Yuan Q Q. Hyperspectral image restoration using low-rank matrix recovery. IEEE Transactions on Geoscience & Remote Sensing, 2014, 52(8): 4729–4743

    Article  Google Scholar 

  261. Zhang Z, Xu Y, Yang J, Li X L, Zhang D. A survey of sparse representation: algorithms and applications. IEEE Access, 2015, 3: 490–530

    Article  Google Scholar 

  262. Burger M, Franek M, Schönlieb C B. Regularized regression and density estimation based on optimal transport. Applied Mathematics Research eXpress, 2012, 2012(2): 209–253

    MathSciNet  MATH  Google Scholar 

  263. Osher S, Solè A, Vese L. Image decomposition and restoration using total variation minimization and the H 1 norm. Multiscale Modeling & Simulation, 2003, 1(3): 349–370

    Article  MathSciNet  MATH  Google Scholar 

  264. Barbara K. Iterative regularization methods for nonlinear ill-posed problems. Algebraic Curves & Finite Fields Cryptography & Other Applications, 2008, 6

    Google Scholar 

  265. Miettinen K. Nonlinear Multiobjective Optimization. Springer Science & Business Media, 2012

    MATH  Google Scholar 

  266. Marler R T, Arora J S. Survey of multi-objective optimization methods for engineering. Structural and Multidisciplinary Optimization, 2004, 26(6): 369–395

    Article  MathSciNet  MATH  Google Scholar 

  267. Gong M G, Jiao L C, Yang D D, Ma W P. Research on evolutionary multi-objective optimization algorithms. Journal of Software, 2009, 20(20): 271–289

    Article  MathSciNet  MATH  Google Scholar 

  268. Fonseca C M, Fleming P J. Genetic algorithm for multiobjective optimization: formulation, discussion and generation. In: Proceedings of the International Conference on Genetic Algorithms. 1993, 416–423

    Google Scholar 

  269. Srinivas N, Deb K. Multiobjective optimization using non-dominated sorting in genetic algorithms. Evolutionary Computation, 1994, 2(3): 221–248

    Article  Google Scholar 

  270. Horn J, Nafpliotis N, Goldberg D E. A niched Pareto genetic algorithm for multiobjective optimization. In: Proceedings of the 1st IEEE Conference on Evolutionary Computation. 1994, 1: 82–87

    Google Scholar 

  271. Zitzler E, Thiele L. Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach. IEEE Transactions on Evolutionary Computation, 1999, 3(4): 257–271

    Article  Google Scholar 

  272. Zitzler E, Laumanns M, Thiele L. SPEA2: improving the strength Pareto evolutionary algorithm. Eurogen, 2001, 3242(103): 95–100

    Google Scholar 

  273. Kim M, Hiroyasu T, Miki M, Watanabe S. SPEA2+: improving the performance of the strength Pareto evolutionary algorithm 2. In: Proceedings of the International Conference on Parallel Problem Solving from Nature. 2004, 742–751

    Google Scholar 

  274. Knowles J D, Corne D W. Approximating the non-dominated front using the Pareto archived evolution strategy. Evolutionary Computation, 2000, 8(2): 149–172

    Article  Google Scholar 

  275. Corne D W, Knowles J D, Oates M J. The Pareto-envelope based selection algorithm for multi-objective optimization. In: Proceedings of the Internatioal Conference on Parallel Problem Solving from Nature. 2000, 869–878

    Google Scholar 

  276. Corne D W, Jerram N R, Knowles J D, Oates M J. PESA-II: regionbased selection in evolutionary multi-objective optimization. In: Proceedings of the Genetic and Evolutionary Computation Conference. 2001, 283–290

    Google Scholar 

  277. Deb K, Agrawal S, Pratap A, Meyarivan T. A fast elitist nondominated sorting genetic algorithm for multi-objective optimization: NSGA-II. Lecture Notes in Computer Science, 2000, 1917: 849–858

    Article  Google Scholar 

  278. Zhang Q F, Li H. MOEA/D: a multiobjective evolutionary algorithm based on decomposition. IEEE Transactions on Evolutionary Computation, 2007, 11(6): 712–731

    Article  Google Scholar 

  279. Ishibuchi H, Sakane Y, Tsukamoto N, Nojima Y. Simultaneous use of different scalarizing functions in MOEA/D. In: Proceedings of the 12th Annual Conference on Genetic and Evolutionary Computation. 2010, 519–526

    Chapter  Google Scholar 

  280. Wang L P, Zhang Q F, Zhou A M, Gong M G, Jiao L C. Constrained subproblems in decomposition based multiobjective evolutionary algorithm. IEEE Transactions on Evolutionary Computation, 2016, 20(3): 475–480

    Article  Google Scholar 

  281. Li K, Fialho A, Kwong S, Zhang Q F. Adaptive operator selection with bandits for a multiobjective evolutionary algorithm based on decomposition. IEEE Transactions on Evolutionary Computation, 2014, 18(1): 114–130

    Article  Google Scholar 

  282. Ke L J, Zhang Q F, Battiti R. Hybridization of decomposition and local search for multiobjective optimization. IEEE Transactions on Cybernetics, 2014, 44(10): 1808–1820

    Article  Google Scholar 

  283. Cai X Y, Wei O. A hybrid of decomposition and domination based evolutionary algorithm for multi-objective software next release problem. In: Proceedings of the 10th IEEE International Conference on Control and Automation. 2013, 412–417

    Google Scholar 

  284. Deb K, Jain H. An evolutionary many-objective optimization algorithm using reference-point-based nondominated sorting approach, part I: solving problems with box constraints. IEEE Transactions on Evolutionary Computation, 2014, 18(4): 577–601

    Article  Google Scholar 

  285. Yuan Y, Xu H, Wang B. An improved NSGA-III procedure for evolutionary many-objective optimization. In: Proceedings of ACM Annual Conference on Genetic & Evolutionary Computation. 2014, 661–668

    Google Scholar 

  286. Seada H, Deb K. U-NSGA-III: a unified evolutionary optimization procedure for single, multiple, and many objectives: proof-ofprinciple results. In: Proceedings of the International Conference on Evolutionary Multi-Criterion Optimization. 2015, 34–49

    Google Scholar 

  287. Zhu Z X, Xiao J, Li J Q, Zhang Q F. Global path planning of wheeled robots using multi-objective memetic algorithms. Integrated Computer-Aided Engineering, 2015, 22(4): 387–404

    Article  Google Scholar 

  288. Zhu Z X, Jia S, He S, Sun Y W, Ji Z, Shen L L. Three-dimensional Gabor feature extraction for hyperspectral imagery classification using a memetic framework. Information Sciences, 2015, 298: 274–287

    Article  Google Scholar 

  289. Zhu Z X, Xiao J, He S, Ji Z, Sun Y W. A multi-objective memetic algorithm based on locality-sensitive hashing for one-to-many-to-one dynamic pickup-and-delivery problem. Information Sciences, 2015, 329: 73–89

    Article  Google Scholar 

  290. Li H, Gong M G, Wang Q, Liu J, Su L Z. A multiobjective fuzzy clustering method for change detection in synthetic aperture radar images. Applied Soft Computing, 2016, 46: 767–777

    Article  Google Scholar 

  291. Jin Y, Sendhoff B. Pareto based approach to machine learning: an overview and case studies. IEEE Transactions on Systems, Man, and Cybernetics, Part C, 2008, 38(3): 397–415

    Article  Google Scholar 

  292. Plumbley M D. Recovery of sparse representations by polytope faces pursuit. In: Proceedings of the 6th International Conference on Independent Component Analysis and Blind Source Separation. 2006, 206–213

    Chapter  Google Scholar 

  293. Wright S J, Nowak R D, Figueiredo M A T. Sparse reconstruction by separable approximation. IEEE Transactions on Signal Processing, 2009, 57(7): 2479–2493

    Article  MathSciNet  Google Scholar 

  294. Yang Y, Yao X, Zhou Z H. On the approximation ability of evolutionary optimization with application to minimum set cover. Artificial Intelligence, 2012, 180(2): 20–33

    MathSciNet  MATH  Google Scholar 

  295. Qian C, Yu Y, Zhou Z H. An analysis on recombination in multiobjective evolutionary optimization. Artificial Intelligence, 2013, 204(1): 99–119

    Article  MathSciNet  MATH  Google Scholar 

  296. Gong M G, Zhang M Y, Yuan Y. Unsupervised band selection based on evolutionary multiobjective optimization for hyperspectral images. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(1): 544–557

    Article  Google Scholar 

  297. Qian C, Yu Y, Zhou Z H. Pareto ensemble pruning. In: Proceedings of AAAI Conference on Artificial Intelligence. 2015, 2935–2941

    Google Scholar 

  298. Qian C, Yu Y, Zhou Z H. On constrained Boolean Pareto optimization. In: Proceedings of the 24th International Joint Conference on Artificial Intelligence. 2015, 389–395

    Google Scholar 

  299. Qian C, Yu Y, Zhou Z H. Subset selection by Pareto optimization. In: Proceedings of the Neural Information Processing Systems Conference. 2015, 1765–1773

    Google Scholar 

  300. Gong M G, Liu J, Li H, Cai Q, Su L Z. A multiobjective sparse feature learning model for deep neural networks. IEEE Transactions on Neural Networks and Learning Systems, 2015, 26(12): 3263–3277

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Nature Science Foundation of China (Grant No. 61422209), in part by the National Program for Support of Top-Notch Young Professionals of China.

Author information

Authors and Affiliations

  1. Key Laboratory of Intelligent Perception and Image Understanding of Ministry of Education, Xidian University, Xi’an, 710071, China

    Maoguo Gong, Xiangming Jiang & Hao Li

Authors
  1. Maoguo Gong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiangming Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Maoguo Gong.

Additional information

Maoguo Gong received the BS degree in electronic engineering and PhD degree in electronic science and technology from Xidian University, China in 2003 and 2009, respectively. He is currently a full professor with Xidian University. His research interests are in the area of computational intelligence with applications to optimization, learning, data mining and image understanding. He is a senior member of IEEE and Chinese Computer Federation. He is the awardee of the NSFC Excellent Young Scholars Program in 2014.

Xiangming Jiang received the BS degree in mathematics and applied mathematics from Xidian University, China in 2015. He is currently pursuing the MS degree in pattern recognition and intelligent systems at the School of Electronic Engineering, Xidian University. His research interests include image understanding and inverse problem.

Hao Li received the BS degree in electronic engineering from Xidian University, China in 2013. He is currently pursuing the PhD degree in pattern recognition and intelligent systems at the School of Electronic Engineering, Xidian University. His research interests include computational intelligence and image understanding.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gong, M., Jiang, X. & Li, H. Optimization methods for regularization-based ill-posed problems: a survey and a multi-objective framework. Front. Comput. Sci. 11, 362–391 (2017). https://doi.org/10.1007/s11704-016-5552-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11704-016-5552-0

Keywords

Search

Navigation