Skip to main content
SpringerLink
Log in

Multi-parameter regularization and its numerical realization

  • Published:
Numerische Mathematik Aims and scope Submit manuscript

Abstract

In this paper we propose and analyse a choice of parameters in the multi-parameter regularization of Tikhonov type. A modified discrepancy principle is presented within the multi-parameter regularization framework. An order optimal error bound is obtained under the standard smoothness assumptions. We also propose a numerical realization of the multi-parameter discrepancy principle based on the model function approximation. Numerical experiments on a series of test problems support theoretical results. Finally we show how the proposed approach can be successfully implemented in Laplacian Regularized Least Squares for learning from labeled and unlabeled examples.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bauer F., Ivanyshyn O.: Optimal regularization with two interdependent regularization parameters. Inverse Probl. 23, 331–342 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  2. Bauer F., Pereverzev S.V.: An utilization of a rough approximation of a noise covariance within the framework of multi-parameter regularization. Int. J. Tomogr. Stat. 4, 1–12 (2006)

    MATH  MathSciNet  Google Scholar 

  3. Bauer F., Pereverzev S.V., Rosasco L.: On regularization algorithms in learning theory. J. Complex. 23, 52–72 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  4. Belge M., Kilmer M.E., Miller E.L.: Efficient determination of multiple regularization parameters in a generalized L-curve framework. Inverse Probl. 18, 1161–1183 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  5. Belge, M., Miller, E.L.: Wavelet domain Bayesian image restoration using edge-preserving prior models. In: Prc. ICIP’98 (1998)

  6. Belkin M., Niyogi P., Sindhwani V.: Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J. Mach. Learn. Res. 7, 2399–2434 (2006)

    MathSciNet  Google Scholar 

  7. Brezinski C., Redivo-Zaglia M., Rodriguez G., Seatzu S.: Multi-parameter regularization techniques for ill-conditioned linear systems. Numer. Math. 94, 203–228 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  8. Brooks D.H., Ahmad G.F., MacLeod R.S., Maratos M.G.: Inverse electrocardiography by simultaneous imposition of multiple constraints. IEEE Trans. Biomed. Eng. 46, 3–18 (1998)

    Article  Google Scholar 

  9. Chapelle D.: Training a support vector machine in the primal. Neural Comput. 19, 1155–1178 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  10. Chen Z., Lu Y., Xu Y., Yang H.: Multi-parameter Tikhonov regularization for linear ill-posed operator equations. J. Comp. Math. 26, 37–55 (2008)

    MATH  MathSciNet  Google Scholar 

  11. De Vito E., Pereverzev S.V., Rosasco L.: Adaptive Kernel Methods via the Balancing Principle. Found. Comput. Math. 10, 455–479 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  12. Düvelmeyer D., Hofmann B.: A multi-parameter regularization approach for estimating parameters in jump diffusion processes. J. Inverse Ill-Posed Probl. 14, 861–880 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  13. Engl H.W., Hanke M., Neubauer A.: Regularization of Inverse Problems. Kluwer, Dordrecht (1996)

    MATH  Google Scholar 

  14. Hansen P.C.: Regularization tools: a Matlab package for analysis and solution of discrete ill-posed problems. Numer. Algorithms 6, 1–35 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  15. Hofmann B.: Regularization of Applied Inverse and Ill-posed Problems. Teubner, Leipzig (1998)

    Google Scholar 

  16. Jin, B., Lorenz, D.A., Schiffler, S.: Elastic-net regularization: error estimates and active set methods. Inverse Probl. 25, 115022, p. 26 (2009)

    Google Scholar 

  17. Kaltenbacher B., Neubauer A., Scherzer O.: Iterative Regularization Methods for Nonlinear Problems. de Gruyter, Berlin (2008)

    Book  MATH  Google Scholar 

  18. Krein S., Petunin Y.I.: Scales of Banach spaces. Russian Math. Surv. 14, 85–159 (1966)

    Article  Google Scholar 

  19. Kunisch K., Zou J.: Iterative choices of regularization parameters in linear inverse problems. Inverse Probl. 14, 1247–1264 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  20. Lu S., Pereverzev S.V.: Sparse recovery by the standard Tikhonov method. Numer. Math. 112, 403–424 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  21. Lu, S., Pereverzev, S.V., Tautenhahn, U.: A model function method in total least squares. RICAM report (2008)

  22. Lu S., Pereverzev S.V., Tautenhahn U.: Dual regularized total least squares and multi-parameter regularization. Comput. Methods Appl. Math. 8, 253–262 (2008)

    MATH  MathSciNet  Google Scholar 

  23. Lu, S., Pereverzev, S.V., Shao, Y., Tautenhahn, U.: On the generalized discrepancy principle for Tikhonov regularization in Hilbert scales, RICAM 2009-19. J. Integral Equ. Appl. (2010, accepted)

  24. Micchelli C.A., Pontil M.: Learning the kernel function via regularization. J. Mach. Learn. Res. 6, 1099–1125 (2005)

    MathSciNet  Google Scholar 

  25. Morozov V.A.: On the solution of functional equations by the method of regularization. Soviet Math. Dokl. 7, 414–417 (1966)

    MATH  MathSciNet  Google Scholar 

  26. Natterer F.: Error bounds for Tikhonov regularization in Hilbert scales. Appl. Anal. 18, 29–37 (1984)

    Article  MATH  MathSciNet  Google Scholar 

  27. Phillips D.: A technique for the numerical solution of certain integral equation of the first kind. J. Assoc. Comput. Mach. 9, 84–97 (1962)

    MATH  MathSciNet  Google Scholar 

  28. Poggio T., Smale S.: The mathematics of learning: dealing with data. Notices Am. Math. Soc. 50, 537–544 (2003)

    MATH  MathSciNet  Google Scholar 

  29. Xie J., Zou J.: An improved model function method for choosing regularization parameters in linear inverse problems.. 18, 631–643 (2002)

    MATH  MathSciNet  Google Scholar 

  30. Xu P.L., Fukuda Y., Liu Y.M.: Multiple parameter regularization: numerical solutions and applications to the determination of geopotential from precise satellite orbits. J. Geod. 80, 17–27 (2006)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Johann Radon Institute for Computational and Applied Mathematics, Austrian Academy of Sciences, Altenbergstrasse 69, 4040, Linz, Austria

    Shuai Lu & Sergei V. Pereverzev

  2. School of Mathematical Sciences, Fudan University, Shanghai, 200433, China

    Shuai Lu

Authors
  1. Shuai Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sergei V. Pereverzev
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shuai Lu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lu, S., Pereverzev, S.V. Multi-parameter regularization and its numerical realization. Numer. Math. 118, 1–31 (2011). https://doi.org/10.1007/s00211-010-0318-3

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00211-010-0318-3

Mathematics Subject Classification (2000)

Search

Navigation