Skip to main content
SpringerLink
Log in

A Dynamical Tikhonov Regularization for Solving Ill-posed Linear Algebraic Systems

  • Published:
Acta Applicandae Mathematicae Aims and scope Submit manuscript

Abstract

The Tikhonov method is a famous technique for regularizing ill-posed linear problems, wherein a regularization parameter needs to be determined. This article, based on an invariant-manifold method, presents an adaptive Tikhonov method to solve ill-posed linear algebraic problems. The new method consists in building a numerical minimizing vector sequence that remains on an invariant manifold, and then the Tikhonov parameter can be optimally computed at each iteration by minimizing a proper merit function. In the optimal vector method (OVM) three concepts of optimal vector, slow manifold and Hopf bifurcation are introduced. Numerical illustrations on well known ill-posed linear problems point out the computational efficiency and accuracy of the present OVM as compared with classical ones.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Barzilai, J., Borwein, J.M.: Two point step size gradient methods. IMA J. Numer. Anal. 8, 141–148 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  2. Bhaya, A., Kaszkurewicz, E.: Control Perspectives on Numerical Algorithms and Matrix Problems. Advances in Design and Control, vol. 10. SIAM, Philadelphia (2006)

    Book  MATH  Google Scholar 

  3. Chehab, J.-P., Laminie, J.: Differential equations and solution of linear systems. Numer. Algorithms 40, 103–124 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  4. Dai, Y.H., Liao, L.H.: R-linear convergence of the Barzilai and Borwein gradient method. IMA J. Numer. Anal. 22, 1–10 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  5. Dai, Y.H., Yuan, Y.: Alternate minimization gradient method. IMA J. Numer. Anal. 23, 377–393 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  6. Dai, Y.H., Yuan, J.Y., Yuan, Y.: Modified two-point stepsize gradient methods for unconstrained optimization. Comput. Optim. Appl. 22, 103–109 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  7. Davidenko, D.: On a new method of numerically integrating a system of nonlinear equations. Dokl. Akad. Nauk SSSR 88, 601–604 (1953)

    MathSciNet  MATH  Google Scholar 

  8. Engl, H.W.: Discrepancy principles for Tikhonov regularization of ill-posed problems leading to optimal convergence rates. J. Optim. Theory Appl. 52, 209–215 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  9. Engl, H.W.: On the choice of the regularization parameter for iterated Tikhonov regularization of ill-posed problems. J. Approx. Theory 49, 55–63 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  10. Fletcher, R.: On the Barzilai-Borwein gradient method. In: Qi, L., Teo, K., Yang, X. (eds.) Optimization and Control with Applications, pp. 235–256. Springer, New York (2005)

    Chapter  Google Scholar 

  11. Friedlander, A., Martinez, J.M., Molina, B., Raydan, M.: Gradient method with retards and generalizations. SIAM J. Numer. Anal. 36, 275–289 (1999)

    Article  MathSciNet  Google Scholar 

  12. Gfrerer, H.: An a posteriori parameter choice for ordinary and iterated Tikhonov regularization of ill-posed problems leading to optimal convergence rates. Math. Comput. 49, 507–522 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  13. Goloub, B., Heath, M., Wahba, G.: Generalized cross-validation as a method for choosing a good ridge parameter. Technometrics 21, 215–223 (1979)

    Article  MathSciNet  Google Scholar 

  14. Hansen, P.C.: Analysis of discrete ill-posed problems by means of the L-curve. SIAM Rev. 34, 561–580 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  15. Hansen, P.C., O’Leary, D.P.: The use of the L-curve in the regularization of discrete ill-posed problems. SIAM J. Sci. Comput. 14, 1487–1503 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  16. Helmke, U., Moore, J.B.: Optimization and Dynamical Systems. Springer, Berlin (1994)

    Book  Google Scholar 

  17. Hoang, N.S., Ramm, A.G.: Solving ill-conditioned linear algebraic systems by the dynamical systems method (DSM). Inverse Probl. Sci. Eng. 16, 617–630 (2008)

    Article  MathSciNet  Google Scholar 

  18. Hoang, N.S., Ramm, A.G.: Dynamical systems gradient method for solving ill-conditioned linear algebraic systems. Acta Appl. Math. 111, 189–204 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  19. Jacoby, S.L.S., Kowalik, J.S., Pizzo, J.T.: Iterative Methods for Nonlinear Optimization Problems. Prentice-Hall, Englewood Cliffs (1972)

    MATH  Google Scholar 

  20. Kilmer, M.E., O’Leary, D.P.: Choosing regularization parameter in iterative methods for ill-posed problems. SIAM J. Matrix Anal. Appl. 22, 1204–1221 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  21. Ku, C.-Y., Yeih, W., Liu, C.-S.: Solving non-linear algebraic equations by a scalar Newton-homotopy continuation method. Int. J. Nonlinear Sci. Numer. Simul. 11, 435–450 (2010)

    Article  Google Scholar 

  22. Kunisch, K., Zou, J.: Iterative choices of regularization parameters in linear inverse problems. Inverse Probl. 14, 1247–1264 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  23. Landweber, L.: An iteration formula for Fredholm integral equations of the first kind. Am. J. Math. 73, 615–624 (1951)

    Article  MathSciNet  MATH  Google Scholar 

  24. Liu, C.-S.: Intermittent transition to quasiperiodicity demonstrated via a circular differential equation. Int. J. Non-Linear Mech. 35, 931–946 (2000)

    Article  MATH  Google Scholar 

  25. Liu, C.-S.: A study of type I intermittency of a circular differential equation under a discontinuous right-hand side. J. Math. Anal. Appl. 331, 547–566 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  26. Liu, C.-S.: A revision of relaxed steepest descent method from the dynamics on an invariant manifold. Comput. Model. Eng. Sci. 80, 57–86 (2011)

    Google Scholar 

  27. Liu, C.-S.: The concept of best vector used to solve ill-posed linear inverse problems. Comput. Model. Eng. Sci. 83, 499–525 (2012)

    Google Scholar 

  28. Liu, C.-S.: Optimally scaled vector regularization method to solve ill-posed linear problems. Appl. Math. Comput. 218, 10602–10616 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  29. Liu, C.-S.: Modifications of steepest descent method and conjugate gradient method against noise for ill-posed linear systems. Commun. Numer. Anal. 2012, cna-00115 (2012). 24 pages

    Google Scholar 

  30. Liu, C.-S., Atluri, S.N.: A novel time integration method for solving a large system of non-linear algebraic equations. Comput. Model. Eng. Sci. 31, 71–83 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  31. Liu, C.-S., Atluri, S.N.: A fictitious time integration method for the numerical solution of the Fredholm integral equation and for numerical differentiation of noisy data, and its relation to the filter theory. Comput. Model. Eng. Sci. 41, 243–261 (2009)

    MathSciNet  Google Scholar 

  32. Liu, C.-S., Atluri, S.N.: A highly accurate technique for interpolations using very high-order polynomials, and its applications to some ill-posed linear problems. Comput. Model. Eng. Sci. 43, 253–276 (2009)

    MathSciNet  MATH  Google Scholar 

  33. Liu, C.-S., Atluri, S.N.: Simple “residual-norm” based algorithms, for the solution of a large system of non-linear algebraic equations, which converge faster than the Newton’s method. Comput. Model. Eng. Sci. 71, 279–304 (2011)

    MathSciNet  MATH  Google Scholar 

  34. Liu, C.-S., Atluri, S.N.: An iterative algorithm for solving a system of nonlinear algebraic equations, F(x)=0, using the system of ODEs with an optimum α in \(\dot{\mathbf{x}}=\lambda [\alpha \mathbf{F}+(1-\alpha) \mathbf{B}^{\mathrm{T}}\mathbf{F}]\); B ij =∂F i /∂x j . Comput. Model. Eng. Sci. 73, 395–431 (2011)

    MathSciNet  MATH  Google Scholar 

  35. Liu, C.-S., Atluri, S.N.: An iterative method using an optimal descent vector, for solving an ill-conditioned system Bx=b, better and faster than the conjugate gradient method. Comput. Model. Eng. Sci. 80, 275–298 (2011)

    Google Scholar 

  36. Liu, C.-S., Chang, C.W.: Novel methods for solving severely ill-posed linear equations system. J. Mar. Sci. Technol. 17, 216–227 (2009)

    Google Scholar 

  37. Liu, C.-S., Hong, H.K., Atluri, S.N.: Novel algorithms based on the conjugate gradient method for inverting ill-conditioned matrices, and a new regularization method to solve ill-posed linear systems. Comput. Model. Eng. Sci. 60, 279–308 (2010)

    MathSciNet  MATH  Google Scholar 

  38. Liu, C.-S., Kuo, C.-L.: A dynamical Tikhonov regularization method for solving nonlinear ill-posed problems. Comput. Model. Eng. Sci. 76, 109–132 (2011)

    MathSciNet  Google Scholar 

  39. Liu, C.-S., Yeih, W., Atluri, S.N.: On solving the ill-conditioned system Ax=b: general-purpose conditioners obtained from the boundary-collocation solution of the Laplace equation, using Trefftz expansions with multiple length scales. Comput. Model. Eng. Sci. 44, 281–311 (2009)

    MathSciNet  Google Scholar 

  40. Liu, C.-S., Yeih, W., Kuo, C.-L., Atluri, S.N.: A scalar homotopy method for solving an over/under-determined system of non-linear algebraic equations. Comput. Model. Eng. Sci. 53, 47–71 (2009)

    MathSciNet  MATH  Google Scholar 

  41. Lukas, M.A.: Comparison of parameter choice methods for regularization with discrete noisy data. Inverse Probl. 14, 161–184 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  42. Maleknejad, K., Mahmoudi, Y.: Numerical solution of linear integral equation by using hybrid Taylor and block-pulse functions. Appl. Math. Comput. 149, 799–806 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  43. Maleknejad, K., Mollapourasl, R., Nouri, K.: Convergence of numerical solution of the Fredholm integral equation of the first kind with degenerate kernel. Appl. Math. Comput. 181, 1000–1007 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  44. Morozov, V.A.: On regularization of ill-posed problems and selection of regularization parameter. J. Comput. Math. Phys. 6, 170–175 (1966)

    Google Scholar 

  45. Morozov, V.A.: Methods for Solving Incorrectly Posed Problems. Springer, New York (1984)

    Book  Google Scholar 

  46. Ostrowski, A.M.: Solution of Equations in Euclidean and Banach Spaces, 3rd edn. Academic Press, New York (1973)

    MATH  Google Scholar 

  47. Ramm, A.G.: Dynamical System Methods for Solving Operator Equations. Elsevier, Amsterdam (2007)

    Google Scholar 

  48. Ramm, A.G.: Dynamical systems method for solving linear ill-posed problems. Ann. Pol. Math. 95, 253–272 (2009)

    Article  MATH  Google Scholar 

  49. Raydan, M.: On the Barzilai and Borwein choice of steplength for the gradient method. IMA J. Numer. Anal. 13, 321–326 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  50. Raydan, M.: The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem. SIAM J. Optim. 7, 26–33 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  51. Raydan, M., Svaiter, B.F.: Relaxed steepest descent and Cauchy-Barzilai-Borwein method. Comput. Optim. Appl. 21, 155–167 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  52. Resmerita, E.: Regularization of ill-posed problems in Banach spaces: convergence rates. Inverse Probl. 21, 1303–1314 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  53. Stewart, G.: Introduction to Matrix Computations. Academic Press, New York (1973)

    MATH  Google Scholar 

  54. Sweilam, N.H., Nagy, A.M., Alnasr, M.H.: An efficient dynamical systems method for solving singularly perturbed integral equations with noise. Comput. Math. Appl. 58, 1418–1424 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  55. Tikhonov, A.N., Arsenin, V.Y.: Solutions of Ill-posed Problems. Wiley, New York (1977)

    MATH  Google Scholar 

  56. Todd, J.: The condition of finite segments of the Hilbert matrix. In: Taussky, O. (ed.) The Solution of Systems of Linear Equations and the Determination of Eigenvalues. Nat. Bur. of Standards Appl. Math. Series, vol. 39, pp. 109–116 (1954)

    Google Scholar 

  57. Wang, W.: A new mechanical algorithm for solving the second kind of Fredholm integral equation. Appl. Math. Comput. 172, 946–962 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  58. Wang, Y., Xiao, T.: Fast realization algorithms for determining regularization parameters in linear inverse problems. Inverse Probl. 17, 281–291 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  59. Xie, J., Zou, J.: An improved model function method for choosing regularization parameters in linear inverse problems. Inverse Probl. 18, 631–643 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  60. Yuan, Y.: A new stepsize for the steepest descent method. J. Comput. Math. 24, 149–156 (2006)

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

Taiwan’s National Science Council project NSC-100-2221-E-002-165-MY3 and the 2011 Outstanding Research Award, and the anonymous referee comments to improve the quality of this article are highly appreciated.

Author information

Authors and Affiliations

  1. Department of Civil Engineering, National Taiwan University, Taipei, Taiwan

    Chein-Shan Liu

Authors
  1. Chein-Shan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chein-Shan Liu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Liu, CS. A Dynamical Tikhonov Regularization for Solving Ill-posed Linear Algebraic Systems. Acta Appl Math 123, 285–307 (2013). https://doi.org/10.1007/s10440-012-9766-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10440-012-9766-3

Keywords

Mathematics Subject Classification (2010)

Search

Navigation