Efficiently solving total least squares with Tikhonov identical regularization

Article
  • 45 Downloads

Abstract

The Tikhonov identical regularized total least squares (TI) is to deal with the ill-conditioned system of linear equations where the data are contaminated by noise. A standard approach for (TI) is to reformulate it as a problem of finding a zero point of some decreasing concave non-smooth univariate function such that the classical bisection search and Dinkelbach’s method can be applied. In this paper, by exploring the hidden convexity of (TI), we reformulate it as a new problem of finding a zero point of a strictly decreasing, smooth and concave univariate function. This allows us to apply the classical Newton’s method to the reformulated problem, which converges globally to the unique root with an asymptotic quadratic convergence rate. Moreover, in every iteration of Newton’s method, no optimization subproblem such as the extended trust-region subproblem is needed to evaluate the new univariate function value as it has an explicit expression. Promising numerical results based on the new algorithm are reported.

Keywords

Fractional programming Quadratic programming Total least square Tikhonov regularization Bisection method Newton’s method Trust-region subproblem S-lemma 

Mathematics Subject Classification

90C26 90C20 90C32 

References

  1. 1.
    Beck, A., Ben-Tal, A.: On the solution of the Tikhonov regularization of the total least squares problem. SIAM J. Optim. 17(1), 98–118 (2006)MathSciNetCrossRefMATHGoogle Scholar
  2. 2.
    Beck, A., Teboulle, M.: A convex optimization approach for minimizing the ratio of indefinite quadratic functions over an ellipsoid. Math. Program. Ser. A. 118(1), 13–15 (2009)MathSciNetCrossRefMATHGoogle Scholar
  3. 3.
    Beck, A., Ben-Tal, A., Teboulle, M.: Finding a global optimal solution for a quadratically constrained fractional quadratic problem with applications to the regularized total least squares. SIAM J. Matrix Anal. Appl. 28(2), 425–445 (2006)MathSciNetCrossRefMATHGoogle Scholar
  4. 4.
    Charnes, A., Cooper, W.W.: Programming with linear fractional functionals. Nav. Res. Logist. Q. 9, 181–186 (1962)MathSciNetCrossRefMATHGoogle Scholar
  5. 5.
    Dinkelbach, W.: On nonlinear fractional programming. Manag. Sci. 13(7), 492–498 (1967)MathSciNetCrossRefMATHGoogle Scholar
  6. 6.
    Gol’stein, E.G.: Dual problems of convex and fractionally-convex programming in functional spaces. Sov. Math. Dokl. 8, 212–216 (1967)Google Scholar
  7. 7.
    Golub, G.H., Van Loan, C.F.: An analysis of the total least-squares problem. SIAM J. Numer. Anal. 17(6), 883–893 (1980)MathSciNetCrossRefMATHGoogle Scholar
  8. 8.
    Golub, G.H., Van Loan, C.F.: Matrix Computations, 3rd edn. The Johns Hopkins University Press, Baltimore (1996)MATHGoogle Scholar
  9. 9.
    Hansen, P.C.: Regularization tools: a Matlab package for analysis and solution of discrete ill-posed problems. Numer. Algorithm 6(1), 1–35 (1994)MathSciNetCrossRefMATHGoogle Scholar
  10. 10.
    Hansen, P.C., O’Leary, D.P.: The use of the L-curve in the regularization of discrete ill-posed problems. SIAM J. Sci. Comput. 14(6), 1487–1503 (1993)MathSciNetCrossRefMATHGoogle Scholar
  11. 11.
    Hebden, M.D.: An algorithm for minimization using exact second derivatives. Int. J. Distrib. Sens. Netw. 2014(3), 1–8 (1973)Google Scholar
  12. 12.
    Jagannathan, R.: On some properities of programming problems in parametric form pertaining to fractional programming. Manag. Sci. 12, 609–615 (1966)CrossRefMATHGoogle Scholar
  13. 13.
    Jain, A.K.: Fundamentals of Digital Image Processing. Prentice-Hall, Englewood Cliffs (1989)MATHGoogle Scholar
  14. 14.
    Lampe, J., Voss, H.: Large-scale Tikhonov regularization of total least squares. J. Comput. Appl. Math. 238, 95–108 (2013)MathSciNetCrossRefMATHGoogle Scholar
  15. 15.
    Moosaei, H., Ketabchi, S., Pardalos, P.M.: Tikhonov regularization for infeasible absolute value equations. Optimization 65(8), 1531–1537 (2016)MathSciNetCrossRefMATHGoogle Scholar
  16. 16.
    Moré, J.J.: Generalization of the trust region problem. Optim. Methods Softw. 2(3), 189–209 (1993)CrossRefGoogle Scholar
  17. 17.
    Renaut, R.A., Guo, H.: Efficient algorithms for solution of regularized total least squares. SIAM J. Matrix Anal. Appl. 26(2), 457–476 (2005)MathSciNetCrossRefMATHGoogle Scholar
  18. 18.
    Schaible, S.: Fractional programming II: on Dinkelbach’s algorithm. Manag. Sci. 22(8), 868–873 (1976)MathSciNetCrossRefMATHGoogle Scholar
  19. 19.
    Sima, D., Van Huffel, S., Golub, G.H.: Regularized total least squares based on quadratic eigenvalue problem solvers. BIT 44, 793–812 (2004)MathSciNetCrossRefMATHGoogle Scholar
  20. 20.
    Tikhonov, A.N., Arsenin, V.Y.: Solution of Ill-Posed Problems. V.H. Winston, Washington (1977)MATHGoogle Scholar
  21. 21.
    Van Huffel, S., Lemmerling, P.: Total Least Squares and Errors-in-Variables Modeling. Kluwer Academic Publishers, Dordrecht (2002)CrossRefMATHGoogle Scholar
  22. 22.
    Van Huffel, S., Vandewalle, J.: The Total Least Squares Problem: Computational Aspects and Analysis. Frontiers in Applied Mathematics. SIAM, Philadelphia (1991)CrossRefMATHGoogle Scholar
  23. 23.
    Xia, Y., Wang, S., Sheu, R.L.: S-Lemma with equality and its applications. Math. Program. Ser. A. 156(1), 513–547 (2016)MathSciNetCrossRefMATHGoogle Scholar
  24. 24.
    Zhang, A., Hayashi, S.: Celis–Dennis–Tapia based approach to quadratic fractional programming problems with two quadratic constraints. Numer. Algebra Control Optim. 1(1), 83–98 (2011)MathSciNetCrossRefMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.LMIB of the Ministry of Education, School of Mathematics and System SciencesBeihang UniversityBeijingPeople’s Republic of China
  2. 2.Department of Industrial EngineeringUniversity of HoustonHoustonUSA

Personalised recommendations