Skip to main content
SpringerLink
Log in

A Gauss–Newton method for mixed least squares-total least squares problems

  • Published:
Calcolo Aims and scope Submit manuscript

Abstract

The approximate linear equation \(Ax\approx b\) with some columns of A error-free can be solved via mixed least squares-total least squares (MTLS) model by minimizing a nonlinear function. This paper is devoted to the Gauss–Newton iteration for the MTLS problem. With an appropriately chosen initial vector, each iteration step of the standard Gauss–Newton method requires to solve a smaller-size least squares problem, in which the QR of the coefficient matrix needs a rank-one modification. To improve the convergence, we devise a relaxed Gauss–Newton (RGN) method by introducing a relaxation factor and provide the convergence results as well. The convergence is shown to be closely related to the ratio of the square of subspace-restricted singular values of [A, b]. The RGN can also be modified to solve the total least squares (TLS) problem. Applying the RGN method to a Bursa–Wolf model in parameter estimation, numerical results show that the RGN-based MTLS method behaves much better than the RGN-based TLS method. Theoretical convergence properties of the RGN-MTLS algorithm are also illustrated by numerical tests.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Algorithm 1
Algorithm 2
Algorithm 3
Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Data Availability

All data generated or analysed during this study are included in this manuscript.

References

  1. Arun, K.S., Huang, T.S., Blostein, S.D.: Least-squares fitting of two 3-D point sets. IEEE Trans. Pattern Anal. Mach. Intell. PAMI 9(5), 698–700 (1987)

    Article  CAS  Google Scholar 

  2. Bai, Z.Z., Wu, W.T., Muratova, G.V.: The power method and beyond. Appl. Numer. Math. 164, 29–42 (2021)

    Article  MathSciNet  Google Scholar 

  3. Björck, Å.: Numerical Methods for Least Squares Problems. SIAM, Philadelphia (1996)

    Book  Google Scholar 

  4. Chen, J.H., Li, W.G.: Convergence of Gauss–Newton’s method and uniqueness of the solution. Appl. Math. Comput. 170, 686–705 (2005)

    MathSciNet  Google Scholar 

  5. Daniel, J.W., Gragg, W.B., Kaufman, L., Stewart, G.W.: Reorthogonalization and stable algorithms for updating the Gram–Schmidt QR factorization. Math. Comput. 30, 772–795 (1976)

    MathSciNet  Google Scholar 

  6. Fasino, D., Fazzi, A.: A Gauss–Newton iteration for total least squares problems. BIT Numer. Math. 58, 281–299 (2018)

    Article  MathSciNet  Google Scholar 

  7. Golub, G.H., Van Loan, C.F.: An analysis of the total least squares problem. SIAM J. Numer. Anal. 17(6), 883–893 (1980)

    Article  MathSciNet  Google Scholar 

  8. Golub, G.H., Van Loan, C.F.: Matrix Computations, 4th edn. Johns Hopkins University Press, Baltimore (2013)

    Book  Google Scholar 

  9. Golub, G.H., Hoffman, A., Stewart, G.W.: A generalization of the Eckart–Young–Mirsky matrix approximation theorem. Linear Algebra Appl. 88(89), 322–327 (1987)

    MathSciNet  Google Scholar 

  10. Han, F., Wei, Y.: TLS-EM algorithm of mixture density models for exponential families. J. Comput. Appl. Math. 403, 113829 (2022)

    Article  MathSciNet  Google Scholar 

  11. Hochstenbach, M.E., Reichel, L.: Subspace-restricted singular value decompositions for linear discrete ill-posed problems. J. Comput. Appl. Math. 235, 1053–1064 (2010)

    Article  MathSciNet  Google Scholar 

  12. Li, C., Zhang, W.H., Jin, X.Q.: Convergence and uniqueness properties of Gauss–Newton’s method. Comput. Math. Appl. 47, 1057–1067 (2004)

    Article  MathSciNet  Google Scholar 

  13. Liu, Q.H., Wang, M.H.: On the weighting method for mixed least squares-total least squares problems. Numer. Linear Algebra Appl. 24(5), e2094 (2017)

    Article  MathSciNet  Google Scholar 

  14. Liu, Q.H., Li, C.G., Wei, Y.M.: Condition numbers of multidimensional mixed least squares-total least squares problems. Appl. Numer. Math. 178, 52–68 (2022)

    Article  MathSciNet  Google Scholar 

  15. Nocedal, J., Wright, S.J.: Numerical Optimization. Springer, Berlin (1999)

    Book  Google Scholar 

  16. Paige, C.C., Wei, M.S.: Analysis of the generalized total least squares problem \(AX \approx B\) when some columns are free of errors. Numer. Math. 65, 177–202 (1993)

    Article  MathSciNet  Google Scholar 

  17. Stoica, P., S\(\ddot{\rm o}\)derstr\(\ddot{\rm o}\)m, T.: Bias correction in least squares identification. Int. J. Control 35, 449–457 (1982)

  18. Van Huffel, S., Vandewalle, J.: Analysis and properties of the generalized total least squares problem \(AX \approx B\) when some or all columns in \(A\) are subject to error. SIAM J. Matrix Anal. Appl. 10, 294–315 (1989)

    Article  MathSciNet  Google Scholar 

  19. Van Huffel, S., Vandewalle, J.: Comparison of total least squares and instrumental variable methods for parameter estimation of transfer function models. Int. J. Control 50, 1039–1056 (1989)

    Article  Google Scholar 

  20. Van Huffel, S., Vandewalle, J.: The Total Least-Squares Problems: Computational Aspects and Analysis. Frontiers in Applied Mathematics, vol. 9. SIAM, Philadelphia (1991)

    Book  Google Scholar 

  21. Wolf, P.R., Ghilani, C.D.: Adjustment Computations: Statistics and Least Squares in Surveying and GIS. Wiley, New York (1997)

    Google Scholar 

  22. Wu, Y., Li, M.F., Wang, C., Song, W.T.: Application of scaled total least squares in coordinate transformation. Geomat. Spatial Inf. Technol. 38, 45–47 (2015). ((in Chinese))

    Google Scholar 

  23. Young, P.: The instrumental variable method: a practical approach to identification and parameter estimation. IFAC Proc. 18(5), 1–15 (1985)

    Google Scholar 

  24. Zare, H., Hajarian, M.: An efficient Gauss–Newton algorithm for solving regularized total least squares problems. Numer. Algorithms 89, 1049–1073 (2022)

    Article  MathSciNet  Google Scholar 

  25. Zhang, P.P., Wang, Q.: Perturbation analysis and condition numbers of mixed least squares-scaled total least squares problem. Numer. Algorithms 89, 1223–1246 (2022)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

The authors are grateful to the handling editor and the anonymous referees for their useful comments and suggestions, which greatly improved the original presentation.

Author information

Authors and Affiliations

  1. Department of Mathematics, Shanghai University and Newtouch Center for Mathematics of Shanghai University, Shanghai, 200444, People’s Republic of China

    Qiaohua Liu

  2. School of Mathematical Sciences and Key Laboratory of Mathematics for Nonlinear Sciences, Fudan University, Shanghai, 200433, People’s Republic of China

    Shan Wang & Yimin Wei

Authors
  1. Qiaohua Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yimin Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yimin Wei.

Ethics declarations

Conflict of interest Statement

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Q. Liu is supported by Natural Science Foundation of Shanghai under Grant 23ZR1422400 and Large-scale Numerical Simulation Computing Sharing Platform of Shanghai University. Y. Wei is supported by the Science and Technology Commission of Shanghai Municipality under Grant 23JC1400501 and the Ministry of Science and Technology of China under Grant G2023132005L.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, Q., Wang, S. & Wei, Y. A Gauss–Newton method for mixed least squares-total least squares problems. Calcolo 61, 18 (2024). https://doi.org/10.1007/s10092-024-00568-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10092-024-00568-2

Keywords

Search

Navigation