Numerical Algorithms

, Volume 72, Issue 1, pp 211–241 | Cite as

A new error in variables model for solving positive definite linear system using orthogonal matrix decompositions

Original Paper
First Online:
Received:
Accepted:
  • 110 Downloads

Abstract

The need to estimate a positive definite solution to an overdetermined linear system of equations with multiple right hand side vectors arises in several process control contexts. The coefficient and the right hand side matrices are respectively named data and target matrices. A number of optimization methods were proposed for solving such problems, in which the data matrix is unrealistically assumed to be error free. Here, considering error in measured data and target matrices, we present an approach to solve a positive definite constrained linear system of equations based on the use of a newly defined error function. To minimize the defined error function, we derive necessary and sufficient optimality conditions and outline a direct algorithm to compute the solution. We provide a comparison of our proposed approach and two existing methods, the interior point method and a method based on quadratic programming. Two important characteristics of our proposed method as compared to the existing methods are computing the solution directly and considering error both in data and target matrices. Moreover, numerical test results show that the new approach leads to smaller standard deviations of error entries and smaller effective rank as desired by control problems. Furthermore, in a comparative study, using the Dolan-Moré performance profiles, we show the approach to be more efficient.

Keywords

Error in variables models Positive definiteness constraints Overdetermined linear system of equations Multiple right hand side vectors 

Mathematics Subject Classification (2010)

65F05 65F20 49M05 
This is a preview of subscription content, log in to check access.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Alizadeh, F., Pierre, J., Heaberly, A., Overton, M.L.: Primal-dual interior point methods for semidefinite programming: convergence rates, stability and numerical result. SIAM J. Optim. 8, 746–768 (1998)MathSciNetCrossRefMATHGoogle Scholar
  2. 2.
    Aubry, A., Maio, A.D., Pallotta, L., Farina, A.: Maximum likelihood estimation of a structured covariance matrix with a condition number constraint. IEEE Trans. On Signal Processing 60(6), 3004–3021 (2012)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Cheng, C.L., Kukush, A., Mastronardi, N., Paige, C., Van Huffel, S.: Total least squares and errors-in-variables modeling. Comput. Stat. Data Anal. 52, 1076–1079 (2007)MathSciNetCrossRefMATHGoogle Scholar
  4. 4.
    Deng, Y., Boley, D.: On the optimal approximation for the symmetric procrustes problems of the matrix equation AXB = C Proceedings of the International Conference on Computational and Mathematical Methods in Science and Engineering, Chicago, pp 159–168 (2007)Google Scholar
  5. 5.
    Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2012)MathSciNetCrossRefMATHGoogle Scholar
  6. 6.
    Golub, G.H., Van Loan, C.F.: An analysis of the total least squares problem. SIAM J. Numer. Anal. 17, 883–893 (1980)MathSciNetCrossRefMATHGoogle Scholar
  7. 7.
    Hayami, K., Yin, J.F., Ito, T.: GMRES method for least squares problems. SIAM J. Matrix Anal. Appl. 31(5), 2400–2430 (2010)MathSciNetCrossRefMATHGoogle Scholar
  8. 8.
    Hnetynková, I., Plesinger, M., Sima, D.M., Strakos, Z., Van Huffel, S.: The total least squares problem in A X˜B, A new classification with the relationship to the classical works. SIAM J. Matrix Anal. Appl. 32(3), 748–770 (2011)MathSciNetCrossRefMATHGoogle Scholar
  9. 9.
    Hu, H., Olkin, I.: A numerical procedure for finding the positive definite matrix closest to a patterned matrix. Stat. Probabil. Lett. 12, 511–515 (1991)MathSciNetCrossRefMATHGoogle Scholar
  10. 10.
    Hu, H.: Positive definite constrained least-squares estimation of matrices. Linear Algebra Appl. 229, 167–174 (1995)MathSciNetCrossRefMATHGoogle Scholar
  11. 11.
    Van Huffel, S., Vandewalle, J.: Algebraic connections between the least squares and total least squares problems. Numer. Math. 55, 431–449 (1989)MathSciNetCrossRefMATHGoogle Scholar
  12. 12.
    Kang, B., Jung, S., Park, P.: A new iterative method for solving total least squares problem. Proceeding of the 8th Asian Control Conference (ASCC). Kaohsiung (2011)Google Scholar
  13. 13.
    Larson, H.J.: Least squares estimation of the components of a symmetric matrix. Technometrics 8(2), 360–362 (1966)MathSciNetCrossRefMATHGoogle Scholar
  14. 14.
    McInroy, J., Hamann, J.C.: Design and control of flexure jointed hexapods. IEEE Trans. Robot. Autom. 16(4), 372–381 (2000)CrossRefGoogle Scholar
  15. 15.
    Moré, J.J., Wild, S.M.: Benchmarking derivative-free optimization algorithms. SIAM J. Optim. 20, 172–191 (2009)MathSciNetCrossRefMATHGoogle Scholar
  16. 16.
    Paige, C. C., Strakos, Z.: Scaled total least squares fundamentals. Numer. Math. 91, 117–146 (2000)MathSciNetCrossRefMATHGoogle Scholar
  17. 17.
    Poignet, P., Gautier, M.: Comparison of weighted least squares and extended kalman filtering methods for dynamic identification of robots Proceedings of the IEEE Conference on Robotics and Automation, San Francisco, pp 3622–3627 (2000)Google Scholar
  18. 18.
    Woodgate, K.G.: Least-squares solution of F=P G over positive semidefinite symmetric P. Linear Algebra Appl. 245, 171–190 (1996)MathSciNetCrossRefMATHGoogle Scholar
  19. 19.
    Zhou, L., Lin, L., Wei, Y., Qiao, S.: Perturbation analysis and condition numbers of scaled total least squares problems. Numer. Algorithms 51, 381–399 (2009)MathSciNetCrossRefMATHGoogle Scholar
  20. 20.
    Banerjee, S., Roy, A.: Quadratic Forms, Linear Algebra and Matrix Analysis for Statistics. Chapman & Hall/CRC Texts in Statistical Sciences, pp 441–442 (2014)Google Scholar
  21. 21.
    Carlen, E.: Trace inequalities and quantum entropy: an introductory course. Contemporary Math. AMS 529, 73–140 (2009)Google Scholar
  22. 22.
    Gill, P.E., Murray, W., Wright, M.H.: Numerical Linear Algebra and Optimization. Addison Wesley (1991)Google Scholar
  23. 23.
    Higham, N.J.: Functions of Matrices: Theory and Computation. SIAM, Philadelphia (2008)CrossRefMATHGoogle Scholar
  24. 24.
    Horn, R.A., Johnson, C.R.: Topics in Matrix Analysis. Cambridge University Press (1991)Google Scholar
  25. 25.
    Van Huffel, S., Vandewalle, J.: The Total Least Squares Problem: Computational Aspects and Analysis. SIAM, Philadelphia (1991)Google Scholar
  26. 26.
    Krislock, N.G.: Numerical Solution of Semidefinite Constrained Least Squares Problems, M. Sc. Thesis, University of British Colombia (2003)Google Scholar
  27. 27.
    Demmel, J.W.: Applied Numerical Linear Algebra, 3rd edn. SIAM, Philadelphia (1996)MATHGoogle Scholar
  28. 28.
    Golub, G.H., Van Loan, C.F.: Matrix Computation, 4th edn. JHU Press (2012)Google Scholar
  29. 29.
    Lancaster, P., Rodman, L.: Algebraic Riccati Equations. Clarendon Press (1995)Google Scholar
  30. 30.
    Magnus, J.R., Neudecker, H.: Matrix Differential Calculus with Applications in Statistics and Econometrics, 2nd edn. Wiley (1999)Google Scholar
  31. 31.
    Nocedal, J., Wright, S.J.: Numerical Optimization. Springer, New York (1999)CrossRefMATHGoogle Scholar
  32. 32.
    Higham N.J. Computing the nearest correlation matrix (A problem from nance), MIMS EPrint: 2006.70. http://eprints.ma.man.ac.uk/ (2006). Accessed 26 June 2012
  33. 33.
    Petersen, K.B., Pedersen, M.S. The matrix cookbook. http://orion.uwaterloo.ca/hwolkowi/matrixcookbook.pdf (2008). Accessed 11 January 2013
  34. 34.
    Vershynin, R. Introduction to the non-asymptotic analysis of random matrices. http://arxiv.org/pdf/1011.3027v7.pdf (2011). Accessed 01 February 2013
  35. 35.
    American Mathematical Society, Eigenvalues and sums of Hermitian matrices. http://www.ams.org/bookstore/pspdf/gsm-132-prev.pdf (2009). Accessed 18 March 2013

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  1. 1.Faculty of Mathematical SciencesSharif University of TechnologyTehranIran

Personalised recommendations