# Parameter Estimation, Variance Components and Statistical Analysis in Errors-in-Variables Models

• Peiliang Xu
Living reference work entry
Part of the Springer Reference Naturwissenschaften book series (SRN)

## Zusammenfassung

Der Beitrag diskutiert statistische und numerische Aspekte von (,,errors-in-variable“) EIV-Modellen mit oder ohne Nebenbedingungen. Die Parameter in einem EIV-Modell lassen sich oftmals unter Benutzung dreier methodischer Kategorien bewerten: konventionelle gewichtete LS-Methode, normierte orthogonale Regression und gewichtete TLS-Methode. Die vorliegende Arbeit beschftigt sich mit der Reformulierung eines EIV-Models als nichtlineares Ausgleichungsmodell ohne Nebenbedingungen.

## Keywords

Errors-in-variables Estimability of variance components Nonlinear adjustment Orthogonal regression Total least squares Variance components Weighted least squares

## Abstract

This chapter discusses statistical and numerical aspects of constrained and unconstrained errors-in-variables (EIV) models. The parameters in an EIV model can often be estimated by using three categories of methods: the conventional weighted least squares (LS) method, normed orthogonal regression, and the weighted total least squares (TLS) method. The conventional weighted LS method is of significantly computational advantage but not rigorous statistically. We systematically investigate the effects of random errors in the design matrix on the weighted LS estimates of parameters and variance components, construct the N-calibrated almost unbiased weighted LS estimator of parameters and derive almost unbiased estimates for the variance of unit weight. Although orthogonal regression can be used to estimate the parameters in an EIV model, it is not statistically optimal either. The weighted TLS method is most rigorous and optimal to statistically estimate the parameters in an EIV model at the cost of substantially increasing computation. We reformulate an EIV model as a nonlinear adjustment model without constraints and investigate the statistical effects of nonlinearity on the nonlinear TLS estimate, including the first order approximation of accuracy, nonlinear confidence region and bias of the nonlinear TLS estimate. Closed form solutions to coordinate transformation have been presented as well. Finally, we prove that variance components in an EIV model with the simplest stochastic structure are not estimable.

## References

1. 1.
Adcock, R.J.: Note on the method of least squares. Analyst 4, 183–184 (1877)
2. 2.
Adcock, R.J.: A problem in least squares. Analyst 5, 53–54 (1878)
3. 3.
Akyilmaz, O.: Total least squares solution of coordinate transformation. Surv. Rev. 39, 68–80 (2007)
4. 4.
Altmann, S.L.: Rotations, Quaternions, and Double Groups. Clarendon Press, Oxford (1982)Google Scholar
5. 5.
Amiri-Simkooei, A., Jazaeri, S.: Data-snooping procedure applied to errors-in-variables models. Stud. Geophys. Geod. 57, 426–441 (2013)
6. 6.
Amiri-Simkooei, A.R., Zangeneh-Nejad, F., Asgari, J.: On the covariance matrix of weighted total least-squares estimates. J. Surv. Eng. 142, 04015014 (2016)
7. 7.
Aoki, M., Yue, P.C.: On a priori error estimates of some identification methods. IEEE Trans. Auto. Contr. AC-15, 541–548 (1970)
8. 8.
Arun, K.S., Huang, T.S., BLOSTEIN, S.D.: Least-squares fitting of two 3-D point sets. IEEE Trans. Pattern Anal. Mach. Intell. PAMI-9, 698–700 (1987)
9. 9.
Bab-Hadiashar, A., Suter, D.: Robust total least squares based optic flow computation. Int. J. Comput. Vis. 29, 566–573 (1998)
10. 10.
Beale, E.M.L.: Confidence regions in non-linear estimation (with discussions). J. R. Stat. Soc. B22, 41–88 (1960)Google Scholar
11. 11.
Ben-Israel, A., Greville, T.: Generalized Inverses: Theory and Applications. Springer, Berlin (2000)Google Scholar
12. 12.
Boor, B.D.: Structured total least squares and L2 approximation problems. Lin. Algebra. Appl. 188, 163–205 (1993)Google Scholar
13. 13.
Bormann, P., Wylegalla, K.: Untersuchung der Korrelationsbeziehungen zwischen verschiedenen Arten der Magnitudenbestimmung der Station Moxa in Abhangigkeit vom Gerraetetyp und vom Herdgebiet. Publications of Institute of Geophysics, Pol. Acad. Sci. 93, 159–175 (1975)Google Scholar
14. 14.
Box, M.J.: Bias in nonlinear estimation (with discussions). J. R. Stat. Soc. B33, 171–201 (1971)Google Scholar
15. 15.
Branham R.L. Jr.: Astronomical data reduction with total least squares. New Astron. Rev. 45, 649–661 (2001)
16. 16.
Cai, J.Q., Grafarend, E.: Systematical analysis of the transformation between Gauss-Krueger-coordinate/DHDN and UTM-coordinate/ETRS89 in Baden-Württemberg with different estimation methods. In: Drewes, H. (ed.) Geodetic Reference Frames, International Association of Geodesy Symposia 134, pp. 205–211. Springer, Berlin (2009)Google Scholar
17. 17.
Carroll, R.J., Ruppert, D., Stefanski, L.A., Crainiceanu, C.M.: Measurement Error in Nonlinear Models – a Modern Perspective, 2nd edn. Chapman and Hall, London (2006)
18. 18.
Castellaro, S., Mulargia, F., Kagan, Y.Y.: Seismology regression problems for magnitudes. Geophys. J. Int. 165, 913–930 (2006)
19. 19.
Chang, G.B., Xu, T.H., Wang, Q.X.: M-estimator for the 3D symmetric Helmert coordinate transformation. J. Geod. 92, 47–58 (2018)
20. 20.
Cheng, C., van Ness, J.W.: Generalized M-estimators for errors-in-variables regression. Ann. Stat. 20, 385–397 (1992)
21. 21.
Coolidge, J.L.: Two geometrical applications of the method of least squares. Am. Math. Mon. 20, 187–190 (1913)
22. 22.
Davies, R.B., Hutton, B.: The effect of errors in the independent variables in linear regression. Biometrika 62, 383–391 (1975)
23. 23.
Deming, W.E.: The application of least squares. Philos. Mag. 11, 146–158 (1931)
24. 24.
Deming, W.E.: On the application of least squares — II. Philos. Mag. 17, 804–829 (1934)
25. 25.
Deming, W.E.: Statistical Adjustment of Data. Dover Publications, New York (1964)Google Scholar
26. 26.
Dennis, Jr., Schnabel, R.B.: Numerical Methods for Unconstrained Optimization and Nonlinear Equations. SIAM Classics in Applied Mathematics. SIAM, Philadelphia (1996)Google Scholar
27. 27.
Dermanis, A.: The photogrammetric inner constraints. ISPRS J. Photogramm. Remote Sens. 49, 25–39 (1994)
28. 28.
Dermanis, A.: Generalized inverses of nonlinear mappings and the nonlinear geodetic datum problem. J. Geod. 72, 71–100 (1998)
29. 29.
Fang, X.: Weighted total least squares solutions for applications in geodesy. Wissenschaftliche Arbeiten der Fachrichtung Geodäsie und Geoinformatik, PhD Dissertation Nr.294, Leibniz University Hannover (2011)Google Scholar
30. 30.
Felus, Y.A.: Application of total least squares for spatial point process analysis. J. Surv. Eng. 130, 126–133 (2004)
31. 31.
Felus, Y.A., Burtch, R.C.: On symmetrical three-dimensional datum conversion. GPS Solut. 13, 65–74 (2009)
32. 32.
Fuller, W.A.: Measurement Error Models. Wiley Interscience, New York (1987)
33. 33.
Fuller, W.A.: Introduction to Statistical Time Series, 2nd edn. Wiley, New York (1996)Google Scholar
34. 34.
Gerhold, G.A.: Least-squares adjustment of weighted data to a general linear equation. Am. J. Phys. 37, 156–161 (1969)
35. 35.
Gleser, L.: Estimation in a multivariate “errors in variables” regression model: large sample results. Ann. Stat. 9, 24–44 (1981)
36. 36.
Golub, G.H., van Loan, C.F.: An analysis of the total least squares problem. SIAM J. Numer. Anal. 17, 883–893 (1980)
37. 37.
Grafarend, E., Schaffrin, B.: Unbiased free net adjustment. Surv. Rev. 22, 200–218 (1974)
38. 38.
Grafarend, E., Schaffrin, B.: Equivalence of estimable quantities and invariants in geodetic networks. ZfV 101, 485–491 (1976)Google Scholar
39. 39.
Hodges, S.D., Moore, P.G.: Data uncertainties and least squares regression. Appl. Stat. 21, 185–195 (1972)
40. 40.
Horn, B.: Closed-form solution of absolute orientation using unit quaternions. J. Opt. Soc. Am. 4, 629–642 (1987)
41. 41.
Horn, B., Hilden, H.M., Negahdripour, S.: Closed-form solution of absolute orientation using orthonormal matrices. J. Opt. Soc. Am. 5, 1127–1135 (1988)
42. 42.
Horn, S.D., Horn, R.A., Duncan, D.B.: Estimating heteroscedastic variances in linear models. J. Am. Stat. Ass. 70, 380–385 (1975)
43. 43.
Kanatani, K., Niitsuma, H.: Optimal computation of 3-D similarity: Gauss-Newton vs. Gauss-Helmert. Comput. Stat. Data Anal. 56, 4470–4483 (2012)
44. 44.
Keat, J.E.: Analysis of least-squares attitude determination routine DOAOP. Technical Report CSC/TM-77/6034, Computer Sciences Corp. (1977)Google Scholar
45. 45.
Koch, K.R.: S-transformations and projections for obtaining estimable parameters. In: 40 Years of Thought, Anniversary Volume on the Occasion of Prof. Baarda’s 65th Birthday, Delft, Vol. 1, pp. 136–144 (1982)Google Scholar
46. 46.
Koch, K.R.: Parameter estimation and hypothesis testing in linear models, 2nd edn. Springer, Berlin (1999)
47. 47.
Kotsakis, C.: Reference frame stability and nonlinear distortion in minimum-constrained network adjustment. J. Geod. 86, 755–774 (2012)
48. 48.
Kotsakis, C.: Generalized inner constraints for geodetic network densification problems. J. Geod. 87, 661–673 (2013)
49. 49.
Krakiwsky, E.D., Thomson, D.B.: Mathematical models for the combination of terrestrial and satellite networks. Can. Surv. 28, 606–615 (1974)Google Scholar
50. 50.
Kummell, C.H.: Reduction of observation equations which contain more than one observed quantity. Analyst 6, 97–105 (1879)
51. 51.
Lemmerling, P., van Huffel, S.: Structured total least squares. In: van Huffel, S., Lemmerling, P. (eds.) Total Least Squares and Errors-in-Variables Modelling, pp. 79–91. Kluwer Academic Publishers, Dordrecht (2002)
52. 52.
Li, Y., Tang, H., Lin, X.: Spatial linear mixed models with covariate measurement errors. Stat. Sin. 19, 1077–1093 (2009)Google Scholar
53. 53.
Liu, J.N.: The equivalence of coordinate transformation models for the combination of satellite and terrestrial networks. J. Wuhan Tech. Univ. Surv. Mapp. 8, 37–50 (1983). (in Chinese with English abstract)Google Scholar
54. 54.
Liu, J.N., Liu, D.J.: The influence of the accuracy in geodetic and geocentric coordinates on combined adjustment. Acta Geod. Cartogr. Sinica. 14, 133–144 (1985). (in Chinese with English abstract)Google Scholar
55. 55.
Liu, J.N., Liu, D.J., Cui, X.Z.: Theory and applications of combined adjustment of satellite and terrestrial networks. J. Wuhan Tech. Univ. Surv. Mapp. 12(4), 1–9 (1987). (in Chinese with English abstract)Google Scholar
56. 56.
Lu, G.: Development of a GPS multi-antenna system for attitude determination. PhD Thesis, UCGE Reports No. 20073. Department of Geomatics Engineering, The University of Calgary (1995)Google Scholar
57. 57.
Magnus, J.R., Neudecker, H.: Matrix differential calculus with applications in statistics and econometrics. Wiley, New York (1988)
58. 58.
Markovsky, I., van Huffel, S.: High-performance numerical algorithms and software for structured total least squares. J. Comput. Appl. Math. 180, 311–331 (2005)
59. 59.
Markovsky, I., van Huffel, S.: On weighted structured total least squares. In: Lirkov, I., Margenov, S., Waśniewski, J. (eds.) LSSC 2005, LNCS 3743, pp. 695–702 (2006)Google Scholar
60. 60.
Markovsky, I., van Huffel, S.: Overview of total least squares methods. Sig. Proc. 87, 2283–2302 (2007)
61. 61.
Mastronardi, N., Lemmerling, M., van Huffel, S.: Fast structured total least squares algorithm for solving the basic deconvolution problem. SIAM J. Matrix Anal. Appl. 22, 533–553 (2000)
62. 62.
Meissl, P.: Die innere Genauigkeit eines Punkthaufens. Öster. ZfV 50, 159–165, 186–194 (1962)Google Scholar
63. 63.
Meissl, P.: Über die innere Genauigkeit dreidimensionaler Punkthaufen. ZfV 90, 109–118 (1965)Google Scholar
64. 64.
Neitzel, F.: Generalization of total least-squares on example of unweighted and weighted 2D similarity transformation. J. Geod. 84, 751–762 (2010)
65. 65.
Nievergelt, Y.: Total least squares: state-of-the-art regression in numerical analysis. SIAM Rev. 36, 258–264 (1994)
66. 66.
Papo, H., Perelmuter, A.: Datum definition by free net adjustment. Bull. Geod. 55, 218–226 (1981)
67. 67.
Pearson, K.: On lines and planes of closest fit to systems of points in space. Philos. Mag. 2, 559–572 (1901)
68. 68.
Rao, C.R., Kleffe, J.: Estimation of Variance Components and Applications. North-Holland, Amsterdam (1988)Google Scholar
69. 69.
Rao, C.R., Mitra, S.K.: Generalized Inverse of Matrices and Its Applications. Wiley, New York (1971)Google Scholar
70. 70.
Ratkowsky, D.A.: Nonlinear Regression Modeling – A Practical Unified Approach. Marcel Dekker, New York (1983)Google Scholar
71. 71.
Rinner, K.: Systematic investigations of geodetic networks in space. European Res Office, Contract No. 91–591-EUC 3584 (1966)Google Scholar
72. 72.
Rosen, J.B., Park, H., Glick, J.: Total least squares formulation and solution for structured problems. SIAM J. Matrix Anal. Appl. 17, 110–126 (1996)
73. 73.
Ross, G.J.S.: Nonlinear Estimation. Springer, New York (1990)
74. 74.
Sanso, F.: An exact solution of the roto-translation problem. Photogrammetria 29, 203–216 (1973)
75. 75.
Schaffrin, B.: Aspects of network design. In: Grafarend, E., Sanso, F. (eds.) Optimization and Design of Geodetic Networks, pp. 548–597. Springer, Berlin (1985)
76. 76.
Schaffrin, B.: A note on constrained total least-squares estimation. Linear Algebra Appl. 417, 245–258 (2006)
77. 77.
Schaffrin, B., Wieser, A.: On weighted total least-squares adjustment for linear regression. J. Geod. 82, 415–421 (2008)
78. 78.
Schaffrin, B., Felus, Y.A.: An algorithmic approach to the total least-squares problem with linear and quadratic constraints. Stud. Geophys. Geod. 53, 1–16 (2009)
79. 79.
Schaffrin, B., Lee, I., Choi, Y., Felus, Y.A.: Total least-squares (TLS) for geodetic straight-line and plane adjustment. Boll. Geod. Sci. Aff. LXV, 141–168 (2006)Google Scholar
80. 80.
Schönemann, P.H.: A generalized solution of the orthogonal procrustes problem. Psychometrika 31, 1–10 (1966)
81. 81.
Searle, S.R.: Linear Models. Wiley, New York (1971)Google Scholar
82. 82.
Seber, G., Wild, C.: Nonlinear Regression. Wiley, New York (1989)
83. 83.
Shen, Y.Z., Li, B.F., Chen, Y.: An iterative solution of weighted total least squares adjustment. J. Geod. 85, 229–238 (2011)
84. 84.
Shi, Y., Xu, P.L.: Unidentifiability of errors-in-variables models with rank deficiency from measurements (2018, submitted)Google Scholar
85. 85.
Shi, Y., Xu, P.L., Liu, J.N., Shi, C.: Alternative formulae for parameter estimation in partial errors-in-variables models. J. Geod. 89, 13–16 (2015)
86. 86.
Teunissen, P.: Zero order design: generalized inverse, adjustment, the datum problem and S-transformations. In: Grafarend, E., Sanso, F. (eds.) Optimization and Design of Geodetic Networks, pp. 11–55. Springer, Berlin (1985)
87. 87.
Thompson, E.H.: An exact linear solution of the problem of absolute orientation. Photogrammetria 15, 163–179 (1959)
88. 88.
Tong, X.H., Jin, Y.M., Zhang, S.L., Li, L.Y., Liu, S.J.: Bias-corrected weighted total least-squares adjustment of condition equations. J. Surv. Eng. 141, 04014013 (2015)
89. 89.
Umeyama, S.: Least squares estimation of transformation parameters between two point patterns. IEEE Trans. Pattern Anal. Mach. Intell. 13, 376–380 (1991)
90. 90.
Ursin, B.: Methods for estimating the seismic reflection response. Geophysics 62, 1990–1995 (1997)
91. 91.
van Huffel, S., Vandewalle, J.: The Total Least Squares Problem: Computational Aspects and Analysis. SIAM, Philadelphia (1991)
92. 92.
Wang, N., Davidian, M.: A note on covariate measurement error in nonlinear mixed effects models. Biometrika 83, 801–812 (1996)
93. 93.
Wang, N., Lin, X., Gutierrez, R., Carroll, R.J.: Bias analysis and SIMEX approach in generalized linear mixed measurement error models. J. Am. Stat. Ass. 93, 249–261 (1998)
94. 94.
Wolf, H.: Helmerts Lösung zum Problem der freien Netze mit singulärer Normalgleichungsmatrix. ZfV 97, 189–192 (1972)Google Scholar
95. 95.
Wolf, H.: Die Helmert-Inverse bei freien geodätischen Netzen. ZfV 98, 396–398 (1973)Google Scholar
96. 96.
Wolf, H.: Scale and orientation in combined doppler and triangulation nets. Bull Géod 54, 45–53 (1980)
97. 97.
Xu, P.L.: Testing the hypotheses of non-estimable functions in free net adjustment models. Manuscr. Geodaet. 20, 73–81 (1995)Google Scholar
98. 98.
Xu, P.L.: A general solution in nonlinear rank-defect geodetic models. Boll. Geod. Sci. Aff. 56, 1–25 (1997)Google Scholar
99. 99.
Xu, P.L.: Biases and accuracy of, and an alternative to, discrete nonlinear filters. J. Geod. 73, 35–46 (1999)
100. 100.
Xu, P.L.: Nonlinear filtering of continuous systems: foundational problems and new results. J. Geod. 77, 247–256 (2003)
101. 101.
Xu, P.L.: Determination of regional stress tensors from fault-slip data. Geophys. J. Int. 157, 1316–1330 (2004)
102. 102.
Xu, P.L.: The effect of errors-in-variables on variance component estimation. J. Geod. 90, 681–701 (2016)
103. 103.
Xu, P.L., Grafarend, E.: Statistics and geometry of the eigenspectra of 3-D second-rank symmetric random tensors. Geophys. J. Int. 127, 744-756 (1996)
104. 104.
Xu, P.L., Liu, J.N.: Variance components in errors-in-variables models: estimability, stability and bias analysis. Invited talk, VIII Hotine-Marussi Symposium on Mathematical Geodesy, Rome, 17–21 June (2013)Google Scholar
105. 105.
Xu, P.L., Liu, J.N.: Variance components in errors-in-variables models: estimability, stability and bias analysis. J. Geod. 88, 719–734 (2014)
106. 106.
Xu, P.L., Shimada, S.: Least squares parameter estimation in multiplicative noise models. Commun. Stat. B29, 83–96 (2000)
107. 107.
Xu, P.L., Liu, J.N., Shi, C.: Total least squares adjustment in partial errors-in-variables models: algorithm and statistical analysis. J. Geod. 86, 661–675 (2012)
108. 108.
Xu, P.L., Liu, J.N., Zeng, W., Shen, Y.Z.: Effects of errors-in-variables on weighted least squares estimation. J. Geod. 88, 705–716 (2014)
109. 109.
Zeng, H.: Analytical algorithm of weighted 3D datum transformation using the constraint of orthonormal matrix. EPS 67, art.105 (2015)Google Scholar
110. 110.
Zhou, J.W.: Quasi-stable adjustment of monitoring networks. Wuhan Inst. Geod. Geophys. Spec. Publ. No. 2 (1980). (in Chinese)Google Scholar
111. 111.
Zumar, R.H.: Robust estimation in the errors-in-variables model. Biometrika 76, 149–160 (1989)