Abstract
In the geodetic community, an adjustment framework is established by the four components of model choice, parameter estimation, variance component estimation (VCE), and quality control. For linear ill-posed models, the parameter estimation and VCE have been extensively investigated. However, at least to the best of our knowledge, nothing is known about the quality control of hypothesis testing in ill-posed models although it is indispensable and rather important. In this paper, we extend the theory of hypothesis testing to ill-posed models. As the Tikhonov regularization is typically applied to stabilize the solution of ill-posed models, the solution and its associated residuals are biased. We first derive the statistics of overall-test, w-test and minimal detectable bias for an ill-posed model. Due to the bias influence, both overall-test and w-test statistics do not follow the distributions as those used in well-posed models. Then, we develop the bias-corrected statistics. As a result, the bias-corrected w-test statistic can sufficiently be approximated by a standard normal distribution; while the bias-corrected overall-test statistic can be approximated by two non-central chi-square distributions, respectively. Finally, some numerical experiments with a Fredholm first kind integral equation are carried out to demonstrate the performance of our designed hypothesis testing statistics in an ill-posed model.
Similar content being viewed by others
Data availability
The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.
References
Amiri-Simkooei A (2007) Least squares variance component estimation: theory and GPS applications. PhD dissertation, Delft Univ. Technol., Delft, The Netherlands
Amiri-Simkooei A, Teunissen PJG, Tiberius C (2009) Application of least-squares variance component estimation to GPS observables. J Surv Eng 135(4):149–160
Arnold S (1981) The theory of linear models and multivariate analysis. Wiley, New York
Baarda W (1967) Statistical concepts in geodesy. Netherlands Geodetic Commission, Publ. on geodesy, New series, vol 2, no 4, Delft, The Netherlands
Baarda W (1968) A testing procedure for use in geodetic networks. Netherlands Geodetic Commission, Publ. on geodesy, New Series, vol 2, no 5, Delft, The Netherlands
Calvetti D, Morigi S, Reichel L, Sgallari F (2000) Tikhonov regularization and the L-curve for large discrete ill-posed problems J. Comput Appl Math 123(1–2):423–446
Golub G, Heath M, Wahba G (1979) Generalized cross-validation as a method for choosing a good ridge parameter. Technometrics 21(2):215–223
Grafarend E, Schaffrin B (1993) Adjustment computations in linear models (in German). Bibliographical Institute, Mannheim
Hansen P (1990) Truncated singular value decomposition solutions to discrete ill-posed problems with ill-determined numerical rank. SIAM J Sci Comput 11(3):503–518
Hansen P, O’Leary D (1993) The use of the L-curve in the regularization of discrete ILL-posed problems. SIAM J Sci Comput 14(6):1487–1503
Helmert F (1907) Die Ausgleichungsrechnung nach der Methode der kleinsten quadrate, 2nd edn. Teubner, Leipzig
Hoerl A, Kennard R (1970a) Ridge regression: biased estimation for nonorthogonal problems. Technometrics 12(1):55–67
Hoerl A, Kennard R (1970b) Ridge regression: application to nonorthogonal problems. Technometrics 12(1):69–82
Kent J, Mohammadzadeh M (2000) Global optimization of the generalized cross-validation criterion. Stat Comput 10(3):231–236
Koch K (1978) Schätzung von Varianzkomponenten. Allg Ver-mess Nachr 85:264–269
Koch K (1986) Maximum likelihood estimate of variance components. J Geod 60(4):329–338
Koch K (1999) Parameter estimation and hypothesis testing in linear models, 2nd edn. Springer, Berlin
Koch K (2007) Introduction to Bayesian Statistics, 2nd edn. Springer, Berlin
Koch K, Kusche J (2002) Regularization of geopotential determination from satellite data by variance components. J Geod 76(5):259–268
Kubik K (1970) The estimation of the weights of measured quantities within the method of least squares. Bull Geod 95(1):21–40
Kusche J (2003) Noise variance estimation and optimal weight determination for GOCE gravity recovery. Adv Geosci Eur Geosci Union 1:81–85
Leick A (1980) Adjustment computations. Lecture Notes in Surveying Engineering. University of Maine, Orono
Li B (2016) Stochastic modeling of triple-frequency BeiDou signals: estimation, assessment and impact analysis. J Geod 90(7):593–610
Li B, Shen Y, Xu P (2008) Assessment of stochastic models for GPS measurements with different types of receivers. Chin Sci Bull 53(20):3219–3225
Li B, Shen Y, Lou L (2011) Efficient estimation of variance and covariance components: A case study for GPS stochastic model evaluation. IEEE Trans Geosci Remote Sens 49(1):203–210
Li B, Lou L, Shen Y (2016) GNSS elevation-dependent stochastic modeling and its impacts on the statistic testing. J Surv Eng 142(2):04015012
Li B, Zhang L, Verhagen S (2017) Impacts of BeiDou stochastic model on reliability: overall test, w-test and minimal detectable bias. GPS Solut 21(3):1095–1112
Mathai A, Provost S (1992) Quadratic forms in random variables: theory and applications. Marcel Dekker, New York
Pope A (1975) The statistics of residuals and the detection of outliers, presented at the XVI General Assembly of IUGG, Grenoble, France, 1975, reprinted as NOAA Tech. Report Nos. 65 NGS 1, 1976.
Pukelsheim F (1976) Estimating variance components in linear models. J Multivar Anal 6(4):626–629
Rao C (1971a) Estimation of variance and covariance components–MINQUE theory. J Multivar Anal 1(3):257–275
Rao C (1971b) Minimum variance quadratic unbiased estimation of variance components. J Multivar Anal 1(4):445–456
Rao C, Toutenburg H (1995) Linear models: least-squares and alternatives. Springer, Heidelberg
Schaffrin B (1983) Estimation of variance-covariance components for heterogenous replicated measurements (in German), German Geodetic Community, Publ. C-282, Munich
Schaffrin B (2008a) On penalized least squares: its mean squared error and a quasi-optimal weight ratio. In: Recent advances in linear models and related areas. Physica-Verlag HD, pp 313–322
Schaffrin B (2008b) Minimum mean squared error (MSE) adjustment and the optimal Tikhonov-Phillips regularization parameter via reproducing best invariant quadratic uniformly unbiased estimates (repro-BIQUUE). J Geod 82(2):113–121
Schwartz J (2012) Introduction to matrices and vectors. Series: Dover books on mathematics (republished). Dover Publications Incorporation, Mineola
Shen Y, Xu P, Li B (2012) Bias-corrected regularized solution to inverse ill-posed models. J Geod 86:597–608
Strang G, Borre K (1997) Linear Algebra, Geodesy, and GPS. Wellesley-Cambridge Press, Wellesley
Teunissen PJG (1990) An integrity and quality control procedure for use in multi sensor integration. ION GPS (republished in ION Red Book Series, vol. 7, 2010)
Teunissen PJG (1998a) Minimal detectable biases of GPS data. J Geod 72(4):236–244
Teunissen PJG (1998b) Quality control and GPS. Chap. 7 in GPS for Geodesy, 2nd edn., pp 187–229
Teunissen PJG (2003) Adjustment theory: an introduction, 2nd edn. Delft University Press, Delft
Teunissen PJG (2006) Testing theory: an introduction, 2nd edn. Delft University Press, Delft
Teunissen PJG (2018) Distributional theory for the DIA method. J Geod 92:59–80
Teunissen PJG, Amiri-Simkooei A (2008) Least-squares variance component estimation. J Geod 82(2):65–82
Teunissen PJG, Salzmann M (1989) A recursive slippage test for use in state-space filtering. Manus Geod 14(6):383–390
Teunissen PJG, Simons D, Tiberius C (2005) Probability and observation theory. Lecture Notes Delft University of Technology, Delft
Tiberius C, Kenselaar F (2003) Variance component estimation and precise GPS positioning: case study. J Surv Eng 129(1):11–18
Tienstra J (1956) Theory of the adjustment of normally distributed observations. Argus, Amsterdam
Tikhonov A (1963) Solution of incorrectly formulated problems and the regularization method. Soviet Math Dokl 4:1035–1038
Tikhonov A, Arsenin V (1977) Solutions of ill-posed problems. Wiley, New York
Tikhonov A, Goncharsky V, Steppanov V, Yagola A (1995) Numerical methods for the solution of ill-posed problems. Kluwer Academic Publishers, Dordrecht
Wang J, Stewart M, Sakiri M (1998) Stochastic modeling for static GPS baseline data processing. J Surv Eng 124(4):171–181
Wang J, Satirapod C, Rizos C (2002) Stochastic assessment of GPS carrier phase measurements for precise static relative positioning. J Geod 76(2):95–104
Wu Z, Bian S, Xiang C, Tong Y (2013) A new method for TSVD regularization truncated parameter selection. Math Probl Eng 2013:161834. https://doi.org/10.1155/2013/161834
Xu P (1992) Determination of surface gravity anomalies using gradiometric observables. Geophys J Int 110(2):321–332
Xu P (1998) Truncated SVD methods for linear discrete ill-posed problems. Geophys J Int 135(2):505–514
Xu P (2009) Iterative generalized cross-validation for fusing heteroscedastic data of inverse ill-posed problems. Geophys J Int 179(1):182–200
Xu P, Rummel R (1994a) A simulation study of smoothness methods in recovery of regional gravity fields. Geophys J Int 117(2):472–486
Xu P, Rummel R (1994b) Generalized ridge regression with applications in determination of potential fields. Manus Geod 20(1):8–20
Xu P, Shen Y, Fukuda Y, Liu Y (2006) Variance component estimation in linear inverse ill-posed models. J Geod 80(2):69–81
Xu P, Liu Y, Shen Y, Fukuda Y (2007) Estimability analysis of variance and covariance components. J Geod 81:593–602
Yang Y, Xu T, Song L (2005) Robust estimation of variance components with application in global positioning system network adjustment. J Surv Eng 131(4):107–112
Yang Y, Zeng A, Zhang J (2009) Adaptive collocation with application in height system transformation. J Geod 83(5):403–410
Yu Z (1996) A universal formula of maximum likelihood estimation of variance-covariance components. J Geod 70(4):233–240
Acknowledgements
This work is supported by the National Natural Science Funds of China (41874030, 42074026), the Program of Shanghai Academic Research Leader (20XD1423800) and National Key Research and Development Program of China (2016YFB0501802), and the Fundamental Research Funds for the Central Universities.
Author information
Authors and Affiliations
Contributions
BL proposed this study and developed the theory. MW conducted the numerical computation and analysis. BL and MW wrote the manuscript and YS reviewed and commented on the manuscript. All authors were involved in discussions throughout the development.
Corresponding author
Appendices
Appendix A: The derivation of (23)
We start with
Inserting \({\mathbf{v}}_{\kappa }\) of (7a) into (A1) yields
Inserting \({\mathbf{R}}_{\kappa }={\mathbf{I}}_{m}-\mathbf{A}{\mathbf{N}}_{\kappa }^{-1}{\mathbf{A}}^{\mathrm{T}}\mathbf{P}\) into (A2) yields
Further inserting \({\mathbf{N}}_{\kappa }^{-1}\mathbf{N}={\mathbf{I}}_{n}-\kappa {\mathbf{Q}}_{\kappa s}\) into (A3) obtains
Substituting \({\mathbf{N}}_{\kappa }^{-1}\mathbf{N}={\mathbf{I}}_{n}-\kappa {\mathbf{Q}}_{\kappa s}\) and (A4) into (A2), it follows
Appendix B: The derivation of (34)
The non-centrality parameter follows
Substituting \({\mathbf{R}}_{\kappa }\mathbf{A}=\kappa \mathbf{A}{\mathbf{N}}_{\kappa }^{-1}\mathbf{S}\) and \({\mathbf{R}}_{\kappa }={\mathbf{I}}_{m}-\mathbf{A}{\mathbf{N}}_{\kappa }^{-1}{\mathbf{A}}^{\mathrm{T}}\mathbf{P}\) into (B1) obtains
Inserting \({\mathbf{N}}_{\kappa }^{-1}\mathbf{N}={\mathbf{I}}_{n}-\kappa {\mathbf{Q}}_{\kappa s}\) into (B2) yields
Appendix C: The derivation of (35)
The expectation of quadratic form of bias-corrected residuals \({\mathbf{v}}_{\kappa ,1}\) with respect to weight matrix \(\mathbf{P}\) is \(\Omega =\mathrm{E}\left({\mathbf{v}}_{\kappa ,1}^{\mathrm{T}}\mathbf{P}{\mathbf{v}}_{\kappa ,1}\right)=\mathrm{Tr}\left\{\mathbf{P}\mathrm{E}\left({\mathbf{v}}_{\kappa ,1}{\mathbf{v}}_{\kappa ,1}^{\mathrm{T}}\right)\right\}\). Inserting (32a) yields
with \(\delta {\widehat{\mathbf{x}}}_{\kappa }=\mathbf{x}-{\widehat{\mathbf{x}}}_{\kappa }\). It follows from the knowledge of linear algebra that (Strang and Borre 1997)
Inserting \(\mathrm{E}\left(\delta {\widehat{\mathbf{x}}}_{\kappa }\right)=\kappa {\mathbf{Q}}_{\kappa s}\mathbf{x}\) and \(\mathrm{D}\left(\delta {\widehat{\mathbf{x}}}_{\kappa }\right)={\sigma }_{0}^{2}{\mathbf{N}}_{\kappa }^{-1}\mathbf{N}{\mathbf{N}}_{\kappa }^{-1}\) yields
It is rather easy to prove that \(\mathrm{E}\left(\delta {\widehat{\mathbf{x}}}_{\kappa }{{\varvec{\upvarepsilon}}}^{\mathrm{T}}\right)=-{\sigma }_{0}^{2}{\mathbf{N}}_{\kappa }^{-1}{\mathbf{A}}^{\mathrm{T}}\). Then
Substituting \({\mathbf{N}}_{\kappa }^{-1}\mathbf{N}={\mathbf{I}}_{n}-\kappa {\mathbf{Q}}_{\kappa s}\) into (C4) yields the expectation of \({T}_{\kappa ,1}\) as
Appendix D: The derivation of (39)
The non-centrality parameter \({\lambda }_{t}^{2}\) reads
In terms of \({\mathbf{R}}_{\kappa }\mathbf{A}=\kappa \mathbf{A}{\mathbf{Q}}_{\kappa s}\), we have the recursion
Substituting (D2a) and (D2b) into (D1) yields
Inserting \({\mathbf{N}}_{\kappa }^{-1}\mathbf{N}={\mathbf{I}}_{n}-\kappa {\mathbf{Q}}_{\kappa s}\) into (D3) gives
Appendix E: The derivation of (40)
With bias-corrected residual \({\mathbf{v}}_{\kappa ,\mathrm{t}}={\mathbf{R}}_{\kappa }^{t+1}\mathbf{A}\mathbf{x}+{\mathbf{R}}_{\kappa }^{t+1}{\varvec{\upvarepsilon}}\) of (37a), the quadratic form is
Inserting \({\mathbf{N}}_{\kappa }^{-1}\mathbf{N}={\mathbf{I}}_{n}-\kappa {\mathbf{Q}}_{\kappa s}\) into (E1) gives the expectation of \({T}_{\kappa ,t}\) as
Following the binomial theorem of matrix product (Schwartz 2012, pp.158–159), the expansion of \({\mathbf{R}}_{\kappa }^{p}\) is
Then, the trace of \({\mathbf{R}}_{\kappa }^{p}\) is
where \(\left(\begin{array}{c}p\\ r\end{array}\right)={\prod }_{l=1}^{r}\frac{p-l+1}{l}\) is a binomial coefficient. Again applying the binomial theorem of matrix product yields
and then
The summation of constant terms in (E6) reads
The coefficient sum of \(\mathrm{Tr}\left\{{\mathbf{Q}}_{\kappa s}^{i}\right\}\) is (\(i>0\))
In terms of equation \(\left(\begin{array}{c}n\\ m\end{array}\right)\left(\begin{array}{c}m\\ k\end{array}\right)=\left(\begin{array}{c}n\\ k\end{array}\right)\left(\begin{array}{c}n-k\\ m-k\end{array}\right)\) for any \(k\le m\le n\) (Schwartz 2012), (E8) can be rewritten as
Inserting \(r=j+i\) into (E9) yields
Then we have
Finally, the trace of \({\mathbf{R}}_{\kappa }^{2t+2}\) reads
Appendix F: The derivation of (50a, b)
To prove the convergence of \({\lambda }_{t}^{2}\) and \({b}_{{T}_{\kappa ,t}}\) with respect to t, we assume the regularization matrix \(\mathbf{S}={\mathbf{I}}_{n}\) without loss of generality. Since \({b}_{{T}_{\kappa ,t}}={\lambda }_{t}^{2}+{\kappa }^{2t+2}\mathrm{Tr}\left\{{\mathbf{Q}}_{\kappa s}^{2t+2}\right\}\), it is equivalent to prove the convergence of \({\lambda }_{t}^{2}\) and \({q}_{t}={\kappa }^{2t+2}\mathrm{Tr}\left\{{\mathbf{Q}}_{\kappa s}^{2t+2}\right\}\).
Let the singular value decomposition of \(\kappa {\mathbf{Q}}_{\kappa s}\) as
where \(\mathbf{U}\) is an orthogonal matrix, and its ith column denotes by \({\mathbf{u}}_{i}\). \({\varvec{\Lambda}}=\mathrm{diag}([{\lambda }_{1},\dots ,{\lambda }_{n}])\) is a diagonal matrix consists of all n singular values. Since all singular values of the positive-definite matrix \({\mathbf{N}}_{\kappa }^{-1}\mathbf{N}={\mathbf{I}}_{n}-\kappa {\mathbf{Q}}_{\kappa s}\) are positive and smaller than 1 due to \(\kappa >0\), it follows \(0<{\lambda }_{i}<1\). With (F1), we rewrite \({q}_{t}\) and \({\lambda }_{t}^{2}\) as
The first and second-order derivatives of \({\lambda }_{t}^{2}\) and \({q}_{t}\) read
The above derivatives indicate that both \({\lambda }_{t}^{2}\) and \({q}_{t}\) are monotonically decreased with increasing correction times t. Moreover, for any t, both \({\lambda }_{t}^{2}\) and \({q}_{t}\) are larger than 0. Therefore, both \({\lambda }_{t}^{2}\) and \({q}_{t}\) will converge to 0 for sufficient large t.
Rights and permissions
About this article
Cite this article
Li, B., Wang, M. & Shen, Y. The hypothesis testing statistics in linear ill-posed models. J Geod 95, 11 (2021). https://doi.org/10.1007/s00190-020-01465-6
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s00190-020-01465-6