Abstract
The concept of outlier detection by statistical hypothesis testing in geodesy is briefly reviewed. The performance of such tests can only be measured or optimized with respect to a proper alternative hypothesis. Firstly, we discuss the important question whether gross errors should be treated as non-random quantities or as random variables. In the first case, the alternative hypothesis must be based on the common mean shift model, while in the second case, the variance inflation model is appropriate. Secondly, we review possible formulations of alternative hypotheses (inherent, deterministic, slippage, mixture) and discuss their implications. As measures of optimality of an outlier detection, we propose the premium and protection, which are briefly reviewed. Finally, we work out a practical example: the fit of a straight line. It demonstrates the impact of the choice of an alternative hypothesis for outlier detection.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abdi H (2007) The Bonferonni and Šidák corrections for multiple comparisons. In: Neil Salkind (ed) Encyclopedia of measurement and statistics. Sage, Thousand Oaks
Anscombe FJ (1960) Rejection of outliers. Technometrics 2(2):123–147
Baarda W (1968) A testing procedure for use in geodetic networks. Publication on Geodesy, Netherlands
Barnett V, Lewis T (1994) Outliers in statistical data. Wiley, Chichester. ISBN: 0-471-93094-6
Clerici E, Harris MW (1980) A premium-protection method applied to detection and rejection of erroneous observations. Manuscripta Geodaetica 5:282–298
Clerici E, Harris MW (1983) A review of the premium-protection method and its possible application in detection of displacements. J Geod 57(1–4):1–9. doi:10.1007/BF02520908
Fan H (2010) Theory of errors and least squares adjustment. Royal Institute of Technology (KTH), Division of Geodesy and Geoinformatics Stockholm (Sweden), Geodesy Report No. 2015. ISBN: 91-7170-200-8
Gui Q, Li X, Gong Y, Li B, Li GA (2011) Bayesian unmasking method for locating multiple gross errors based on posterior probabilities of classification variables. J Geod 85:191–203
Hawkins D (1980) Identification of outliers. Chapman and Hall, New York
Hekimoglu S, Koch KR (2000) How can reliability of the test for outliers be measured? Allgemeine Vermessungsnachrichten. VDE Verlag Berlin Offenbach, S. 247–253
Kargoll B (2012) On the Theory and Application of Model Misspecification Tests in Geodesy. Deutsche Geodätsche Kommission Reihe C, Nr. 674, München
Koch KR (1999) Parameter estimation and hypothesis testing in linear models. Springer Verlag, Berlin
Koch KR (2007) Introduction to Bayesian statistics, 2nd edn. Springer, Berlin
Lehmann R (2010) Normalized residuals - how large is too large? (in German). Allgemeine Vermessungsnachrichten. VDE, Berlin, pp 53–661
Lehmann R, Scheffler T (2011) Monte Carlo based data snooping with application to a geodetic network. J Geod 5(3–4):123–134
Lehmann R (2012) Geodetic error calculus by the scale contaminated normal distribution (in German). Allgemeine Vermessungsnachrichten. VDE, Berlin, pp 143–149
Lehmann R (2012b) Improved critical values for extreme normalized and studentized residuals in Gauss-Markov models. J Geod 86:1137–1146. doi:10.1007/s00190-012-0569-0
Möller HP (1972) Ausreißer in Stichproben aus normalverteilten Grundgesamtheiten. PhD Thesis, Cologne University
Mood AM, Graybill FA, Boes DC (1974) Introduction to the theory of statistics. McGraw-Hill, Tokyo
Nadarajah S (2005) A generalized normal distribution. J Appl Stat 32(7):685–694. doi:10.1080/02664760500079464
Neumann I, Kutterer H, Schön St (2006) Outlier Detection in Geodetic Applications with respect to Observation Imprecision. Proceedings of the NSF Workshop on Reliable Engineering Computing - Modeling Errors and Uncertainty in Engineering Computations-. Savannah (Georgia), USA, pp. 75–90
Pope AJ (1976) The statistics of residuals and the detection of outliers. NOAA Technical Report NOS65 NGS1, US Department of Commerce, National Geodetic Survey Rockville, Maryland
Rousseeuw PJ, Leroy AM (2003) Robust regression and outlier detection. Wiley, New Jersey
Teunissen PJG (2000) Testing theory; an introduction, 2nd edn. Delft University of Technology, The Netherlands, Series on Mathematical Geodesy and Positioning. ISBN: 13 978-90-407-1975-2
Thompson W (1935) On the criterion for the rejection of observations and the distribution of the ratio of deviation to sample standard deviation. Ann Math Stat 6:214–219
Yang Y (1991) Robust Bayesian estimation. Bull Geod 65(3):145–150
Yang Y (1999) Robust estimation of geodetic datum transformation. J Geod 73:8–274
Acknowledgments
This work has been completed while the author spent his sabbatical at Technische Universität Berlin with Prof. Dr.-Ing. Frank Neitzel as his host. The support is gratefully acknowledged.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Lehmann, R. On the formulation of the alternative hypothesis for geodetic outlier detection. J Geod 87, 373–386 (2013). https://doi.org/10.1007/s00190-012-0607-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00190-012-0607-y