Advertisement

Studia Geophysica et Geodaetica

, Volume 63, Issue 1, pp 55–70 | Cite as

Empirical estimation of the power of test in outlier detection problem

  • Bahattin ErdoganEmail author
  • Serif Hekimoglu
  • Utkan Mustafa Durdag
  • Taylan Ocalan
Article
  • 15 Downloads

Abstract

Classical outlier detection test methods such as Baarda test and Pope test are generally preferred in geodetic problems. They depend on the Least Square Estimation (LSE) and LSE is very sensitive to the variations of the model. The capacity of the LSE changes depending on the different significance level, different type of outlier, the number of outlier, magnitude of outlier, number of observations and the number of unknowns. In statistics, the power of test is the probability of rejecting the null hypothesis when the null hypothesis is false. It is a theoretical assumption and depends on the significance level α (Type I error) and β (Type II error). The different types of the outliers, such as random or non-random, affect the results of the test methods; but the power of test is the same for all different types of the outliers. In this study, empirical estimation of the power of test is presented as Mean Success Rate (MSR). The theoretical power of test and empirical MSR have been estimated for univariate model and linear model by using Baarda test; according to the obtained results, MSR can be used as empirical value of the power of test and capacity of the test models. Also, MSR reflects more realistic results than the theoretical power of test.

Keywords

power of test efficacy mean success rate Baarda test outlier 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Atkinson A.C., 1985. Plots, Transformations and Regression. Oxford University Press, Oxford, U.K.Google Scholar
  2. Aydin C., 2012. Power of global test in deformation analysis. J. Surv. Eng., 138, 51–56.CrossRefGoogle Scholar
  3. Baarda W., 1967. Statistical concepts in geodesy. Publication on Geodesy, New Series, 2(4), Netherlands Geodetic Commission, Delft, The Netherlands.Google Scholar
  4. Baarda W., 1968. A testing procedure for use in geodetic Networks. Publication on Geodesy, New Series, 2(5), Netherlands Geodetic Commission, Delft, The Netherlands.Google Scholar
  5. Chen Y.Q., Kavouras M. and Chrzanowski A., 1987. A strategy for detection of outlying observations in measurements of high precision. The Canadian Surveyor, 41, 529–540.Google Scholar
  6. Cook R.D. and Weisberg S., 1982. Residuals and Influence in Regression. Chapman & Hall, New York.Google Scholar
  7. Donoho D.L. and Huber P.J., 1983. The notion of breakdown point. In: Bickel P.J., Doksum K. and Hodges J.L. Jr. (Eds), A Festschrift for Erich Lehmann. Wadsworth, Belmont, CA, 157–184.Google Scholar
  8. Hadi A.S. and Simonoff J.S., 1993. Procedures for the identification of multiple outliers in linear models. J. Am. Stat. Assoc., 88, 1264–1272.CrossRefGoogle Scholar
  9. Hampel F., Ronchetti E., Rousseeuw P. and Stahel W., 1986. Robust Statistics: the Approach Based on Influence Functions. John Wiley and Sons, New York.Google Scholar
  10. Heck B., 1981. Der Einfluss einzelner Beobachtungen auf das Ergebnis einer und die Suche nach Ausreissern in den beobachtungen. Allg. Verm. Nachricht., 88, 17–34.Google Scholar
  11. Hekimoglu S., 1997. The finite sample breakdown points of the conventional iterative outlier detection procedures. J. Surv. Eng., 123, 15–31.CrossRefGoogle Scholar
  12. Hekimoglu S. and Koch K.R., 1999. How can reliability of the robust methods be measured? In: Altan M.O. and Gründige L. (Eds), Third Turkish-German Joint Geodetic Days, Volume 1. Istanbul Technical University, Istanbul, Turkey, 179–196.Google Scholar
  13. Hekimoglu S. and Koch K.R., 2000. How can reliability of the test for outliers be measured? Allg. Verm. Nachricht., 107, 247–254.Google Scholar
  14. Hekimoglu S., Erdogan B., Erenoglu R.C. and Hosbas R.G., 2011. Increasing the efficacy of the tests for outliers for geodetic networks. Acta Geod. Geophys. Hung., 46, 291–308.CrossRefGoogle Scholar
  15. Hekimoglu S. and Erdogan B., 2012. New median approach to define configuration weakness of deformation networks. J. Surv. Eng., 138, 101–108.CrossRefGoogle Scholar
  16. Huber P.J., 1981. Robust Statistics. John Wiley and Sons., New York.CrossRefGoogle Scholar
  17. Knight N.L., Wang J. and Rizos C., 2010. Generalised measures of reliability for multiple outliers. J. Geodesy, 84, 625–635.CrossRefGoogle Scholar
  18. Koch K.R., 1999. Parameter Estimation and Hypothesis Ttesting in Linear Models, 2nd Edition. Springer-Verlag, Berlin, Germany.CrossRefGoogle Scholar
  19. Koch K.R., 2013a. Robust estimation by expectation maximization algorithm. J. Geodesy, 87, 107–116.CrossRefGoogle Scholar
  20. Koch K.R., 2013b. Comparison of two robust estimations by expectation maximization algorithms with Huber’s method and outlier tests. J. Appl. Geodesy, 7, 115–123.Google Scholar
  21. Kok J.J., 1984. On Data Snooping and Multiple Outlier Testing. NOAA Technical Report, 30. U.S. Department of Commerce, Rockville, MD.Google Scholar
  22. Kuang S., 1991. Optimization and Design of Deformation Monitoring Schemes. Ph.D. Thesis. Report 157. Department of Surveying Engineering, University of New Brunswick, Fredericton, NB, Canada.Google Scholar
  23. Lehmann R., 2010. Normierte Verbeßserungen-wie groß ist zu groß? Allgemeine Vermessungsnachrichten, 2, 53–61 (in German.Google Scholar
  24. Lehmann R., 2012a. Geodätische Fehlerrechnung mit der skalenkontaminierten Normalverteilung. Allgemeine Vermessungsnachrichten, 143–149 (in German).Google Scholar
  25. Lehmann R., 2012b. Improved critical values for extreme normalized and studentized residuals in Gauss-Markov models. J. Geodesy, 86, 1137–1146.CrossRefGoogle Scholar
  26. Maronna R., Martin D. and Yohai V., 2006. Robust Statistics. John Wiley and Sons., New York.Google Scholar
  27. Neyman J. and Pearson E.S., 1933. On the problem of the most efficient tests of statistical hypotheses. Philos. Trans. R. Soc. Lond. Ser. A-Math. Phys. Eng. Sci., 231, 289–337.CrossRefGoogle Scholar
  28. Pope A.J., 1976. The Statistics of Residuals and the Outlier Detection of Outliers. NOAA Technical Report, 65. U.S. Department of Commerce, Rockville, MD.Google Scholar
  29. Teunissen P.J.G., 2000. Testing Theory-an Introduction. Delft University, Delft, The Netherlands.Google Scholar
  30. Teunissen P.J.G., 2018. Distributional theory for the DIA method. J. Geodesy, 92, 59–80.CrossRefGoogle Scholar
  31. Wang J. and Chen Y.Q., 1999. Outlier detection and reliability measures for singular adjustment models. Geomat. Res. Australasia, 71, 57–72.Google Scholar
  32. Wang J. and Knight N.L., 2012. New outlier separability test and its application in GNSS positioning. J. Glob. Posit. Syst., 11, 46–57.CrossRefGoogle Scholar
  33. Yang L., Wang J., Knight N.L. and Shen Y., 2013. Outlier separability analysis with a multiple alternative hypotheses test. J. Geodesy, 87, 591–604.CrossRefGoogle Scholar
  34. Zaminpardaz S. and Teunissen P.J.G., 2018. DIA-datasnooping and identifiability. J. Geodesy, DOI:  https://doi.org/10.1007/s00190-018-1141-3(in print).Google Scholar

Copyright information

© Institute of Geophysics of the ASCR, v.v.i 2019

Authors and Affiliations

  • Bahattin Erdogan
    • 1
    Email author
  • Serif Hekimoglu
    • 1
  • Utkan Mustafa Durdag
    • 1
  • Taylan Ocalan
    • 1
  1. 1.Department of Geomatic Engineering, Faculty of Civil EngineeringYildiz Technical UniversityIstanbulTurkey

Personalised recommendations