Skip to main content

Advertisement

SpringerLink
Log in

Method of Cleaning Outliers from Measurement Data: Search for the Optimal Solution with the Minimum Number of Rejected Measured Data

  • MEASUREMENTS IN INFORMATION TECHNOLOGIES
  • Published:
Measurement Techniques Aims and scope

This article discusses the problem of cleaning rough measurements (outliers) from a time series of noisy data received from measuring devices. Solving this problem is required to improve the accuracy of modern measuring equipment with the latest mathematical software. To minimize the amount of rejected data, the author of this article has previously formulated a problem of searching for the optimal solution and proposed a robust algorithm to solve it. The resulting solution approximates the optimal solution and does not always provide the desired minimization. An improved method of cleaning outliers from data is proposed, based on the search for the optimal solution with a minimum number of rejected measurement data. To implement this method, a problem is set that considers an unknown average of the original number series as a parameter to be determined. Additionally, an algorithm is proposed that enables us to assuredly find the optimal solution in no more than N steps, each of which requires an order of N arithmetic operations, where N is the number of initial measurement data. This improved method can be used to automatically detect and eliminate outliers from a time series of measuring data during their preliminary processing in information and measuring systems as well as in solving scientific, applied, managerial and other problems in various fields of human activity.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1.

Similar content being viewed by others

Notes

  1. GOST R 8.596-2002. GSI. Metrological support of measuring systems. Basic provisions.

References

  1. R. Dach, G. Beutler, U. Hugentobler, et al., J. Geod., 77, No. 1–2, 1–14 (2003), https://doi.org/10.1007/s00190-002-0296-z.

    Article  ADS  Google Scholar 

  2. S. I. Donchenko, I. Y. Blinov, I. B. Norets et al., Meas. Tech., 63, No. 1, 34–37 (2020), https://doi.org/10.1007/s11018-020-01746-8.

    Article  Google Scholar 

  3. A. I. Perov and V. N. Kharisov, GLONASS: Principles of Construction and Functioning, 4th ed., Radiotekhnika, Moscow (2010).

    Google Scholar 

  4. S. L. Pasynok, Meas. Tech., 63, No. 1, 38–44 (2020), https://doi.org/10.1007/s11018-020-01747-7.

    Article  Google Scholar 

  5. International GNSS Service, available at: http://www.igs.org/network (accessed: 04/05/2022).

  6. Bernese GNSS Software Version 5.2. User manual, Ed. R. Dach, S. Lutz, P. Walser, P. Fridez, Astronomical Institute, University of Bern, Bern Open Publishing (2015), https://doi.org/10.7892/boris.72297.

  7. T. E. Mehlstäubler, G. Grosche, C. Lisdat, P. O. Schmidt, and H. Denker, Rep. Prog. Phys., 81, No. 6 (2018), https://doi.org/10.1088/1361-6633/aab409.

  8. J. Levine, Metrologia, 45, S162–S174 (2008), https://doi.org/10.1088/0026-1394/45/6/S22.

    Article  ADS  Google Scholar 

  9. A. A. Karaush, Automatic System for Remote Timescales Comparison and Prediction using Carrier-Phase Measurements of the Global Navigation Satellite Systems, Izmer. Tekh., No. 9, 40–44 (2022), https://doi.org/10.32446/0368-1025it.2022-9-40-44.

  10. I. V. Bezmenov, I. Yu. Ignatenko, and S. L. Pasynok, Trans. IAA RAS, No. 60, 12–20 (2022), https://doi.org/10.32876/ApplAstron.60.12-20.

  11. A. Blazquez-Garcia, A. Conde, U. Mori, and J. A. Lozano, ACM Comput. Surv., 54, No. 3, 1–33 (2021), https://doi.org/10.1145/3444690.

    Article  Google Scholar 

  12. I. V. Bezmenov, A. V. Naumov, and S. L. Pasynok, Meas. Tech., 61, No. 9, 878–884 (2018), https://doi.org/10.1007/s11018-018-1518-y.

    Article  Google Scholar 

  13. I. V. Bezmenov, Effective Algorithms for Detection Outliers and Cycle Slip Repair in GNSS Data Measurements, chapter 9 in book Satellite Systems: Design, Modeling, Simulation and Analysis, ed. Tien M. Nguyen, IntechOpen, London, UK, 177–209 (2021), https://doi.org/10.5772/intechopen.92658.

  14. I. V. Bezmenov, A. E. Drozdov, and S. L. Pasynok, Meas. Tech., 65, No. 1, 29–34 (2022), https://doi.org/10.1007/s11018-022-02085-6.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Russian Metrological Institute of Technical Physics and Radio Engineering, Mendeleevo, Moscow region, Russia

    I. V. Bezmenov

Authors
  1. I. V. Bezmenov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to I. V. Bezmenov.

Additional information

Translated from Izmeritel'naya Tekhnika, No. 1, pp. 16–23, January, 2023.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bezmenov, I.V. Method of Cleaning Outliers from Measurement Data: Search for the Optimal Solution with the Minimum Number of Rejected Measured Data. Meas Tech 66, 14–23 (2023). https://doi.org/10.1007/s11018-023-02184-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11018-023-02184-y

Keywords

Search

Navigation