Advertisement

Environmental Monitoring and Assessment

, Volume 185, Issue 6, pp 4483–4489 | Cite as

Detecting and correcting sensor drifts in long-term weather data

  • Georg von ArxEmail author
  • Matthias Dobbertin
  • Martine Rebetez
Article

Abstract

Quality control of long-term monitoring data of thousands and millions of individual records as present in meteorological data is cumbersome. In such data series, sensor drifts, stalled values, and scale shifts may occur and potentially result in flawed conclusions if not noticed and handled properly. However, there is no established standard procedure to perform quality control of high-frequency meteorological data. In this paper, we outline a procedure to remove sensor drift in high-frequency data series using the example of 15-year-long sets of hourly relative humidity (RH) data from 28 stations subdivided into 202 individual sensor operation periods. The procedure involves basic quality control, relative homogeneity testing, and drift removal. Significant sensor drifts were observed in 40.6 % of all sensor operation periods. The drifts varied between data series and depended in a complex, usually inconsistent way on absolute RH values; within single series for instance, a drift could be negative in the lower RH range and positive in the upper RH range. Detrending changed RH values by, on average, 1.96 %. For one fifth of the detrended data, adjustments were 2.75 % and more of the measured value, and in one tenth 4.75 % and more. Overall, drifts were strongest for RH values close to 100 %. The detrending procedure proved to effectively remove sensor drifts. The principles of the procedure also apply to other meteorological parameters and more generally to any time series of data for which comparable reference data are available.

Keywords

Detrending Homogeneity testing Quality control Relative humidity Sensor drift Swiss Long-Term Forest Ecosystem Research Programme LWF 

Notes

Acknowledgments

We thank Gustav Schneiter for meteorological data collection and sharing his expertise of local station situation and Peter Jakob for help with the data base. Michael Begert is appreciated for his valuable advice and input to quality control, data adjustment, and the text. An anonymous reviewer made valuable suggestions to a previous version of this paper. This project was supported by a grant within the program “Forest and Climate Change” from the Swiss Federal Office for the Environment FOEN and Swiss Federal Research Institute for Forest, Snow and Landscape Research WSL.

References

  1. Aguilar, E., Auer, I., Brunet, M., Peterson, T. C., & Wieringa, J. (2003). Guidelines on climate metadata and homogenization. WMO-TD 53. WMO-TD no. 1186, pp. 51Google Scholar
  2. Begert, M., Schlegel, T., & Kirchhofer, W. (2005). Homogeneous temperature and precipitation series of Switzerland from 1864 to 2000. International Journal of Climatology, 25(1), 65–80. doi: 10.1002/joc.1118.CrossRefGoogle Scholar
  3. Begert, M., Zenklusen, E., Häberli, C., Appenzeller, C., & Klok, L. (2008). An automated procedure to detect discontinuities; performance assessment and application to a large European climate data set. Meteorologische Zeitschrift, 17(5), 663–672.CrossRefGoogle Scholar
  4. Brandsma, T., & Konnen, G. P. (2006). Application of nearest-neighbor resampling for homogenizing temperature records on a daily to sub-daily level. International Journal of Climatology, 26(1), 75–89. doi: 10.1002/joc.1236.CrossRefGoogle Scholar
  5. Conrad, V., & Pollack, L. W. (1950). Methods in climatology (2nd ed.). Cambridge: Harvard University Press.Google Scholar
  6. Costa, A. C., & Soares, A. (2009). Homogenization of climate data: review and new perspectives using geostatistics. Mathematical Geosciences, 41(3), 291–305. doi: 10.1007/s11004-008-9203-3.CrossRefGoogle Scholar
  7. Della-Marta, P. M., & Wanner, H. (2006). A method of homogenizing the extremes and mean of daily temperature measurements. Journal of Climate, 19(17), 4179–4197. doi: 10.1175/jcli3855.1.CrossRefGoogle Scholar
  8. Innes, J. L. (1995). Theoretical and practical criteria for the selection of ecosystem monitoring plots in Swiss forests. Environmental Monitoring and Assessment, 36(3), 271–294.CrossRefGoogle Scholar
  9. Jakob, P., Sutter, F., Waldner, P., & Schneiter, G. (2007). Designing a flexible ict framework for processing remote gauging-data information. In J. M. Gómez, M. Sonnenschein, M. Müller, H. Welsch, & C. Rautenstrauch (Eds.), Technologies in environmental engineering (pp. 211–220). Berlin: Springer. Environmental Science and Engineering.CrossRefGoogle Scholar
  10. Kramer, J. P., & Boyer, J. S. (1995). Water relations of plants and soils (2nd ed.). San Diego: Academic.Google Scholar
  11. Peterson, T. C., & Easterling, D. R. (1994). Creation of homogeneous composite climatological reference series. International Journal of Climatology, 14(6), 671–679. doi: 10.1002/joc.3370140606.CrossRefGoogle Scholar
  12. Trewin, B. C., & Trevitt, A. C. F. (1996). The development of composite temperature records. International Journal of Climatology, 16(11), 1227–1242. doi:10.1002/(sici)1097-0088(199611)16:11<1227::aid-joc82>3.0.co;2-p.CrossRefGoogle Scholar
  13. von Arx, G., Dobbertin, M., & Rebetez, M. (2012). Spatio-temporal effects of forest canopy on understory microclimate in a long-term experiment in Switzerland. Agricultural and Forest Meteorology. doi: 10.1016/j.agrformet.2012.07.018.

Copyright information

© Springer Science+Business Media B.V. 2012

Authors and Affiliations

  • Georg von Arx
    • 1
    Email author
  • Matthias Dobbertin
    • 1
  • Martine Rebetez
    • 1
    • 2
  1. 1.Swiss Federal Institute for Forest, Snow and Landscape Research WSLBirmensdorfSwitzerland
  2. 2.University of NeuchatelInstitute of GeographyNeuchatelSwitzerland

Personalised recommendations