Abstract
Outliers could have an impact on misspecification of models, the bias in estimating of parameters, incorrect results, and poor forecasts. Outliers in weather data arise due to human error and instrument error. Outliers can be categorized into three categories: point outliers, collective outliers, and contextual outliers. In this paper, we propose a point outlier detection method based on a moving median filter. The method consists of three key steps. They are autocorrelation, moving median filter, and threshold determination. The autocorrelation step gives the information of similarity between immediate neighbors. The result of the autocorrelation step defines window width of the median filter. The output of the median filter gives candidate outliers and exact outliers detected based on the threshold values. Out analysis of the results demonstrate in this paper is successful and effective in detecting outliers in meteorological data.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Liu, H., Shah, S., Jiang, W.: On-line outlier detection and data cleaning. Comput. Chem. Eng. 28(9), 1635–1647 (2004). https://doi.org/10.1016/j.compchemeng.2004.01.009
Chang-Tein, L., Yufeng, K., Jiang, Z., Li, C.: Detecting and tracking regional outliers in meteorological data. Inf. Sci. 177(7), 1609–1632 (2007). https://doi.org/10.1016/j.ins.2006.09.013
Ali, R., Salim, A., Omar, Z., Omar, A., Negar, K., Keivan, K., Reda, A., Jon, R.: Graph-based approach for outlier detection in sequential data and its application on stock market and weather data. Knowl.-Based Syst. 61, 89–97 (2014). https://doi.org/10.1016/j.knosys.2014.02.008
Gogoi, P., Bhattacharyya, D.K., Borah, B., Kalita, J.K.: A survey of outlier detection methods in network anomaly identification. Comput. J. 54(4), 570–588 (2011). https://doi.org/10.1093/comjnl/bxr026
Battaglia, F.: Outliers in functional autoregressive time series. Stat. Prob. Lett. 72(4), 323–332 (2005). https://doi.org/10.1016/j.spl.2005.02.003
Bellini, T.: The forward search interactive outlier detection in cointegrated VAR analysis. Adv. Data Anal. Class. 10(3), 351–373 (2016). https://doi.org/10.1007/s11634-015-0216-8
Silva, M., Pereira, I.: Detection of additive outliers in Poisson integer-valued autoregressive time series. Statistics (2012). arXiv: 1204.6516
Baragona, R., Battaglia, F., Cucina, D.: Empirical likelihood for outlier detection and estimation in autoregressive time series. J. Time Ser. Anal. 37(3), 315–336 (2015). https://doi.org/10.1111/jtsa.12145
Lu, H., Liu, Y., Fei, Z., Guan, C.: An outlier detection algorithm based on cross-correlation analysis for time series dataset. IEEE Access 6, 53593–53610 (2018). https://doi.org/10.1109/ACCESS.2018.2870151
Chandola, V., Banerjee, A., Kumar, V.: Anomaly detection: a survey. ACM Comput. Surveys 41(3), (2009). https://doi.org/10.1145/1541880.1541882
Citizen Weather Observer Program, CWOP Database. http://www.wxqa.com
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Sukhbaatar, J., Zagd, B., Jambaljav, N. (2021). Detection of Point Outliers in Meteorological Data (Case Study: Ulaanbaatar, Mongolia). In: Pan, JS., Li, J., Ryu, K.H., Meng, Z., Klasnja-Milicevic, A. (eds) Advances in Intelligent Information Hiding and Multimedia Signal Processing. Smart Innovation, Systems and Technologies, vol 212. Springer, Singapore. https://doi.org/10.1007/978-981-33-6757-9_9
Download citation
DOI: https://doi.org/10.1007/978-981-33-6757-9_9
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-33-6756-2
Online ISBN: 978-981-33-6757-9
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)