ICONIP 2015: Neural Information Processing pp 82-89 | Cite as
Adaptive Threshold for Anomaly Detection Using Time Series Segmentation
Abstract
Time series data are generated from almost every domain and anomaly detection becomes extremely important in the last decade. It consists in detecting anomalous patterns through identifying some new and unknown behaviors that are abnormal or inconsistent relative to most of the data. An efficient anomaly detection algorithm has to adapt the detection process for each system condition and each time series behavior. In this paper, we propose an adaptive threshold able to detect anomalies in univariate time series. Our algorithm is based on segmentation and local means and standard deviations. It allows us to simplify time series visualization and to detect new abnormal data as time series jumps within different time series behavior. On synthetic and real datasets the proposed approach shows good ability in detecting abnormalities.
Keywords
Anomaly detection Time series Unsupervised detectionReferences
- 1.Esling, P., Agon, C.: Time-series data mining. ACM Comput. Surv. 45(1), 34 (2012)CrossRefMATHGoogle Scholar
- 2.Gupta, M., Gao, J., Aggarwa, C.C., Han, J.: Outlier detection for temporal data: a survey. IEEE Trans. Knowl. Data Eng. 26, 2250–2267 (2014)CrossRefGoogle Scholar
- 3.Chandola, V., Banerjee, A., Kumar, V.: Anomaly detection: a survey. ACM Comput. Surv. 41(3), 58 (2009). Article 15CrossRefGoogle Scholar
- 4.Dani, M.-C., Freixo, C., Jollois, F.-X., Nadif, M.: Unsupervised anomaly detection for aircraft condition monitoring system. In: IEEE Aeroconf 2015 (2015)Google Scholar
- 5.Martí, L., Sanchez-Pi, N., Molina, J.M., Bicharra Garcia, A.C.: YASA: yet another time series segmentation algorithm for anomaly detection in big data problems. In: Polycarpou, M., de Carvalho, A.C.P.L.F., Pan, J.-S., Woźniak, M., Quintian, H., Corchado, E. (eds.) HAIS 2014. LNCS, vol. 8480, pp. 697–708. Springer, Heidelberg (2014) CrossRefGoogle Scholar
- 6.Salvador, S., Chan, P.: Learning states and rules for detecting anomalies in time series. Appl. Intell. 23(3), 241–255 (2005)CrossRefGoogle Scholar
- 7.Keogh, E., Lin, J., Ada, F.: HOT SAX: efficiently finding the most unusual time series subsequence. In: Proceedings of the Fifth IEEE International Conference on Data Mining (ICDM 2005), pp. 226–233. IEEE Computer Society, Washington, DC (2005)Google Scholar
- 8.Breunig, M.M., Kriegel, H.-P., Ng, R.T., Sander, J.: LOF: identifying density-based local outliers. SIGMOD Rec. 29(2), 93–104 (2000)CrossRefGoogle Scholar
- 9.Chen, C., Liu, L.-M.: Joint estimation of model parameters and outlier effects in time series. J. Am. Stat. Assoc. 88, 284–297 (1993)MATHGoogle Scholar
- 10.Mnz, G., Li, S., Carle, G.: Traffic anomaly detection using k-means clustering. In: Proceedings of Leistungs-, Zuverlssigkeits-und Verlsslichkeitsbewertung von Kommunikationsnetzen und Verteilten Systemen, GI/ITG-Workshop MMBnet, September 2007Google Scholar
- 11.Gupta, M., Sharma, A.B., Chen, H., Jiang, G.: Context-aware time series anomaly detection for complex systems. In: Proceedings of the SDM Workshop (2013)Google Scholar
- 12.Rosner, B.: Percentage points for a generalized ESD many-outlier procedure. Technometrics 25(2), 165–172 (1983)CrossRefMATHGoogle Scholar
- 13.Miller, J.: Reaction time analysis with outlier exclusion: bias varies with sample size. Q. J. Exper. Psychol. 43(4), 907–912 (1991)CrossRefGoogle Scholar
- 14.Chakrabarti, K., Keogh, E., Mehrotra, S., Pazzani, M.: Locally adaptive dimensionality reduction for indexing large time series databases. ACM Trans. Database Syst. 27(2), 188–228 (2002)CrossRefGoogle Scholar
- 15.Haar, A.: Zur Theorie der orthogonalen Funktionensysteme. Math. Ann. 38–53 (1911). GermanGoogle Scholar
- 16.Yahoo! Research Webscope datasets, A Labeled Anomaly Detection Dataset, version 1.0’. http://webscope.sandbox.yahoo.com/catalog.php?datatype=s