Advertisement

An Unsupervised Anomaly Detection Algorithm for Time Series Big Data

  • Wenqing Wang
  • Junpeng BaoEmail author
  • Hui He
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 911)

Abstract

Many time series anomaly detection algorithms are hard to be applied in real scenarios for two reasons. Firstly some of them are supervised since training data is required to define the normal behavior, but it is expensive to annotate the normal part for large volume data. Secondly, many algorithms are parameter-laden, which are hard to be generalized to different dataset. This paper is motivated to overcome these disadvantages. It is believed that a normal behavior is a subsequence which is similar to some subsequences in a time series while an anomaly is a subsequence which is distinct from the others. In order to improve the efficiency of searching anomaly, we first select candidate anomalies rather than check all subsequences. We roughly distinguish the candidate anomalies from normal subsequences by transforming each subsequence into a string. If a string corresponds to only one subsequence, then it is a candidate anomaly. And the subsequences of the same string represent a kind of normal behavior. Secondly, similarity threshold is calculated according to the similarity between normal behaviors. If the similarity between a candidate anomaly and its nearest neighbor is lower than the threshold, then this candidate is determined to be anomalous. We conduct extensive experiments on benchmark datasets from diverse domains and compare our method with the state-of-the-art method. The empirical results show that our method can reach high detection rate in an unsupervised and parameter-lite manner.

Keywords

Unsupervised anomaly detection Parameter-lite Real-valued time series 

References

  1. 1.
    Keogh, E., Lin, J., Fu, A.: HOT SAX: finding the most unusual time series subsequence: algorithms and applications. In: Proceedings of International Conference on Data Mining, pp. 226–233 (2005)Google Scholar
  2. 2.
    Jones, M., Nikovski, D., Imamura, M., et al.: Exemplar learning for extremely efficient anomaly detection in real-valued time series. Data Min. Knowl. Disc. 30(6), 1427–1454 (2016)MathSciNetCrossRefGoogle Scholar
  3. 3.
  4. 4.
    Liu, B., Chen, H., Sharma, A., et al.: Modeling heterogeneous time series dynamics to profile big sensor data in complex physical systems. In: Proceedings of IEEE International Conference on Big Data, pp. 631–638 (2013)Google Scholar
  5. 5.
    Keogh, E., Lonardi, S., Ratanamahatana, C.: Towards parameter-free data mining. In: Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 206–215 (2004)Google Scholar
  6. 6.
    Ma, J., Perkins, S.: Online novelty detection on temporal sequences. In: Proceedings of ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 613–618 (2003)Google Scholar
  7. 7.
    Pankaj, M., Lovekesh, V., Gautam, S., et al.: Long short term memory networks for anomaly detection in time series. In: Proceedings of European Symposium on Artificial Neural Networks, pp. 89–94 (2015)Google Scholar
  8. 8.
    Appice, A., Guccione, P., Malerba, D., et al.: Dealing with temporal and spatial correlations to classify outliers in geophysical data streams. Inf. Sci. 285(1), 162–180 (2014)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Laptev, N., Amizadeh, S., Flint, I.: Generic and scalable framework for automated time-series anomaly detection. In: Proceedings of ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1939–1947 (2015)Google Scholar
  10. 10.
    Zheng, D., Li, F., Zhao, T.: Self-adaptive statistical process control for anomaly detection in time series. Expert Syst. Appl. 57(1), 324–336 (2016)CrossRefGoogle Scholar
  11. 11.
    Wang, H., Tang, M., Park, Y., et al.: Locality statistics for anomaly detection in time series of graphs. IEEE Trans. Signal Process. 62(3), 703–717 (2014)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Ma, J., Sun, L., Wang, H., et al.: Supervised anomaly detection in uncertain pseudoperiodic data streams. ACM Trans. Internet Technol. 16(1), 4–24 (2016)CrossRefGoogle Scholar
  13. 13.
    Burbeck, K., Nadjm-Tehrani, S.: Adaptive real-time anomaly detection with incremental clustering. Inf. Secur. Techn. Rep. 12(1), 56–67 (2007)CrossRefGoogle Scholar
  14. 14.
    Izakian, H., Pedrycz, W.: Anomaly detection and characterization in spatial time series data: a cluster-centric approach. IEEE Trans. Fuzzy Syst. 22(6), 1612–1624 (2014)CrossRefGoogle Scholar
  15. 15.
    Hsiao, K., Xu, K., Calder, J., et al.: Multicriteria similarity-based anomaly detection using pareto depth analysis. IEEE Trans. Neural Netw. Learn. Syst. 27(6), 1307–1321 (2016)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  1. 1.Xi’an Jiaotong UniversityXi’anPeople’s Republic of China

Personalised recommendations