Skip to main content
SpringerLink
Log in

Recurrent auto-encoder with multi-resolution ensemble and predictive coding for multivariate time-series anomaly detection

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

As large-scale time-series data can easily be found in real-world applications, multivariate time-series anomaly detection has played an essential role in diverse industries. It enables productivity improvement and maintenance cost reduction by preventing malfunctions and detecting anomalies based on time-series data. However, multivariate time-series anomaly detection is challenging because real-world time-series data exhibit complex temporal dependencies. For this task, it is crucial to learn a rich representation that effectively contains the nonlinear temporal dynamics of normal behavior. In this study, we propose an unsupervised multivariate time-series anomaly detection model named RAE-MEPC which learns informative normal representations based on multi-resolution ensemble reconstruction and predictive coding. We introduce multi-resolution ensemble encoding to capture the multi-scale dependency from the input time series. The encoder hierarchically aggregates the multi-scale temporal features extracted from the sub-encoders with different encoding lengths. From these encoded features, the reconstruction decoder reconstructs the input time series based on multi-resolution ensemble decoding where lower-resolution information helps to decode sub-decoders with higher-resolution outputs. Predictive coding is further introduced to encourage the model to learn more temporal dependencies of the time series. Experiments on real-world benchmark datasets show that the proposed model outperforms the benchmark models for multivariate time-series anomaly detection.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Algorithm 1
Algorithm 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Chandola V, Banerjee A, Kumar V (2009) Anomaly detection: A survey. ACM computing surveys (CSUR) 41(3):1–58

    Article  Google Scholar 

  2. Chalapathy R, Chawla S (2019) Deep learning for anomaly detection: A survey. arXiv:1901.03407

  3. Lee G-Y, Kim M, Quan Y-J, Kim M-S, Kim TJY, Yoon H-S, Min S, Kim D-H, Mun J-W, Oh JW et al (2018) Machine health management in smart factory: A review. J Mech Sci Technol 32(3):987–1009

    Article  Google Scholar 

  4. Laubscher R (2019) Time-series forecasting of coal-fired power plant reheater metal temperatures using encoder-decoder recurrent neural networks. Energy 189:116187

    Article  Google Scholar 

  5. Pokhrel NR, Rodrigo H, Tsokos CP et al (2017) Cybersecurity: Time series predictive modeling of vulnerabilities of desktop operating system using linear and non-linear approach. J Infor Secur 8(04):362

    Article  Google Scholar 

  6. Wang Y, Perry M, Whitlock D, Sutherland JW (2020) Detecting anomalies in time series data from a manufacturing system using recurrent neural networks. J Manufac Syst

  7. Shen L, Li Z, Kwok J (2020) Timeseries anomaly detection using temporal hierarchical one-class network. Advances in Neural Information Processing Systems 33:13016–13026

    Google Scholar 

  8. Canizo M, Triguero I, Conde A, Onieva E (2019) Multi-head cnn-rnn for multi-time series anomaly detection: An industrial case study. Neurocomputing 363:246–260

    Article  Google Scholar 

  9. Su Y, Zhao Y, Niu C, Liu R, Sun W, Pei D (2019) Robust anomaly detection for multivariate time series through stochastic recurrent neural network. In: Proc 25th ACM SIGKDD Int Conf Knowledge Discovery & Data Mining pp. 2828–2837

  10. Geiger A, Liu D, Alnegheimish S, Cuesta-Infante A, Veeramachaneni K (2020) Tadgan: Time series anomaly detection using generative adversarial networks. In: 2020 IEEE International Conference on Big Data (Big Data) pp. 33–43

  11. Malhotra P, Vig L, Shroff G, Agarwal P (2015) Long short term memory networks for anomaly detection in time series. In: Proceedings, vol.89, pp. 89–94

  12. Hundman K, Constantinou V, Laporte C, Colwell I, Soderstrom T (2018) Detecting spacecraft anomalies using lstms and nonparametric dynamic thresholding. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining pp. 387–395

  13. Shalyga D, Filonov P, Lavrentyev A (2018) Anomaly detection for water treatment system based on neural network with automatic architecture optimization. arXiv:1807.07282

  14. Kravchik M, Shabtai A (2018) Detecting cyber attacks in industrial control systems using convolutional neural networks. In: Proceedings of the 2018 Workshop on Cyber-Physical Systems Security and PrivaCy pp. 72–83

  15. Lai G, Chang W-C, Yang Y, Liu H (2018) Modeling long-and short-term temporal patterns with deep neural networks. In: The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval pp. 95–104

  16. Malhotra P, Ramakrishnan A, Anand G, Vig L, Agarwal P, Shroff G (2016) Lstm-based encoder-decoder for multi-sensor anomaly detection. arXiv:1607.00148

  17. Park D, Hoshi Y, Kemp CC (2018) A multimodal anomaly detector for robot-assisted feeding using an lstm-based variational autoencoder. IEEE Robotics and Automation Letters

  18. Zong B, Song Q, Min MR, Cheng W, Lumezanu C, Cho D, Chen H (2018) Deep autoencoding gaussian mixture model for unsupervised anomaly detection. In: International Conference on Learning Representations

  19. Yoo Y-H, Kim U-H, Kim J-H (2019) Recurrent reconstructive network for sequential anomaly detection. IEEE transactions on cybernetics

  20. Kieu T, Yang B, Guo C, Jensen CS (2019) Outlier detection for time series with recurrent autoencoder ensembles. In: IJCAI, pp. 2725–2732

  21. Shen L, Yu Z, Ma Q, Kwok JT (2021) Time series anomaly detection with multiresolution ensemble decoding. In: Proceedings of the AAAI Conference on Artificial Intelligence pp. 9567–9575

  22. Li D, Chen D, Jin B, Shi L, Goh J, Ng S-K (2019) Mad-gan: Multivariate anomaly detection for time series data with generative adversarial networks. In: International Conference on Artificial Neural Networks, Springer pp. 703–716

  23. Bashar MA, Nayak R (2020) Tanogan: Time series anomaly detection with generative adversarial networks. In: 2020 IEEE Symposium Series on Computational Intelligence (SSCI), pp. 1778–1785

  24. Liu S, Zhou B, Ding Q, Hooi B, bo Zhang, Z, Shen H, Cheng X (2022) Time series anomaly detection with adversarial reconstruction networks. IEEE Transactions on Knowledge and Data Engineering

  25. El Hihi S, Bengio Y (1996) Hierarchical recurrent neural networks for long-term dependencies. In: Advances in Neural Information Processing Systems pp. 493–499

  26. Chung J, Ahn S, Bengio Y (2016) Hierarchical multiscale recurrent neural networks. arXiv:1609.01704

  27. Ma Q, Lin Z, Chen E, Cottrell G (2020) Temporal pyramid recurrent neural network. In: Proceedings of the AAAI Conference on Artificial Intelligence pp. 5061–5068

  28. Srivastava N, Mansimov E, Salakhudinov R (2015) Unsupervised learning of video representations using lstms. In: International Conference on Machine Learning, PMLR pp. 843–852

  29. Mathieu M, Couprie C, LeCun Y (2015) Deep multi-scale video prediction beyond mean square error. arXiv:1511.05440

  30. Lotter W, Kreiman G, Cox D (2016) Deep predictive coding networks for video prediction and unsupervised learning.arXiv:1605.08104

  31. Han T, Xie W, Zisserman A (2019) Video representation learning by dense predictive coding. In: Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops pp. 0–0

  32. Vincent P, Larochelle H, Bengio Y, Manzagol P-A (2008) Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th International Conference on Machine Learning pp. 1096–1103

  33. Sakoe H, Chiba S (1978) Dynamic programming algorithm optimization for spoken word recognition. IEEE transactions on acoustics, speech, and signal processing 26(1):43–49

    Article  MATH  Google Scholar 

  34. Cuturi M, Blondel M (2017) Soft-dtw: a differentiable loss function for time-series. In: International Conference on Machine Learning,PMLR pp. 894–903

  35. Keogh E, Lin J, Fu A (2005) Hot sax: Efficiently finding the most unusual time series subsequence. In: Fifth IEEE International Conference on Data Mining (ICDM’05)IEEE p. 8

  36. Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. arXiv:1412.6980

  37. Wong L, Liu D, Berti-Equille L, Alnegheimish S, Veeramachaneni K, (2022) AER: Auto-encoder with regression for time series anomaly detection.arXiv preprint arXiv:2212.13558

  38. Huang D, Shen L, Yu Z, Zheng Z, Huang M, Ma Q (2022) Efficient time series anomaly detection by multiresolution self-supervised discriminative network. Neurocomputing 491:261–272

  39. Yan J, Meng Y, Lu L, Li L (2017) Industrial big data in an industry 4.0 environment: Challenges, schemes, and applications for predictive maintenance. IEEE Access 5:23484–23491

  40. Hashemian HM, Bean WC (2011) State-of-the-art predictive maintenance techniques. IEEE Transactions on Instrumentation and Measurement 60(10):3480–3492

    Article  Google Scholar 

  41. Li T, Comer ML, Delp EJ, Desai SR, Mathieson JL, Foster RH, Chan MW (2020) Anomaly scoring for prediction-based anomaly detection in time series. 2020 IEEE Aerospace Conference pp. 1–7

  42. Kim J, Kang H, Kang P (2023) Time-series anomaly detection with stacked Transformer representations and 1D convolutional network. Engineering Applications of Artificial Intelligence 120:105964

    Article  Google Scholar 

  43. Blázquez-García A, Conde A, Mori U, Lozano JA (2021) Water leak detection using self-supervised time series classification. Information Sciences 574:528–541

  44. Hayashi T, Cimr D, Studnička F, Fujita H, Bušovskỳ D, Cimler R (2022) OCSTN: One-class time-series classification approach using a signal transformation network into a goal signal. Information Sciences 614:71–86

Download references

Acknowledgements

This work was supported by the National Research Foundation of Korea (NRF) grants funded by the Korea government (MSIT) (No. NRF-2022R1A2C2005455). This work was also supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. 2021-0-00034, Clustering technologies of fragmented data for time-based data analysis)

Author information

Authors and Affiliations

  1. School of Industrial & Management Engineering, Korea University, 145, Anam-Ro, Seongbuk-Gu, Seoul, Republic of Korea

    Heejeong Choi, Subin Kim & Pilsung Kang

Authors
  1. Heejeong Choi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Subin Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pilsung Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pilsung Kang.

Ethics declarations

Conflicts of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Choi, H., Kim, S. & Kang, P. Recurrent auto-encoder with multi-resolution ensemble and predictive coding for multivariate time-series anomaly detection. Appl Intell 53, 25330–25342 (2023). https://doi.org/10.1007/s10489-023-04764-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-023-04764-5

Keywords

Search

Navigation