Skip to main content
SpringerLink
Log in

Variable-wise generative adversarial transformer in multivariate time series anomaly detection

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Multiple variable time series anomaly detection plays a significant role in fields such as AIOps and intelligent healthcare. However, time series often lack labels, and anomalies are infrequent compared to normal sequences. Moreover, high-dimensional nonlinear correlations and substantial distribution variations among variables exist in multivariate time series. These challenges make it difficult for the model to accurately model time series. While contemporary reconstruction-based models address challenges of data scarcity and time series modeling through GAN frameworks, little attention has been given to mitigating the substantial differences in variable distributions among multiple time series. This paper proposes the Variable-wise Generative Adversarial Transformer (VGAT) for achieving unsupervised anomaly detection, with WGAN as the training framework and Transformer as the fundamental structure for the Duplicator and Discriminator. Firstly, VGAT makes the Generator act as the Duplicator, reconstructing samples through direct replication, simplifying the model structure, and enhancing efficiency. Secondly, VGAT employs the Soft-DTW regularization to facilitate a more authentic replication of normal sequences by the Duplicator. Most importantly, VGAT introduces the Variable-wise Gate, dynamically calculating a gate for each variable in the multivariate time series to balance the differences in variable distributions. After VGAT generates anomaly scores, the SPOT algorithm based on Extreme Value Theory (EVT) automatically computes the threshold, enhancing the adaptability to different data of the model. VGAT demonstrates state-of-the-art (SOTA) performance, surpassing baselines on public datasets. Notably, as the dimensionality of the series increases, VGAT exhibits more significant improvements. On WADI dataset, VGAT achieves an impressive 68.19% enhancement in F1-score, showcasing remarkable progress compared to the current SOTA model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Algorithm 1
Algorithm 2
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Wang C, Wu K, Zhou T, Yu G, Cai Z (2022) Tsagen: s‘ynthetic time series generation for KPI anomaly detection. IEEE Trans Netw Serv Manag 19(1):130–145. https://doi.org/10.1109/TNSM.2021.3098784

    Article  Google Scholar 

  2. Liu S, Zhou B, Ding Q, Hooi B, Zhang Z, Shen H, Cheng X (2022) Time series anomaly detection with adversarial reconstruction networks. IEEE Trans Knowl Data Eng 35(4):4293–4306

    Article  Google Scholar 

  3. Sharma DK, Dhankhar T, Agrawal G, Singh SK, Gupta D, Nebhen J, Razzak I (2021) Anomaly detection framework to prevent ddos attack in fog empowered iot networks. Ad Hoc Networks 121:102603. https://doi.org/10.1016/j.adhoc.2021.102603

    Article  Google Scholar 

  4. Geiger A, Liu D, Alnegheimish S, Cuesta-Infante A, Veeramachaneni K (2020) Tadgan: time series anomaly detection using generative adversarial networks. In: 2020 IEEE international conference on big data (Big Data), IEEE, pp 33–43

  5. Li D, Chen D, Jin B, Shi L, Goh J, Ng S-K (2019) Mad-gan: multivariate anomaly detection for time series data with generative adversarial networks. In: Artificial neural networks and machine learning–ICANN 2019: text and time series: 28th international conference on artificial neural networks, Munich, Germany, September 17–19, 2019, Proceedings, Part IV, Springer, pp 703–716

  6. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2020) Generative adversarial networks. Commun of the ACM 63(11):139–144

    Article  MathSciNet  Google Scholar 

  7. Li G, Zhou X, Sun J, Yu X, Han Y, Jin L, Li W, Wang T, Li S (2021) opengauss: an autonomous database system. Proceedings of the VLDB Endowment 14(12):3028–3042

    Article  Google Scholar 

  8. Hundman K, Constantinou V, Laporte C, Colwell I, Soderstrom T (2018) Detecting spacecraft anomalies using lstms and nonparametric dynamic thresholding. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pp 387–395

  9. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  Google Scholar 

  10. Luo X, Jiang Y, Wang E (2022) Men X (2022) Anomaly detection by using a combination of generative adversarial networks and convolutional autoencoders. EURASIP J Adv Signal Process 1:112. https://doi.org/10.1186/s13634-022-00943-7

    Article  Google Scholar 

  11. Huong TT, Ta PB, Long DM, Luong TD, Dan NM, Quang LA, Cong LT, Thang BD, Tran KP (2021) Detecting cyberattacks using anomaly detection in industrial control systems: a federated learning approach. Comput Ind 132:103509. https://doi.org/10.1016/j.compind.2021.103509

    Article  Google Scholar 

  12. LeCun Y, Bengio Y, et al. (1995) Convolutional networks for images, speech, and time series. The handbook of brain theory and neural networks 3361(10):1995

  13. Siffer A, Fouque P-A, Termier A, Largouet C (2017) Anomaly detection in streams with extreme value theory. In: Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pp 1067–1075

  14. Tuli S, Casale G, Jennings NR (2022) Tranad: Deep transformer networks for anomaly detection in multivariate time series data. Proc VLDB Endow 15(6):1201–1214

    Article  Google Scholar 

  15. Arjovsky M, Chintala S, Bottou L (2017) Wasserstein generative adversarial networks. In: International conference on machine learning, PMLR, pp 214–223

  16. Cuturi M, Blondel M (2017) Soft-dtw: a differentiable loss function for time-series. In: International conference on machine learning, PMLR pp 894–903

  17. Aggarwal, CC (2017) An Introduction to Outlier Analysis, Springer, Cham, pp 1–34. https://doi.org/10.1007/978-3-319-47578-3_1

  18. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. Advances in neural information processing systems 30

  19. Peng C, Hu W, Wang L (2022) Spectrum anomaly detection based on spatio-temporal network prediction. Electronics 11(11):1770

    Article  Google Scholar 

  20. Qin H, Yan M, Ji H (2021) Application of controller area network (CAN) bus anomaly detection based on time series prediction. Veh Commun 27:100291

    Google Scholar 

  21. Sun J, Wang J, Hao Z, Zhu M, Sun H, Wei M, Dong K (2022) Ac-lstm: Anomaly state perception of infrared point targets based on cnn+ lstm. Remote Sens 14(13):3221

    Article  Google Scholar 

  22. Su Y, Zhao Y, Niu C, Liu R, Sun W, Pei D (2019) Robust anomaly detection for multivariate time series through stochastic recurrent neural network. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp 2828–2837

  23. Cheong M-S, Wu M-C, Huang S-H (2021) Interpretable stock anomaly detection based on spatio-temporal relation networks with genetic algorithm. IEEE Access 9:68302–68319

    Article  Google Scholar 

  24. Chaabene S, Bouaziz B, Boudaya A, Hökelmann A, Ammar A, Chaari L (2021) Convolutional neural network for drowsiness detection using eeg signals. Sensors 21(5):1734

    Article  Google Scholar 

  25. Cherdo Y, Miramond B, Pegatoquet A, Vallauri A (2023) Unsupervised anomaly detection for cars can sensors time series using small recurrent and convolutional neural networks. Sensors 23(11):5013

    Article  Google Scholar 

  26. Elman JL (1990) Finding structure in time. Cogn Sci 14(2):179–211

    Article  Google Scholar 

  27. Du H, Duan Z (2022) Finder: a novel approach of change point detection for multivariate time series. Appl Intell 52(3):2496–2509

    Article  Google Scholar 

  28. Yin C, Zhang S, Wang J, Xiong NN (2020) Anomaly detection based on convolutional recurrent autoencoder for iot time series. IEEE Trans Syst Man Cyber Syst 52(1):112–122

    Article  Google Scholar 

  29. Chen H, Liu H, Chu X, Liu Q, Xue D (2021) Anomaly detection and critical scada parameters identification for wind turbines based on lstm-ae neural network. Renew Energy 172:829–840

    Article  Google Scholar 

  30. Liu P, Sun X, Han Y, He Z, Zhang W, Wu C (2022) Arrhythmia classification of lstm autoencoder based on time series anomaly detection. Biomed Signal Process Control 71:103228

    Article  Google Scholar 

  31. Maleki S, Maleki S, Jennings NR (2021) Unsupervised anomaly detection with lstm autoencoders using statistical data-filtering. Appl Soft Comput 108:107443

    Article  Google Scholar 

  32. Li Y, Peng X, Zhang J, Li Z, Wen M (2021) Dct-gan: dilated convolutional transformer-based gan for time series anomaly detection. IEEE Trans Knowl Data Eng

  33. Wang P, Chen P, Yuan Y, Liu D, Huang Z, Hou X, Cottrell G (2018) Understanding convolution for semantic segmentation. In: 2018 IEEE winter conference on applications of computer vision (WACV), IEEE, pp 1451–1460

  34. Cheng W, Ma T, Wang X, Wang G (2022) Anomaly detection for internet of things time series data using generative adversarial networks with attention mechanism in smart agriculture. Frontiers in Plant Sci 13:890563

    Article  Google Scholar 

  35. Kong L, Yu J, Tang D, Song Y, Han D (2023) Multivariate time series anomaly detection with generative adversarial networks based on active distortion transformer. IEEE Sensors J

  36. Kong F, Li J, Jiang B, Wang H, Song H (2021) Integrated generative model for industrial anomaly detection via bidirectional lstm and attention mechanism. IEEE Trans Ind Inform 19(1):541–550

    Article  Google Scholar 

  37. Zhang Z, Li W, Ding W, Zhang L, Lu Q, Hu P, Lu Gui T, Lu S (2023) STAD-GAN: unsupervised anomaly detection on multivariate time series with self-training generative adversarial networks. ACM Trans Knowl Discov Data 17(5):71–17118. https://doi.org/10.1145/3572780

    Article  Google Scholar 

  38. Lian Y, Geng Y Tian T (1891) Anomaly detection method for multivariate time series data of oil and gas stations based on digital twin and mtad-gan. Appl Sci 13(3):1891

  39. Choi Y, Lim H, Choi H, Kim I-J (2020) Gan-based anomaly detection and localization of multivariate time series data for power plant. In: 2020 IEEE international conference on big data and smart computing (BigComp), IEEE, pp 71–74

  40. Mao S, Guo J, Gu T, Ma Z () Dis-ae-lstm: Generative adversarial networks for anomaly detection of time series data. In: 2020 international conference on artificial intelligence and computer engineering (ICAICE), IEEE, pp 330–336

  41. Hu J, Shen L, Sun, G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132–7141

  42. Ahmad S, Lavin A, Purdy S, Agha Z (2017) Unsupervised real-time anomaly detection for streaming data. Neurocomputing 262:134–147

    Article  Google Scholar 

  43. Nakamura T, Imamura M, Mercer R, Keogh E (2020) Merlin: parameter-free discovery of arbitrary length anomalies in massive time series archives. In: 2020 IEEE international conference on data mining (ICDM), IEEE, pp 1190–1195

  44. Dau HA, Keogh E, Kamgar K, Yeh C-CM, Zhu Y, Gharghabi S, Ratanamahatana CA, Yanping, Hu B, Begum N, Bagnall A, Mueen A, Batista G, Hexagon-ML (2018) The UCR time series classification archive. https://www.cs.ucr.edu/~eamonn/time_series_data_2018/

  45. Moody GB, Mark RG (2001) The impact of the mit-bih arrhythmia database. IEEE Eng Med Biol Mag 20(3):45–50

    Article  Google Scholar 

  46. Ahmed CM, Palleti VR, Mathur AP (2017) Wadi: a water distribution testbed for research in the design of secure cyber physical systems. In: Proceedings of the 3rd international workshop on cyber-physical systems for smart water networks, pp 25–28

  47. Zong B, Song Q, Min MR, Cheng W, Lumezanu C, Cho D, Chen H (2018) Deep autoencoding gaussian mixture model for unsupervised anomaly detection. In: International conference on learning representations

  48. Zhang C, Song D, Chen Y, Feng X, Lumezanu C, Cheng W, Ni J, Zong B, Chen H, Chawla NV (2019) A deep neural network for unsupervised anomaly detection and diagnosis in multivariate time series data. Proceedings of the AAAI Conference on Artificial Intelligence 33:1409–1416

  49. Audibert J, Michiardi P, Guyard F, Marti S, Zuluaga MA (2020) Usad: unsupervised anomaly detection on multivariate time series. In: Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pp 3395–3404

  50. Zhao H, Wang Y, Duan J, Huang C, Cao D, Tong Y, Xu B, Bai J, Tong J, Zhang Q (2020) Multivariate time-series anomaly detection via graph attention network. In: 2020 IEEE international conference on data mining (ICDM), IEEE, pp 841–850

  51. Zhang Y, Chen Y, Wang J, Pan Z (2021) Unsupervised deep anomaly detection for multi-sensor time-series signals. IEEE Trans Knowl Data Eng

  52. Deng A, Hooi B (2021) Graph neural network-based anomaly detection in multivariate time series. Proceedings of the AAAI conference on artificial intelligence 35:4027–4035

  53. Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, Killeen T, Lin Z, Gimelshein N, Antiga L, et al. (2019) Pytorch: an imperative style, high-performance deep learning library. Adv Neural Inform Process Syst 32

  54. Biewald L et al (2020) Experiment tracking with weights and biases. Software available from wandb. com 2:233

Download references

Author information

Authors and Affiliations

  1. School of Automation, Nanjing University of Science and Technology, Nanjing, 210094, Jiangsu Province, China

    Xuekang Yang & Hui Li

  2. School of Information & Electronic Engineering, Zhejiang Gongshang University, Hangzhou, 310018, Zhejiang Province, China

    Xingyu Feng

  3. School of Automation, Nanjing University of Aeronautics and Astronautics, Nanjing, 211106, Jiangsu Province, China

    Zixiong Jin

Authors
  1. Xuekang Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hui Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xingyu Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zixiong Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hui Li.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, X., Li, H., Feng, X. et al. Variable-wise generative adversarial transformer in multivariate time series anomaly detection. Appl Intell 53, 28745–28767 (2023). https://doi.org/10.1007/s10489-023-05029-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-023-05029-x

Keywords

Search

Navigation