Abstract
Multiple variable time series anomaly detection plays a significant role in fields such as AIOps and intelligent healthcare. However, time series often lack labels, and anomalies are infrequent compared to normal sequences. Moreover, high-dimensional nonlinear correlations and substantial distribution variations among variables exist in multivariate time series. These challenges make it difficult for the model to accurately model time series. While contemporary reconstruction-based models address challenges of data scarcity and time series modeling through GAN frameworks, little attention has been given to mitigating the substantial differences in variable distributions among multiple time series. This paper proposes the Variable-wise Generative Adversarial Transformer (VGAT) for achieving unsupervised anomaly detection, with WGAN as the training framework and Transformer as the fundamental structure for the Duplicator and Discriminator. Firstly, VGAT makes the Generator act as the Duplicator, reconstructing samples through direct replication, simplifying the model structure, and enhancing efficiency. Secondly, VGAT employs the Soft-DTW regularization to facilitate a more authentic replication of normal sequences by the Duplicator. Most importantly, VGAT introduces the Variable-wise Gate, dynamically calculating a gate for each variable in the multivariate time series to balance the differences in variable distributions. After VGAT generates anomaly scores, the SPOT algorithm based on Extreme Value Theory (EVT) automatically computes the threshold, enhancing the adaptability to different data of the model. VGAT demonstrates state-of-the-art (SOTA) performance, surpassing baselines on public datasets. Notably, as the dimensionality of the series increases, VGAT exhibits more significant improvements. On WADI dataset, VGAT achieves an impressive 68.19% enhancement in F1-score, showcasing remarkable progress compared to the current SOTA model.
Similar content being viewed by others
References
Wang C, Wu K, Zhou T, Yu G, Cai Z (2022) Tsagen: s‘ynthetic time series generation for KPI anomaly detection. IEEE Trans Netw Serv Manag 19(1):130–145. https://doi.org/10.1109/TNSM.2021.3098784
Liu S, Zhou B, Ding Q, Hooi B, Zhang Z, Shen H, Cheng X (2022) Time series anomaly detection with adversarial reconstruction networks. IEEE Trans Knowl Data Eng 35(4):4293–4306
Sharma DK, Dhankhar T, Agrawal G, Singh SK, Gupta D, Nebhen J, Razzak I (2021) Anomaly detection framework to prevent ddos attack in fog empowered iot networks. Ad Hoc Networks 121:102603. https://doi.org/10.1016/j.adhoc.2021.102603
Geiger A, Liu D, Alnegheimish S, Cuesta-Infante A, Veeramachaneni K (2020) Tadgan: time series anomaly detection using generative adversarial networks. In: 2020 IEEE international conference on big data (Big Data), IEEE, pp 33–43
Li D, Chen D, Jin B, Shi L, Goh J, Ng S-K (2019) Mad-gan: multivariate anomaly detection for time series data with generative adversarial networks. In: Artificial neural networks and machine learning–ICANN 2019: text and time series: 28th international conference on artificial neural networks, Munich, Germany, September 17–19, 2019, Proceedings, Part IV, Springer, pp 703–716
Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2020) Generative adversarial networks. Commun of the ACM 63(11):139–144
Li G, Zhou X, Sun J, Yu X, Han Y, Jin L, Li W, Wang T, Li S (2021) opengauss: an autonomous database system. Proceedings of the VLDB Endowment 14(12):3028–3042
Hundman K, Constantinou V, Laporte C, Colwell I, Soderstrom T (2018) Detecting spacecraft anomalies using lstms and nonparametric dynamic thresholding. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pp 387–395
Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780
Luo X, Jiang Y, Wang E (2022) Men X (2022) Anomaly detection by using a combination of generative adversarial networks and convolutional autoencoders. EURASIP J Adv Signal Process 1:112. https://doi.org/10.1186/s13634-022-00943-7
Huong TT, Ta PB, Long DM, Luong TD, Dan NM, Quang LA, Cong LT, Thang BD, Tran KP (2021) Detecting cyberattacks using anomaly detection in industrial control systems: a federated learning approach. Comput Ind 132:103509. https://doi.org/10.1016/j.compind.2021.103509
LeCun Y, Bengio Y, et al. (1995) Convolutional networks for images, speech, and time series. The handbook of brain theory and neural networks 3361(10):1995
Siffer A, Fouque P-A, Termier A, Largouet C (2017) Anomaly detection in streams with extreme value theory. In: Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pp 1067–1075
Tuli S, Casale G, Jennings NR (2022) Tranad: Deep transformer networks for anomaly detection in multivariate time series data. Proc VLDB Endow 15(6):1201–1214
Arjovsky M, Chintala S, Bottou L (2017) Wasserstein generative adversarial networks. In: International conference on machine learning, PMLR, pp 214–223
Cuturi M, Blondel M (2017) Soft-dtw: a differentiable loss function for time-series. In: International conference on machine learning, PMLR pp 894–903
Aggarwal, CC (2017) An Introduction to Outlier Analysis, Springer, Cham, pp 1–34. https://doi.org/10.1007/978-3-319-47578-3_1
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. Advances in neural information processing systems 30
Peng C, Hu W, Wang L (2022) Spectrum anomaly detection based on spatio-temporal network prediction. Electronics 11(11):1770
Qin H, Yan M, Ji H (2021) Application of controller area network (CAN) bus anomaly detection based on time series prediction. Veh Commun 27:100291
Sun J, Wang J, Hao Z, Zhu M, Sun H, Wei M, Dong K (2022) Ac-lstm: Anomaly state perception of infrared point targets based on cnn+ lstm. Remote Sens 14(13):3221
Su Y, Zhao Y, Niu C, Liu R, Sun W, Pei D (2019) Robust anomaly detection for multivariate time series through stochastic recurrent neural network. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp 2828–2837
Cheong M-S, Wu M-C, Huang S-H (2021) Interpretable stock anomaly detection based on spatio-temporal relation networks with genetic algorithm. IEEE Access 9:68302–68319
Chaabene S, Bouaziz B, Boudaya A, Hökelmann A, Ammar A, Chaari L (2021) Convolutional neural network for drowsiness detection using eeg signals. Sensors 21(5):1734
Cherdo Y, Miramond B, Pegatoquet A, Vallauri A (2023) Unsupervised anomaly detection for cars can sensors time series using small recurrent and convolutional neural networks. Sensors 23(11):5013
Elman JL (1990) Finding structure in time. Cogn Sci 14(2):179–211
Du H, Duan Z (2022) Finder: a novel approach of change point detection for multivariate time series. Appl Intell 52(3):2496–2509
Yin C, Zhang S, Wang J, Xiong NN (2020) Anomaly detection based on convolutional recurrent autoencoder for iot time series. IEEE Trans Syst Man Cyber Syst 52(1):112–122
Chen H, Liu H, Chu X, Liu Q, Xue D (2021) Anomaly detection and critical scada parameters identification for wind turbines based on lstm-ae neural network. Renew Energy 172:829–840
Liu P, Sun X, Han Y, He Z, Zhang W, Wu C (2022) Arrhythmia classification of lstm autoencoder based on time series anomaly detection. Biomed Signal Process Control 71:103228
Maleki S, Maleki S, Jennings NR (2021) Unsupervised anomaly detection with lstm autoencoders using statistical data-filtering. Appl Soft Comput 108:107443
Li Y, Peng X, Zhang J, Li Z, Wen M (2021) Dct-gan: dilated convolutional transformer-based gan for time series anomaly detection. IEEE Trans Knowl Data Eng
Wang P, Chen P, Yuan Y, Liu D, Huang Z, Hou X, Cottrell G (2018) Understanding convolution for semantic segmentation. In: 2018 IEEE winter conference on applications of computer vision (WACV), IEEE, pp 1451–1460
Cheng W, Ma T, Wang X, Wang G (2022) Anomaly detection for internet of things time series data using generative adversarial networks with attention mechanism in smart agriculture. Frontiers in Plant Sci 13:890563
Kong L, Yu J, Tang D, Song Y, Han D (2023) Multivariate time series anomaly detection with generative adversarial networks based on active distortion transformer. IEEE Sensors J
Kong F, Li J, Jiang B, Wang H, Song H (2021) Integrated generative model for industrial anomaly detection via bidirectional lstm and attention mechanism. IEEE Trans Ind Inform 19(1):541–550
Zhang Z, Li W, Ding W, Zhang L, Lu Q, Hu P, Lu Gui T, Lu S (2023) STAD-GAN: unsupervised anomaly detection on multivariate time series with self-training generative adversarial networks. ACM Trans Knowl Discov Data 17(5):71–17118. https://doi.org/10.1145/3572780
Lian Y, Geng Y Tian T (1891) Anomaly detection method for multivariate time series data of oil and gas stations based on digital twin and mtad-gan. Appl Sci 13(3):1891
Choi Y, Lim H, Choi H, Kim I-J (2020) Gan-based anomaly detection and localization of multivariate time series data for power plant. In: 2020 IEEE international conference on big data and smart computing (BigComp), IEEE, pp 71–74
Mao S, Guo J, Gu T, Ma Z () Dis-ae-lstm: Generative adversarial networks for anomaly detection of time series data. In: 2020 international conference on artificial intelligence and computer engineering (ICAICE), IEEE, pp 330–336
Hu J, Shen L, Sun, G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132–7141
Ahmad S, Lavin A, Purdy S, Agha Z (2017) Unsupervised real-time anomaly detection for streaming data. Neurocomputing 262:134–147
Nakamura T, Imamura M, Mercer R, Keogh E (2020) Merlin: parameter-free discovery of arbitrary length anomalies in massive time series archives. In: 2020 IEEE international conference on data mining (ICDM), IEEE, pp 1190–1195
Dau HA, Keogh E, Kamgar K, Yeh C-CM, Zhu Y, Gharghabi S, Ratanamahatana CA, Yanping, Hu B, Begum N, Bagnall A, Mueen A, Batista G, Hexagon-ML (2018) The UCR time series classification archive. https://www.cs.ucr.edu/~eamonn/time_series_data_2018/
Moody GB, Mark RG (2001) The impact of the mit-bih arrhythmia database. IEEE Eng Med Biol Mag 20(3):45–50
Ahmed CM, Palleti VR, Mathur AP (2017) Wadi: a water distribution testbed for research in the design of secure cyber physical systems. In: Proceedings of the 3rd international workshop on cyber-physical systems for smart water networks, pp 25–28
Zong B, Song Q, Min MR, Cheng W, Lumezanu C, Cho D, Chen H (2018) Deep autoencoding gaussian mixture model for unsupervised anomaly detection. In: International conference on learning representations
Zhang C, Song D, Chen Y, Feng X, Lumezanu C, Cheng W, Ni J, Zong B, Chen H, Chawla NV (2019) A deep neural network for unsupervised anomaly detection and diagnosis in multivariate time series data. Proceedings of the AAAI Conference on Artificial Intelligence 33:1409–1416
Audibert J, Michiardi P, Guyard F, Marti S, Zuluaga MA (2020) Usad: unsupervised anomaly detection on multivariate time series. In: Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pp 3395–3404
Zhao H, Wang Y, Duan J, Huang C, Cao D, Tong Y, Xu B, Bai J, Tong J, Zhang Q (2020) Multivariate time-series anomaly detection via graph attention network. In: 2020 IEEE international conference on data mining (ICDM), IEEE, pp 841–850
Zhang Y, Chen Y, Wang J, Pan Z (2021) Unsupervised deep anomaly detection for multi-sensor time-series signals. IEEE Trans Knowl Data Eng
Deng A, Hooi B (2021) Graph neural network-based anomaly detection in multivariate time series. Proceedings of the AAAI conference on artificial intelligence 35:4027–4035
Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, Killeen T, Lin Z, Gimelshein N, Antiga L, et al. (2019) Pytorch: an imperative style, high-performance deep learning library. Adv Neural Inform Process Syst 32
Biewald L et al (2020) Experiment tracking with weights and biases. Software available from wandb. com 2:233
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Yang, X., Li, H., Feng, X. et al. Variable-wise generative adversarial transformer in multivariate time series anomaly detection. Appl Intell 53, 28745–28767 (2023). https://doi.org/10.1007/s10489-023-05029-x
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-023-05029-x