Abstract
In rapidly evolving industrial IT systems, the integration of sensor networks has become the cornerstone of operational workflows. These networks diligently collect data in the form of time series, where the format intertwines closely with temporal dependencies, crucial for anomaly detection models. Hence, the extraction of information in the time domain is advantageous for anomaly detection. To address this, we adopt a method of time series decomposition to delve into seasonality, trend, and residual components. Additionally, we design a novel algorithm that combines Transformer architecture with convolutional layers, focusing on subtle local dependencies within time series data. Extensive validation on three different real-world datasets highlights the robustness of our approach, demonstrating its proficiency in anomaly detection in time series materials. This underscores the advantage of combining convolutional strategies with Transformer architecture in capturing complex patterns and anomalies.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Hoshi, Y., Park, D., Kemp, C.C.: A multimodal anomaly detector for robot-assisted feeding using an LSTM-based variational autoencoder. IEEE Robot. Autom. Lett. 3, 1544–1551 (2018)
Chung, J., Gulcehre, C., Cho, K.H., Bengio, Y.: Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555 (2014)
Wu, B., et al.: Decompose auto-transformer time series anomaly detection for network management. Electronics 12(2), 354 (2023)
Cleveland, R.B., et al.: STL: a seasonality -trend decomposition. J. Off. Stat 6(1), 3–73 (1990)
Su, Y., et al.: Robust anomaly detection for multivariate time series through stochastic recurrent neural network. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2019)
Audibert, J., et al.: USAD: unsupervised anomaly detection on multivariate time series. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2020)
Deng, A., Hooi, B.: Graph neural network-based anomaly detection in multivariate time series. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 5 (2021)
Zhang, W., Zhang, C., Tsung, F.: Grelen: multivariate time series anomaly detection from the perspective of graph relational learning. In: Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-22, vol. 7 (2022)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Xu, J., et al.: Anomaly transformer: time series anomaly detection with association discrepancy. arXiv preprint arXiv:2110.02642 (2021)
Tuli, S., Casale, G., Jennings, N.R.: Tranad: deep transformer networks for anomaly detection in multivariate time series data. arXiv preprint arXiv:2201.07284 (2022)
Cohen, I., et al.: Pearson correlation coefficient. In: Noise Reduction in Speech Processing, pp. 1–4 (2009)
Zhang, C., et al.: TFAD: a decomposition time series anomaly detection architecture with time-frequency analysis. In: Proceedings of the 31st ACM International Conference on Information and Knowledge Management (2022)
He, X., et al.: OneShotSTL: one-shot seasonality -trend decomposition for online time series anomaly detection and forecasting. arXiv preprint arXiv:2304.01506 (2023)
Ouyang, Z., Jabloun, M., Ravier, P.: STLformer: exploit STL decomposition and rank correlation for time series forecasting. In: 31th European Signal Processing Conference (EUSIPCO) (2023)
Li, S., et al.: Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting. In: Advances in Neural Information Processing Systems, vol. 32 (2019)
Ahmed, C.M., Palleti, V.R., Mathur, A.P.: WADI: a water distribution testbed for research in the design of secure cyber physical systems. In: Proceedings of the 3rd International Workshop on Cyber-Physical Systems for Smart Water Networks (2017)
Mathur, A.P., Tippenhauer, N.O.: SWaT: a water treatment testbed for research and training on ICS security. In: 2016 International Workshop on Cyber-Physical Systems for Smart Water Networks (CySWater). IEEE (2016)
Abdulaal, A., Liu, Z., Lancewicki, T.: Practical approach to asynchronous multivariate time series anomaly detection and localization. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (2021)
Dagum, E.B., Bianconcini, S.: Seasonal Adjustment Methods and Real Time Trend-Cycle Estimation. Springer, Heidelberg (2016). https://doi.org/10.1007/978-3-319-31822-6
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Wu, YX., Dai, BR. (2024). STL-ConvTransformer: Series Decomposition and Convolution-Infused Transformer Architecture in Multivariate Time Series Anomaly Detection. In: Yang, DN., Xie, X., Tseng, V.S., Pei, J., Huang, JW., Lin, J.CW. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2024. Lecture Notes in Computer Science(), vol 14645. Springer, Singapore. https://doi.org/10.1007/978-981-97-2242-6_4
Download citation
DOI: https://doi.org/10.1007/978-981-97-2242-6_4
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-2241-9
Online ISBN: 978-981-97-2242-6
eBook Packages: Computer ScienceComputer Science (R0)