Abstract
Anomaly detection is a hot and practical problem. Most of the existing research is based on the model of the generative model, which judges abnormalities by comparing the data errors between original samples and reconstruction samples. Among them, Variational AutoEncoder (VAE) is widely used, but it has the problem of over-generalization. In this paper, we design an unsupervised deep learning anomaly detection method named VESC and propose the recursive reconstruction strategy. VESC adopts the idea of data compression and three structures on the basis of the original VAE, namely spatial constrained network, reformer structure, and re-encoder. The recursive reconstruction strategy can improve the accuracy of the model by increasing the number and typicality of training samples, and it can apply to most unsupervised learning methods. Experimental results of several benchmarks show that our model outperforms state-of-the-art anomaly detection methods. And our proposed strategy can improve the detection results of the original model.
Similar content being viewed by others
References
Onat I, Miri A (2017) An intrusion detection system for wireless sensor networks. In: International conference on telecommunications
Pozzolo AD, Boracchi G, Caelen O, Alippi C, Bontempi G (2018) Credit card fraud detection: a realistic modeling and a novel learning strategy. IEEE Trans Neural Netw Learn Syst 29(8):3784–3797
Schlegl T, Seeböck P, Waldstein SM, Schmidt-Erfurth U, Langs G (2017) Unsupervised anomaly detection with generative adversarial networks to guide marker discovery
Chalapathy R, Chawla S (2019) Deep learning for anomaly detection: a survey. arXiv:1901.03407
Bayer J, Osendorfer C (2015) Learning stochastic recurrent networks
Pham N, Pagh R (2012) A near-linear time approximation algorithm for angle-based outlier detection in high-dimensional data. ACM, pp 877–885
Pham N (2018) L1-depth revisited: a robust angle-based outlier factor in high-dimensional space. In: Joint European conference on machine learning and knowledge discovery in databases. Springer, pp 105–121
Sathe S, Aggarwal CC (2018) Subspace histograms for outlier detection in linear time. Knowl Inf Syst 1–25
Kim C, Lee J, Kim R, Park Y, Kang J (2018) Deepnap: deep neural anomaly pre-detection in a semiconductor fab. Inf Sci 457:002002551830375
Pukelsheim F (1994) The three sigma rule. Am Stat 48(2):88–91
Grubbs FE (1950) Sample criteria for testing outlying observations. Ann Math Stat 21(1):27–58. https://doi.org/10.1214/aoms/1177729885
Munawar A, Vinayavekhin P, De Magistris G (2017) Limiting the reconstruction capability of generative neural network using negative learning. In: 2017 IEEE 27th international workshop on machine learning for signal processing (MLSP), pp 1–6. https://doi.org/10.1109/MLSP.2017.8168155
Gong D, Liu L, Le V, Saha B, Mansour MR, Venkatesh S, Van Den Hengel A (2019) Memorizing normality to detect anomaly: Memory-augmented deep autoencoder for unsupervised anomaly detection. In: 2019 IEEE/CVF international conference on computer vision (ICCV), pp 1705–1714. https://doi.org/10.1109/ICCV.2019.00179
Kieu T, Yang B, Jensen CS (2018) Outlier detection for multidimensional time series using deep neural networks. In: 2018 19th IEEE international conference on mobile data management (MDM), pp 125–134. https://doi.org/10.1109/MDM.2018.00029
Zong B, Song Q, Min MR, Cheng W, Lumezanu C, Cho D, Chen H (2018) Deep autoencoding gaussian mixture model for unsupervised anomaly detection
Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Advances in neural information processing systems, pp 2672–2680
Schlegl T, Seeböck P, Waldstein SM, Langs G, Schmidt-Erfurth U (2019) f-anogan: fast unsupervised anomaly detection with generative adversarial networks. Med Image Anal 54:30–44
Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Ghahramani Z, Welling M, Cortes C, Lawrence N, Weinberger KQ (eds) Advances in neural information processing systems, vol 27. Curran Associates, Inc.
Hicsonmez S, Samet N, Akbas E, Duygulu P (2020) GANILLA: generative adversarial networks for image to illustration translation. CoRR. arXiv:2002.05638
Kingma DP, Welling M (2013) Auto-encoding variational bayes. arXiv:1312.6114
Liu G, Guo C, Xie L, Liu W, Xiong NN, Chen G (2020) An intelligent cnn-vae text representation technology based on text semantics for comprehensive big data. arXiv:abs/2008.12522
Cui Z, Wang J, Bai B, Guo T, Feng Y (2020) G-vae: a continuously variable rate deep image compression framework. arXiv:abs/2003.02012
Lin S, Clark R, Birke R, Schönborn S, Trigoni N, Roberts SJ (2020) Anomaly detection for time series using vae-lstm hybrid model. ICASSP 2020—2020 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 4322–4326
An J, Cho S (2015) Variational autoencoder based anomaly detection using reconstruction probability
Li Z, Chen W, Pei D (2018) Robust and unsupervised kpi anomaly detection based on conditional variational autoencoder, pp 1–9. https://doi.org/10.1109/PCCC.2018.8710885
Tu Z, Bai X (2009) Auto-context and its application to high-level vision tasks and 3d brain image segmentation. IEEE Trans Pattern Anal Mach Intell 32(10):1744–1757
Wang W, Yu K, Hugonot J, Fua P, Salzmann M (2019) Recurrent u-net for resource-constrained segmentation. In: Proceedings of the IEEE international conference on computer vision, pp 2142–2151
Cai Z, Vasconcelos N (2018) Cascade r-cnn: delving into high quality object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6154–6162
Dau HA, Keogh E, Kamgar K, Yeh C-CM, Zhu Y, Gharghabi S, Ratanamahatana CA, Yanping, Hu B, Begum N, Bagnall A, Mueen A, Batista G (2018) The UCR time series classification archive. https://www.cs.ucr.edu/~eamonn/time_series_data_2018/
Dua D, Graff C (2017) UCI machine learning repository. http://archive.ics.uci.edu/ml
Liu FT, Kai MT, Zhou ZH (2009) Isolation forest. In: Eighth IEEE international conference on data mining
Erfani SM, Rajasegarar S, Karunasekera S, Leckie C (2016) High-dimensional and large-scale anomaly detection using a linear one-class svm with deep learning. Pattern Recognit 58:121–134
Zenati H, Romain M, Foo CS, Lecouat B, Chandrasekhar VR (2018) Adversarially learned anomaly detection
An J, Cho S (2015) Variational autoencoder based anomaly detection using reconstruction probability. Special lecture on IE 2(1)
Donahue J, Krähenbühl P, Darrell T (2016) Adversarial feature learning. arXiv:1605.09782
Akcay S, Atapour-Abarghouei A, Breckon TP (2018) Ganomaly: semi-supervised anomaly detection via adversarial training. In: Asian conference on computer vision, pp 622–637
Ling CX, Huang J, Zhang H (2003) Auc: a statistically consistent and more discriminating measure than accuracy. In: International joint conference on artificial intelligence
Akçay S, Atapour-Abarghouei A, Breckon TP (2019) Skip-ganomaly: Skip connected and adversarially trained encoder-decoder anomaly detection. In: 2019 international joint conference on neural networks (IJCNN). IEEE, pp 1–8
Acknowledgements
This work was supported by Natural Science Foundation of Guangdong Province, China (Grant No. 2020A1515010970) and Shenzhen Research Council (Grant Nos. JCYJ20200109113427092, GJHZ20180928155209705).
Author information
Authors and Affiliations
Corresponding authors
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Zhang, C., Wang, X., Zhang, J. et al. VESC: a new variational autoencoder based model for anomaly detection. Int. J. Mach. Learn. & Cyber. 14, 683–696 (2023). https://doi.org/10.1007/s13042-022-01657-w
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13042-022-01657-w