Skip to main content
SpringerLink
Log in

Trine: Syslog anomaly detection with three transformer encoders in one generative adversarial network

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

System logs provide powerful support for maintaining system security and stability, but the determination of anomalies often relies on sequence context while hiding in the traces under the massive background normal behavior. Recently transformers have shown remarkable success in feature extraction of long sequences and text classification tasks. Thus, we combine our syslog anomaly detection work with implementing multiple application methods in an integrated model. That is, our proposed generative adversarial network based on three transformer encoders, which is called Trine. One of the encoders is used to extract feature representations of the system logs, while the other two respectively serve as a generator and a discriminator for Generative Adversarial Networks to mitigate the class imbalance of the data. We evaluated Trine on two real-world datasets, HDFS and OpenStack. It shows great competitiveness compared with current state-of-the-art models for syslog anomaly detection. The experimental results demonstrate that the best architecture of our model get an F1-score 0.906, at least 27.8% higher than previous methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Arjovsky M, Chintala S, Bottou L (2017) Wasserstein generative adversarial networks. In: International conference on machine learning. PMLR, pp 214–223

  2. Beltagy I, Peters ME, Cohan A (2020) Longformer: The long-document transformer. arXiv:2004.05150. [cs]

  3. Brown A, Tuor A, Hutchinson B, Nichols N (2018) Recurrent neural network attention mechanisms for interpretable system log anomaly detection. In: Proceedings of the First Workshop on Machine Learning for Computing Systems - MLCS’18. https://doi.org/10.1145/3217871.3217872. ACM Press, USA, pp 1–8

  4. Chalapathy R, Chawla S (2019) Deep Learning for Anomaly Detection: A Survey. arXiv:...... [cs, stat]

  5. Chandola V, Banerjee A, Kumar V (2009) Anomaly detection: A survey. ACM Comput Surv 41(3):1–58. https://doi.org/10.1145/1541880.1541882

    Article  Google Scholar 

  6. Devlin J, Chang MW, Lee K, Toutanova K (2019) BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805. [cs]

  7. Du M, Li F, Zheng G, Srikumar V (2017) Deeplog: Anomaly detection and diagnosis from system logs through deep learning. In: Proceedings of the 2017 ACM SIGSAC conference on computer and communications security. https://doi.org/10.1145/3133956.3134015. ACM, Dallas, pp 1285–1298

  8. Goernitz N, Kloft M, Rieck K, Brefeld U (2013) Toward supervised anomaly detection. J Artif Intell Res 46:235–262. https://doi.org/10.1613/jair.3623

    Article  MathSciNet  Google Scholar 

  9. Goodfellow IJ, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial networks. arXiv:1406.2661. [cs, stat]

  10. Han S, Wu Q, Zhang H, Qin B, Hu J, Shi X, Liu L, Yin X (2021) Log-Based Anomaly detection with robust feature extraction and online learning. IEEE Trans Inform Forens Secur 16:2300–2311. https://doi.org/10.1109/TIFS.2021.3053371

    Article  Google Scholar 

  11. He S, Zhu J, He P, Lyu MR (2020) Loghub: A Large Collection of System Log Datasets towards Automated Log Analytics. arXiv:2008.06448. [cs]

  12. Hochreiter S, Schmidhuber J (1997) Long Short-Term memory. Neural Comput 9(8):1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735

    Article  Google Scholar 

  13. Li D, Chen D, Jin B, Shi L, Goh J, Ng SK (2019) MAD-GAN: Multivariate Anomaly Detection for Time Series Data with Generative Adversarial Networks. In: Tetko IV, Kůrková V, Karpov P, Theis F (eds) Artificial Neural Networks and Machine Learning – ICANN 2019: Text and Time Series, Lecture Notes in Computer Science. https://doi.org/10.1007/978-3-030-30490-4_56. Springer International Publishing, Cham, pp 703–716

  14. Lin Q, Zhang H, Lou JG, Zhang Y, Chen X (2016) Log clustering based problem identification for online service systems. In: Proceedings of the 38th International Conference on Software Engineering Companion - ICSE ’16. https://doi.org/10.1145/2889160.2889232. ACM Press, Texas, pp 102–111

  15. Lu S, Wei X, Li Y, Wang L (2018) Detecting anomaly in big data system logs using convolutional neural network. In: 2018 IEEE 16Th intl conf on dependable, autonomic and secure computing, 16th intl conf on pervasive intelligence and computing, 4th intl conf on big data intelligence and computing and cyber science and technology congress(DASC/picom/datacom/ cyberscitech). https://doi.org/10.1109/DASC/PiCom/DataCom/CyberSciTec.2018.00037. IEEE, Athens, pp 151–158

  16. Meng W, Liu Y, Zhu Y, Zhang S, Pei D, Liu Y, Chen Y, Zhang R, Tao S, Sun P, Zhou R (2019) Loganomaly: Unsupervised detection of sequential and quantitative anomalies in unstructured logs. In: Proceedings of the twenty-eighth international joint conference on artificial intelligence. https://doi.org/10.24963/ijcai.2019/658. International Joint Conferences on Artificial Intelligence Organization, China, pp 4739–4745

  17. Mi H, Wang H, Zhou Y, Lyu MRT, Cai H (2013) Toward Fine-Grained, unsupervised, scalable performance diagnosis for Production Cloud Computing Systems. IEEE Transactions on Parallel and Distributed Systems 24(6):1245–1255. https://doi.org/10.1109/TPDS.2013.21

    Article  Google Scholar 

  18. Mirza M, Osindero S (2014) Conditional Generative Adversarial Nets. arXiv:1411.1784. [cs, stat]

  19. Nanduri A, Sherry L (2016) Anomaly detection in aircraft data using Recurrent Neural Networks (RNN). In: 2016 Integrated communications navigation and surveillance (ICNS). https://doi.org/10.1109/ICNSURV.2016.7486356. IEEE , Herndon, pp 5c2–1–5c2–8

  20. Ngo PC, Winarto AA, Kou CKL, Park S, Akram F, Lee HK (2019) Fence GAN: Towards better anomaly detection. In: 2019 IEEE 31St International Conference on tools with artificial intelligence (ICTAI), pp 141–148. https://doi.org/10.1109/ICTAI.2019.00028

  21. Perera P, Patel VM (2019) Learning deep features for One-Class classification. IEEE Trans Image Process 28(11):5450–5463. https://doi.org/10.1109/TIP.2019.2917862

    Article  MathSciNet  Google Scholar 

  22. Ren R, Cheng J, Yin Y, Zhan J, Wang L, Li J, Luo C (2018) Deep convolutional neural networks for log event classification on distributed cluster systems. In: 2018 IEEE International conference on big data (big data). https://doi.org/10.1109/BigData.2018.8622611. IEEE, Seattle, WA, USA, pp 1639–1646

  23. Schlegl T, Seeböck P, Waldstein SM, Langs G, Schmidt-Erfurth U (2019) F-anoGAN: Fast unsupervised anomaly detection with generative adversarial networks. Med Image Anal 54:30–44. https://doi.org/10.1016/j.media.2019.01.010

    Article  Google Scholar 

  24. Schlegl T, Seeböck P, Waldstein SM, Schmidt-Erfurth U, Langs G (2017) Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery. In: Niethammer M, Styner M, Aylward S, Zhu H, Oguz I, Yap PT, Shen D (eds) Information Processing in Medical Imaging, Lecture Notes in Computer Science. https://doi.org/10.1007/978-3-319-59050-9_12. Springer International Publishing, Cham, pp 146–157

  25. Tuor A, Kaplan S, Hutchinson B, Nichols N, Robinson S (2017) Deep Learning for Unsupervised Insider Threat Detection in Structured Cybersecurity Data Streams. arXiv:1710.00811. [cs, stat]

  26. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention Is All You Need. arXiv:1706.03762. [cs]

  27. Wang S, Li BZ, Khabsa M, Fang H, Ma H (2020) Linformer: Self-Attention with Linear Complexity. arXiv:2006.04768. [cs, stat]

  28. Xia B, Bai Y, Yin J, Li Y, Xu J (2020) LogGAN: A log-level generative adversarial network for anomaly detection using permutation event modeling information systems frontiers. https://doi.org/10.1007/s10796-020-10026-3

  29. Yu L, Zhang W, Wang J, Yu Y (2017) SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient Proceedings of the AAAI Conference on Artificial Intelligence 31(1)

  30. Zenati H, Romain M, Foo C, Lecouat B, Chandrasekhar V (2018) Adversarially learned anomaly detection. In: 2018 IEEE International conference on data mining (ICDM), pp 727–736. https://doi.org/10.1109/ICDM.2018.00088

  31. Zhang D, Zheng Y, Wen Y, Xu Y, Wang J, Yu Y, Meng D (2018) Role-based log analysis applying deep learning for insider threat detection. In: Proceedings of the 1st workshop on security-oriented designs of computer architectures and processors - SecArch’18. https://doi.org/10.1145/3267494.3267495. ACM Press, Canada, pp 18–20

  32. Zhou H, Zhang S, Peng J, Zhang S, Li J, Xiong H, Zhang W (2021) Informer: Beyond efficient transformer for long sequence time-series forecasting. arXiv:2012.07436. [cs]

Download references

Acknowledgements

This work was partially supported by the National Key Research and Development Program of China (Grant no. 2018YFB0804103), the CCF-NSFOCUS KunPeng Research Fund (2020007), the National Natural Science Foundation of China (Grant no. 61902262), the National Defense Innovation Special Zone Program of Science and Technology (Grant no. JG2019055).

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Institute for Cyber Security, University of Electronic Science and Technology of China, Chengdu, Sichuan, 611731, China

    Zhenfei Zhao, Weina Niu, Xiaosong Zhang & Zhenqi Yu

  2. NSFOCUS, Beijing, 100089, China

    Runzi Zhang

  3. School of Cyber Science and Engineering, Sichuan University, Chengdu, 610065, China

    Cheng Huang

Authors
  1. Zhenfei Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Weina Niu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaosong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Runzi Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhenqi Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Cheng Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Weina Niu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, Z., Niu, W., Zhang, X. et al. Trine: Syslog anomaly detection with three transformer encoders in one generative adversarial network. Appl Intell 52, 8810–8819 (2022). https://doi.org/10.1007/s10489-021-02863-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02863-9

Keywords

Search

Navigation