Abstract
The Internet is a dangerous terrain! We hear all the time about websites going down due to denial-of-service attacks or dangerous information on the homepages. Multiple watchwords, dispatch addresses, and credit card data have been blurted into the social sphere in other high-profile cases, vulnerable website druggies to both embarrassment and fiscal threat. Anomaly detection plays a major role in this. It is a step of mining data that detect data points, events, and compliances that diverge from a dataset’s normal geste. Anomalous data indicate critical incidents, similar as specialized glitches, or implicit openings, in case of a change in consumer behavior. Involved dashboards, precisely tuned alert rules and quests through logs, making it a stoner experience. Machine learning has been proposed to improve detection technology for anomalies in recent years, especially in the field of anomalies detection. Here, the authors provide a survey/comparative analysis of published research articles on anomaly detection in correlation with web log analysis. The approach varies, from using traditional machine learning algorithms to two-step algorithms, from neural networks to unsupervised learning and natural language processing to modern methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
W. Xu, L. Huang, A. Fox, D. Patterson, and M. I. Jordan: “Detecting large-scale system problems by mining console logs,” in Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principles, 2009, pp. 117–132. 26. J.-G. Lou, Q. Fu, S. Yang, Y. Xu, and J. Li, “Mining invariants from console logs for system problem detection.” in USENIX Annual Technical Conference, 2010, pp. 1–14. Author, F.: Article title. Journal 2(5), 99–110 (2016).
Bodik P., M. Goldszmidt, A. Fox, D. B. Woodard, and H. Andersen: “Fingerprinting the datacenter: automated classification of performance crises,” in Proceedings of the 5th European conference on Computer systems, 2010, pp. 111–124.
Huang S, Liu Y, Fung C, He R, Zhao Y, Yang H, Luan Z (2020) Hitanomaly: Hierarchical transformers for anomaly detection in system log. IEEE Trans Netw Serv Manage 17(4):2064–2076
Dalal R, Khari M, Anzola JP, García-Díaz V (2021) Proliferation of opportunistic routing: A systematic review. IEEE Access 10:5855–5883
Dalal R., Khari M., “Efficacious implementation of deep Q-routing in opportunistic network”, Soft Computing, 1–19, 2023.
Dalal R., Sangwan A., Khari M.: “The Bibliometrics Assessment of Opportunistic Network Protocols & Simulation Tools”, Telematics and Informatics Reports, 100082, 2023.
Haixuan G., Shuhan Y., Xintao W.: “LogBERT:Log Anomaly Detection via BERT”, March 2021
SAGINDA, Baransel “Log Anomaly Detection using Deep Learning Techniques”, 2020.
Sathya S., Dr.E.Ramaraj, Karaikkudi: “Anomaly Detection in web logs using Big Data Analytics” , 2020 JETIR May 2020, Volume 7, Issue 5.
Meng W., Liu Y., Zhu Y., Shenglin Z., Pei D., Liu Y., Chen Y., Zhang R., Tao S., Sun P., Zhu R.: “Log Anomaly: Unsupervised Detection of sequential and quantitative anomalies in Unstructured Logs”, Twenty-Eight International Joint Conference on Artificial Intelligence, August 2019.
Farzad A., Gulliver TA.: “Log Message Anomaly Detection and Classification Using Auto-B/LSTM and Auto-GRU”, November 2019.
Lu S., Wei X., Li Y., Wang L.: “Detecting Anomaly in Big Data System Logs Using Convolutional Neural Network” , 2018 IEEE 16th Intl Conf on Dependable, Automatic and Secure Computing, 16th Intl Conf on Pervasive Intelligence and Computing, 4th Intl Conf on Big Data Intelligence and Computing and Cyber Science and Technology Congress (DASC/PiCom/DataCom/CyberSciTech), August 2018.
Brown A., Tuor A., Hutchinson B., Nichols N.: “Recurrent Neural Network Attention Mechanisms for Interpretable System Log Anomaly Detection”, First Workshop on Machine Learning for Computer Systems, ACM, HPDC 2018.
Landauer M., Wurzenberger M., Skopik F., Settanni G., Filzmoser P.: “Dynamic log file analysis: An unsupervised cluster evolution approach for anomaly detection”, Volume 79, November 2018, Pages 94–116.
Shi J., He G., Liu X.: “Anomaly Detection for Key Performance Indicators Through Machine Learning”, 2018 International Conference on Network Infrastructure and Digital Content (IC-NIDC)
Xhang X., Xu Y., Lin Q., Qiao B., Zhang H., Dang Y.: “Robust Log-Based Anomaly Detection on Unstable Log Data”,2019 Associationo for Computing Machinery. ACM Page 807–817.
Du M., Li F., Zheng G., Srikumar V.: “DeepLog: Anomaly Detection and Diagnosis from System Logs through Deep Learning”, 2017 ACM.
Cao Q., Qiao Y., Lyu Z.: “Machine learning to detect anomalies in web log analysis”, 2017 3rd IEEE International Conference on Computer and Communications (ICCC), December 2017.
Tharishini M., Ragavinodini M., Senthilkumar R.: “Access Log Anomaly Detection”, 2017 Ninth International Conference on Advanced Computing (ICoAC), December 2017.
Wang L., Cao S., Wan L., Wang F.: “Web Anomaly Detection Vased on Frequent Closed Episode Rules”, 2017 IEEE Trustcom/BigDataSE/ICESS, August 2017.
Zhu M., Huang Z.: “Intrusion detection system based on data mining for host log”, 2017 IEEE 2nd Advanced Information Technology, Electronic and Automation Control Conference (IAEAC), March 2017.
He S., Zhu J., He P., Michael RL.: “Experience Report: System Log Analysis for Anomaly Detection”, 2016 IEEE 27th International Symposium on Software Reliability Engineering (ISSRE), October 2016.
Breier J., Branišová J. (2015) Anomaly Detection from Log Files Using Data Mining Techniques. In: Kim K. (eds) Information Science and Applications. Lecture Notes in Electrical Engineering, vol 339. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-46578-3_53
Hanniel JJ., Tricya E. Widagdo, Yudistra D. W. Asnar: “Information system log visualistaion to monitor anomalous user activity based on time”, 2014 International Coneference on Data and Software Engineering (ICODSE), November 2014.
Sipola T., Juvonen A., Lehtonen J. (2011): Anomaly Detection from Network Logs Using Diffusion Maps. In: Iliadis L., Jayne C. (eds) Engineering Applications of Neural Networks. EANN 2011, AIAI 2011. IFIP Advances in Information and Communication Technology, vol 363. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-23957-1_20
Frei A., Rennhard M.: “Histogram Matrix: Log File Visualisation for Anomaly Detection”, 2008 Third International Conference on Availability, Reliability and Security, May 2008
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Dalal, R., Goel, N., Darbari, R., Chauhan, O., Samal, S., Khari, M. (2024). 0A Comprehensive Review on Anomaly Detection Techniques for Web Data Logging. In: Kaiser, M.S., Xie, J., Rathore, V.S. (eds) Intelligent Strategies for ICT. ICTCS 2023. Lecture Notes in Networks and Systems, vol 941. Springer, Singapore. https://doi.org/10.1007/978-981-97-1260-1_18
Download citation
DOI: https://doi.org/10.1007/978-981-97-1260-1_18
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-1259-5
Online ISBN: 978-981-97-1260-1
eBook Packages: EngineeringEngineering (R0)