Advertisement

WebHound: a data-driven intrusion detection from real-world web access logs

  • Te-En WeiEmail author
  • Hahn-Ming Lee
  • Albert B. Jeng
  • Hemank Lamba
  • Christos Faloutsos
Methodologies and Application
  • 46 Downloads

Abstract

Hackers usually discover and exploit vulnerabilities existing in the entry point before invading a corporate environment. The web server exploration and spams are two popular means used by hackers to gain access to the enterprise computer systems. In this paper, we focus on protecting a web server in dealing with such cybersecurity intrusion threat. During the discovery stage, a web vulnerability investigation scanner (e.g., SQLMap, NMap, and Kali) is used by hackers to learn the web server versions and other related vulnerabilities. Then, in the exploitation stage, hackers develop a customized intrusion method which exploits those previously learned vulnerabilities to launch a subsequent attack. Currently, the most popular defense approaches (e.g., IDS, WAF) detect web server intrusion events through domain expert rules and anomaly pattern matches. For example, ModSecurity is an open source WAF which only detects known malware signature by domain expert rules. Thus, those approaches are good to defend the first discovery stage intrusion. However, they are not effective to deal with the customized intrusion in the second exploitation stage since no rules or signatures are available for such kind of intrusion detection. In this paper, in order to resolve the above problem, we propose an unsupervised data-driven anomaly detection known as WebHound. It not only identifies hackers reconnaissance but also detects the customized intrusion means deployed by hackers by analyzing large-scale web access logs. Moreover, WebHoundalso provides intrusion evidence using storyline for recovering intrusion procedure. Among numerous experiments and case studies, we applied WebHoundto a special government case for the intrusion evidence investigation and at the same time, we compared our results with the work done by computer forensic experts. The results showed that WebHoundcould discover more intrusion evidence than human experts. We also compared WebHoundwith ModSecurity which is updated with the newest domain expert rules running in a virtualized corporate environment. The experimental results show that WebHoundhas a better accuracy rate than ModSecurity. In summary, WebHoundalleviates the heavy demand on expert knowledge and human efforts to detect cyber-attack on a web server, and it also enhances detection accuracy and recall rate. Moreover, WebHoundcould provide more evidence for forensic experts to trace the original entry points.

Keywords

Anomaly detection Heterogeneous entity graph Graph mining Tactics techniques and procedures (TTP) 

Notes

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article doesn’t contain any studies with human participates or animals performed by any of the authors.

References

  1. Akoglu L, McGlohon M, Faloutsos C (2010) Oddball: spotting anomalies in weighted graphs. In: Pacific-Asia conference on knowledge discovery and data mining. Springer, pp 410–421Google Scholar
  2. Akoglu L, Tong H, Koutra D (2015) Graph based anomaly detection and description: a survey. Data Min Knowl Discov 29(3):626–688MathSciNetCrossRefGoogle Scholar
  3. Braue D (2015) Cybercrime not “solvable”, requires data-based harm minimisation. http://www.cso.com.au/article/565596/cybercrimeHrB-solvable-requires-data-based-harm-minimisation/HrB. Accessed 5 Feb 2015
  4. Breunig MM, Kriegel H-P, Ng RT, Sander J (2000) LOF: identifying density-based local outliers. In: ACM sigmod record, vol 29. ACM, pp 93–104Google Scholar
  5. Chandola V, Banerjee A, Kumar V (2009) Anomaly detection: a survey. ACM Comput Surv (CSUR) 41(3):15CrossRefGoogle Scholar
  6. Cheng W, Zhang K, Chen H, Jiang G, Chen Z, Wang W (2016) Ranking causal anomalies via temporal and dynamical analysis on vanishing correlations. In: Proceedings of the ACM SIGKDD international conference on knowledge discovery and data mining (SIGKDD)Google Scholar
  7. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297zbMATHGoogle Scholar
  8. Di Mauro M, Di Sarno C (2018) Improving siem capabilities through an enhanced probe for encrypted skype traffic detection. J Inf Secur Appl 38:85–95Google Scholar
  9. Elkan C (2003) Using the triangle inequality to accelerate k-means. In: ICML, pp 147–153Google Scholar
  10. Ge Y, Jiang G, Ding M, Xiong H (2014) Ranking metric anomaly in invariant networks. ACM Trans Knowl Discov Data (TKDD) 8(2):8Google Scholar
  11. Goh V (2016) The anatomy of large-scale cyber attacks. http://www.cso.com.au/article/606694/anatomy-large-scale-cyber-attacks/. Accessed 13 Sept 2016
  12. Golub GH, Reinsch C (1970) Singular value decomposition and least squares solutions. Numerische mathematik 14(5):403–420MathSciNetCrossRefzbMATHGoogle Scholar
  13. Gunestas M, Bilgin Z (2016) Log analysis using temporal logic and reconstruction approach: web server case. J Digit Forensics Secur Law JDFSL 11(2):35Google Scholar
  14. Gyöngyi Z, Garcia-Molina H, Pedersen J (2004) Combating web spam with trustrank. In: Proceedings of the thirtieth international conference on very large data bases, vol 30. VLDB Endowment, pp 576–587Google Scholar
  15. Henderson K, Eliassi-Rad T, Faloutsos C, Akoglu L, Li L, Maruhashi K, Prakash BA, Tong H (2010) Metric forensics: a multi-level approach for mining volatile graphs. In: Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, pp 163–172Google Scholar
  16. Hoplaros D, Tari Z, Khalil I (2014) Data summarization for network traffic monitoring. J Netw Comput Appl 37:194–205CrossRefGoogle Scholar
  17. Jaeger D, Ussath M, Cheng F, Meinel C (2015) Multi-step attack pattern detection on normalized event logs. In: 2015 IEEE 2nd international conference on cyber security and cloud computing (CSCloud). IEEE, pp 390–398Google Scholar
  18. Jiang G, Chen H, Yoshihira K (2006) Modeling and tracking of transaction flow dynamics for fault detection in complex systems. IEEE Trans Dependable Secure Comput 3(4):312–326CrossRefGoogle Scholar
  19. Jiang D, Xu Z, Zhang P, Zhu T (2014) A transform domain-based anomaly detection approach to network-wide traffic. J Netw Comput Appl 40:292–306CrossRefGoogle Scholar
  20. Langville AN, Meyer CD (2005) A survey of eigenvector methods for web information retrieval. SIAM Rev 47(1):135–161MathSciNetCrossRefzbMATHGoogle Scholar
  21. Liao H-J, Lin C-HR, Lin Y-C, Tung K-Y (2013) Intrusion detection system: a comprehensive review. J Netw Comput Appl 36(1):16–24CrossRefGoogle Scholar
  22. Liu C, Yan X, Yu H, Han J, Philip SY (2005) Mining behavior graphs for“ backtrace” of noncrashing bugs. In: SDM. SIAM, pp 286–297Google Scholar
  23. Malkin R (2016) Fighting the growing threat of DDoS attacks down under. http://www.cso.com.au/article/606693/fighting-growing-threat-ddos-attacks-down-under/. Accessed 13 Sept 2016
  24. Manevitz L M, Yousef M (2001) One-class SVMs for document classification. J Mach Learn Res 2:139–154zbMATHGoogle Scholar
  25. Matta V, Di Mauro M, Longo M (2017) Ddos attacks with randomized traffic innovation: botnet identification challenges and strategies. IEEE Trans Inf Forensics Secur 12(8):1844–1859CrossRefGoogle Scholar
  26. Modi C, Patel D, Borisaniya B, Patel H, Patel A, Rajarajan M (2013) A survey of intrusion detection techniques in cloud. J Netw Comput Appl 36(1):42–57CrossRefGoogle Scholar
  27. Modsecurity: open source web application firewall. https://modsecurity.org/
  28. Page L, Brin S, Motwani R, Winograd T (1999) The pagerank citation ranking: bringing order to the web. Technical report, Stanford InfoLabGoogle Scholar
  29. Public security log sharing site. http://log-sharing.dreamhosters.com/
  30. Ray S, Turi RH (1999) Determination of number of clusters in k-means clustering and application in colour image segmentation. In: Proceedings of the 4th international conference on advances in pattern recognition and digital techniques (ICAPRDT’99)Google Scholar
  31. Rossi RA, Gallagher B, Neville J, Henderson K (2013) Modeling dynamic behavior in large evolving graphs. In: Proceedings of the sixth ACM international conference on Web search and data mining. ACM, pp 667–676Google Scholar
  32. Sun J, Tao D, Faloutsos C (2006) Beyond streams and graphs: dynamic tensor analysis. In: Proceedings of the 12th ACM SIGKDD international conference on knowledge discovery and data mining. ACM, pp 374–383Google Scholar
  33. Sun X, Dai J, Liu P, Singhal A, Yen J (2016) Towards probabilistic identification of zero-day attack paths. In: 2016 IEEE conference on communications and network security (CNS). IEEE, pp 64–72Google Scholar
  34. Tao C, Ge Y, Song Q, Ge Y, Omitaomu OA (2014) Metric ranking of invariant networks with belief propagation. In: 2014 IEEE international conference on data mining. IEEE, pp 1001–1006Google Scholar
  35. Tong H, Faloutsos C, Pan J-Y (2006) Fast random walk with restart and its applicationsGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Computer Science and Information EngineeringNational Taiwan University of Science and TechnologyTaipeiTaiwan
  2. 2.CyberTrust Technology InstituteInstitute for Information IndustryTaipeiTaiwan
  3. 3.Institute of Information Science Academia SinicaTaipeiTaiwan
  4. 4.Department of Computer ScienceCarnegie Mellon UniversityPittsburghUSA

Personalised recommendations