Log File Analysis with Context-Free Grammars

  • Gregory Bosman
  • Stefan Gruner
Part of the IFIP Advances in Information and Communication Technology book series (IFIPAICT, volume 410)


Classical intrusion analysis of network log files uses statistical machine learning or regular expressions. Where statistically machine learning methods are not analytically exact, methods based on regular expressions do not reach up very far in Chomsky’s hierarchy of languages. This paper focuses on parsing traces of network traffic using context-free grammars. “Green grammars” are used to describe acceptable log files while “red grammars” are used to represent known intrusion patterns. This technique can complement or augment existing approaches by providing additional precision. Analytically, the technique is also more powerful than existing techniques that use regular expressions.


Intrusion detection log file analysis context-free grammars 


  1. 1.
    S. Axelsson, Intrusion Detection Systems: A Survey and Taxonomy, Technical Report, Department of Computer Science, Chalmers University, Goteborg, Sweden, 2000.Google Scholar
  2. 2.
    S. Gruner and B. Watson, Model-based passive testing of safety-critical components, in Model-Based Testing for Embedded Systems, J. Zander, I. Schieferdecker and P. Mosterman (Eds.), CRC Press, Boca Raton, Florida, pp. 453–483, 2011.Google Scholar
  3. 3.
    R. Harang and P. Guarino, Clustering of Snort alerts to identify patterns and reduce analyst workload, Proceedings of the 2012 Military Communications Conference, 2012.Google Scholar
  4. 4.
    T. Lunt, A survey of intrusion detection techniques, Computers and Security, vol. 12(4), pp. 405–418, 1993.CrossRefGoogle Scholar
  5. 5.
    A. Memon, Log File Categorization and Anomaly Analysis Using Grammar Inference, M.S. Thesis, School of Computing, Queen’s University, Kingston, Canada, 2008.Google Scholar
  6. 6.
    P. Ning and S. Jajodia, Intrusion detection techniques, in The Internet Encyclopedia, Volume 2, H. Bidogli (Ed.), Wiley, Hoboken, New Jersey, pp. 355–367, 2004.Google Scholar
  7. 7.
    M. Olivier and E. Gudes, Wrappers: A mechanism to support state-based authorization in web applications, Data and Knowledge Engineering, vol. 43(3), pp. 281–292, 2002.CrossRefzbMATHGoogle Scholar
  8. 8.
    T. Parr, The Definitive ANTLR Reference: Building Domain-Specific Languages, Pragmatic Bookshelf, Raleigh, North Carolina, 2007.Google Scholar
  9. 9.
    A. Valdez and K. Skinner, Probabilistic alert correlation, Proceedings of the Fourth International Symposium on Recent Advances in Intrusion Detection, pp. 54–68, 2001.CrossRefGoogle Scholar
  10. 10.
    S. Zhang, T. Dean and S. Knight, A lightweight approach to state-based security testing, Proceedings of the Conference of the Center for Advanced Studies on Collaborative Research, article no. 28, 2006. Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2013

Authors and Affiliations

  • Gregory Bosman
    • 1
  • Stefan Gruner
    • 1
  1. 1.Department of Computer ScienceUniversity of PretoriaPretoriaSouth Africa

Personalised recommendations