Implementation of a Stream-Based IP Flow Record Query Language

  • Kaloyan Kanev
  • Nikolay Melnikov
  • Jürgen Schönwälder
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6155)

Abstract

Internet traffic analysis via flow records is an important task for network operators. There is a variety of applications, targeted at identifying, filtering or aggregating flows based on certain criteria. Most of these applications exhibit certain limitations when it comes to the identification of complex network activities. To overcome some of these limitations, a new flow query language has been proposed recently, which allows to express complex time relationships between flows. In this paper, we describe a prototype implementation of this query language and we evaluate its performance.

Keywords

Flow Query Language NetFlow Network Monitoring 

References

  1. 1.
    Marinov, V., Schönwälder, J.: Design of a Stream-Based IP Flow Record Query Language. In: DSOM 2009, pp. 15–28. Springer, Heidelberg (2009)Google Scholar
  2. 2.
    Allen, J.F.: Maintaining Knowledge About Temporal Intervals. Communications of the ACM 26(11), 832–843 (1983)CrossRefMATHGoogle Scholar
  3. 3.
    Alted, F., Vilata, I., et al.: PyTables: Hierarchical datasets in Python (2002), http://www.pytables.org/
  4. 4.
    Beazley, D.M.: Ply, python lex-yacc (2001), http://www.dabeaz.com/ply/
  5. 5.
    Folk, M., McGrath, R.E., Yang, K.: Mapping HDF4 Objects to HDF5 Objects. Technical report, National center for supercomputing applications, University of Illinois (2002)Google Scholar
  6. 6.
    Claise, B.: Cisco Systems NetFlow Services Export Version 9. RFC 3954, Cisco Systems (October 2004)Google Scholar
  7. 7.
    Obarski, M.: Profiling python threads (01-02-2010), http://code.activestate.com/recipes/465831/
  8. 8.
    Marinov, V., Schönwälder, J.: Design of an IP Flow Record Query Language. In: Hausheer, D., Schönwälder, J. (eds.) AIMS 2008. LNCS, vol. 5127, pp. 205–210. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  9. 9.
    McCanne, S., Van Jacobson.: The BSD Packet Filter: A New Architecture for User-level Packet Capture. In: USENIX 1993, Berkeley, CA, USA, p. 2. USENIX (1993)Google Scholar
  10. 10.
    Moore, D., Keys, K., Koga, R., Lagache, E., Claffy, K.C.: The CoralReef Software Suite as a Tool for System and Network Administrators. In: LISA 2001, Berkeley, CA, USA, pp. 133–144. USENIX (2001)Google Scholar
  11. 11.
    Romig, S.: The OSU Flow-tools Package and CISCO NetFlow Logs. In: LISA 2000, Berkeley, CA, USA, pp. 291–304. USENIX (2000)Google Scholar
  12. 12.
    Brownlee, N.: SRL: A Language for Describing Traffic Flows and Specifying Actions for Flow Groups. RFC 2723, University of Auckland (October 1999)Google Scholar
  13. 13.
    Plonka, D.: FlowScan: A Network Traffic Flow Reporting and Visualization Tool. In: LISA 2000, Berkeley, CA, USA, pp. 305–318. USENIX (2000)Google Scholar
  14. 14.
    CERT/NetSA at Carnegie Mellon University. SiLK (System for Internet-Level Knowledge), http://tools.netsa.cert.org/silk [Accessed: July 13, 2009]
  15. 15.
    Nickless, B.: Combining Cisco NetFlow Exports with Relational Database Technology for Usage Statistics, Intrusion Detection, and Network Forensics. In: LISA 2000, Berkeley, CA, USA, pp. 285–290. USENIX (2000)Google Scholar
  16. 16.
    Babcock, B., Babu, S., Datar, M., Motwani, R., Widom, J.: Models and Issues in Data Stream Systems. In: PODS 2002, pp. 1–16. ACM, New York (2002)Google Scholar
  17. 17.
    Cranor, C., Johnson, T., Spataschek, O., Shkapenyuk, V.: Gigascope: a Stream Database for Network Applications. In: SIGMOD 2003, pp. 647–651. ACM, New York (2003)Google Scholar
  18. 18.
    Dean, J., Ghemawat, S.: MapReduce: Simplified Data Processing on Large Clusters. In: OSDI 2004, Berkeley, CA, USA, p. 10. USENIX (2004)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2010

Authors and Affiliations

  • Kaloyan Kanev
    • 1
  • Nikolay Melnikov
    • 1
  • Jürgen Schönwälder
    • 1
  1. 1.Computer ScienceJacobs University BremenGermany

Personalised recommendations